<!--
SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# Instrument an LLM Call

Use this guide when you own the model-provider callback and want NeMo Flow to emit lifecycle events, apply LLM middleware, and preserve the active agent scope around the call.

## What You Build

You will wrap one existing LLM provider invocation with the managed LLM execution API. The result is an LLM call that:

- Receives an LLM request object such as `LLMRequest` in Python or `LlmRequest` in Node.js and Rust.
- Runs LLM request intercepts, guardrails, execution intercepts, and response guardrails.
- Emits LLM start and LLM end events.
- Records model metadata for observability and trajectory export.
- Keeps the LLM span attached to the current agent or request scope.
- Returns the original provider result to the application.

## Before You Start

Complete one binding Quick Start guide first:

- [Python Quick Start](../getting-started/python/index.md)
- [Node.js Quick Start](../getting-started/nodejs.md)
- [Rust Quick Start](../getting-started/rust.md)

Create a scope for the active request or agent run before adding LLM instrumentation. If you have not done that yet, start with [Adding Scopes and Marks](adding-scopes-and-marks.md).

The request and response payloads must be JSON-compatible. If your provider SDK uses clients, streams, callbacks, or other opaque objects, keep those objects in the provider callback and pass only a serializable request projection into NeMo Flow.

## Integration Pattern

Follow these steps to route the provider invocation through NeMo Flow:

1. Identify the stable provider invocation boundary in your application.
2. Create or inherit a scope for the current agent run, request, or workflow.
3. Register a temporary subscriber while validating the integration.
4. Build an LLM request object with provider headers and content.
5. Replace the direct provider invocation with the managed LLM execute helper.
6. Pass the active scope handle and a stable `model_name`.
7. Check that the provider result is unchanged and lifecycle events are emitted.

## Minimal Example

The examples below wrap a demo provider callback and print emitted events.

::::{tab-set}
:sync-group: language

:::{tab-item} Python
:sync: python

```python
import asyncio

import nemo_flow


def log_event(event) -> None:
    print(f"{event.kind} {event.name}")


async def call_provider(request: nemo_flow.LLMRequest):
    return {
        "text": "hello",
        "messages": request.content["messages"],
    }


async def main() -> None:
    nemo_flow.subscribers.register("llm-check", log_event)

    try:
        with nemo_flow.scope.scope("agent-run", nemo_flow.ScopeType.Agent) as handle:
            request = nemo_flow.LLMRequest(
                {},
                {"messages": [{"role": "user", "content": "hello"}]},
            )
            result = await nemo_flow.llm.execute(
                "demo-provider",
                request,
                call_provider,
                handle=handle,
                model_name="demo-model",
            )
            print(result)
    finally:
        nemo_flow.subscribers.deregister("llm-check")


asyncio.run(main())
```

:::

:::{tab-item} Node.js
:sync: node

```js
const {
  LlmRequest,
  ScopeType,
  deregisterSubscriber,
  llmCallExecute,
  registerSubscriber,
  withScope,
} = require("nemo-flow-node");

async function main() {
  registerSubscriber("llm-check", (event) => {
    console.log(`${event.kind} ${event.name}`);
  });

  try {
    await withScope("agent-run", ScopeType.Agent, async (handle) => {
      const request = new LlmRequest(
        {},
        { messages: [{ role: "user", content: "hello" }] },
      );
      const result = await llmCallExecute(
        "demo-provider",
        request,
        async (req) => ({
          text: "hello",
          messages: req.content.messages,
        }),
        handle,
        null,
        null,
        null,
        "demo-model",
      );

      console.log(result);
    });
  } finally {
    deregisterSubscriber("llm-check");
  }
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});
```

:::

:::{tab-item} Rust
:sync: rust

```rust
use nemo_flow::api::llm::{llm_call_execute, LlmCallExecuteParams, LlmRequest};
use nemo_flow::api::scope::{
    self, PopScopeParams, PushScopeParams, ScopeAttributes, ScopeType,
};
use nemo_flow::api::subscriber::{deregister_subscriber, register_subscriber};
use serde_json::json;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    register_subscriber(
        "llm-check",
        Arc::new(|event| {
            println!("{} {}", event.kind(), event.name());
        }),
    )?;

    let handle = scope::push_scope(
        PushScopeParams::builder()
            .name("agent-run")
            .scope_type(ScopeType::Agent)
            .attributes(ScopeAttributes::empty())
            .build(),
    )?;

    let request = LlmRequest {
        headers: Default::default(),
        content: json!({"messages": [{"role": "user", "content": "hello"}]}),
    };
    let result = llm_call_execute(
        LlmCallExecuteParams::builder()
            .name("demo-provider")
            .request(request)
            .func(Arc::new(|req| {
                Box::pin(async move {
                    Ok(json!({
                        "text": "hello",
                        "messages": req.content["messages"]
                    }))
                })
            }))
            .parent(handle.clone())
            .model_name("demo-model")
            .build(),
    )
    .await?;

    println!("{result}");

    scope::pop_scope(PopScopeParams::builder().handle_uuid(&handle.uuid).build())?;
    let _ = deregister_subscriber("llm-check")?;
    Ok(())
}
```

:::

::::

## Validate the Integration

Check both behavior and instrumentation:

- The provider result matches what the application returned before the wrapper was added.
- The subscriber prints an agent or request scope event.
- The subscriber prints LLM start and LLM end events for `demo-provider`.
- LLM start input contains the request after request intercepts and
  sanitize-request guardrails.
- LLM end output contains the provider response after response guardrails.
- The LLM event includes the normalized `model_name` when you provide one.

If only the business result appears, the callback ran but instrumentation did not run. Confirm that the call goes through `llm.execute`, `llmCallExecute`, or `llm_call_execute`.

## Production Checklist

Before deploying to production, ensure the following checklist is completed:

- Keep provider names stable. Subscribers and exporters use names for filtering and dashboards.
- Pass `model_name` separately when the model should be easy to filter or export.
- Keep request and response payloads JSON-compatible.
- Keep SDK clients and transport objects inside the provider callback.
- Use codecs when middleware needs normalized provider request or response semantics.
- Use sanitize guardrails before exporting prompts or model responses in production.

## Common Issues

Check these symptoms first when the workflow does not behave as expected.

- **No LLM events appear**: The application is still calling the provider directly.
- **The LLM appears outside the agent scope**: Pass the current scope handle into the managed execute helper.
- **Middleware sees provider-specific shapes**: Add a codec so request intercepts can work with normalized annotated data.
- **Sensitive prompt data appears in traces**: Add LLM sanitize-request and sanitize-response guardrails before registering production exporters.

## Next Steps

Use these links to continue from this workflow into the next related task.

- Instrument tools with [Instrument a Tool Call](instrument-tool-call.md).
- Add policy or transformation with [Add Middleware](advanced-guide.md).
- Use [Provider Codecs](../integrate-frameworks/provider-codecs.md) when middleware needs normalized LLM request and response data.
- Export events with [Observability](../plugins/observability/about.md).
