---
sidebar_position: 7
---

# Tools (Function Calling)

Some LLMs, in addition to generating text, can also trigger actions.

:::note
All LLMs supporting tools can be found [here](/integrations/language-models) (see the "Tools" column).
:::

:::note
Not all LLMs support tools equally well.
The ability to understand, select, and correctly use tools depends heavily on the specific model and its capabilities.
Some models may not support tools at all, while others might require careful prompt engineering
or additional system instructions.
:::

There is a concept known as "tools," or "function calling".
It allows the LLM to call, when necessary, one or more available tools, usually defined by the developer.
A tool can be anything: a web search, a call to an external API, or the execution of a specific piece of code, etc.
LLMs cannot actually call the tool themselves; instead, they express the intent
to call a specific tool in their response (instead of responding in plain text).
We, as developers, should then execute this tool with the provided arguments and report back
the results of the tool execution.

For example, we know that LLMs themselves are not very good at math.
If your use case involves occasional math calculations, you might want to provide the LLM with a "math tool."
By declaring one or multiple tools in the request to the LLM,
it can then decide to call one of them if it deems it appropriate.
Given a math question along with a set of math tools, the LLM might decide that to properly answer the question,
it should first call one of the provided math tools.

Let's see how this works in practice (with and without tools):

An example of a message exchange without tools:
```
Request:
- messages:
    - UserMessage:
        - text: What is the square root of 475695037565?

Response:
- AiMessage:
    - text: The square root of 475695037565 is approximately 689710.
```
Close, but not correct.

An example of a message exchange with the following tools:
```java
@Tool("Sums 2 given numbers")
double sum(double a, double b) {
    return a + b;
}

@Tool("Returns a square root of a given number")
double squareRoot(double x) {
    return Math.sqrt(x);
}
```

```
Request 1:
- messages:
    - UserMessage:
        - text: What is the square root of 475695037565?
- tools:
    - sum(double a, double b): Sums 2 given numbers
    - squareRoot(double x): Returns a square root of a given number

Response 1:
- AiMessage:
    - toolExecutionRequests:
        - squareRoot(475695037565)


... here we are executing the squareRoot method with the "475695037565" argument and getting "689706.486532" as a result ...


Request 2:
- messages:
    - UserMessage:
        - text: What is the square root of 475695037565?
    - AiMessage:
        - toolExecutionRequests:
            - squareRoot(475695037565)
    - ToolExecutionResultMessage:
        - text: 689706.486532

Response 2:
- AiMessage:
    - text: The square root of 475695037565 is 689706.486532.
```

As you can see, when an LLM has access to tools, it can decide to call one of them when appropriate.

This is a very powerful feature.
In this simple example, we gave the LLM primitive math tools,
but imagine if we gave it, for example, `googleSearch` and `sendEmail` tools
and a query like "My friend wants to know recent news in the AI field. Send the short summary to friend@email.com,"
then it could find recent news using the `googleSearch` tool,
then summarize it and send the summary via email using the `sendEmail` tool.

:::note
To increase the chances of the LLM calling the right tool with the right arguments,
we should provide a clear and unambiguous:
- name of the tool
- description of what the tool does and when it should be used
- description of every tool parameter

A good rule of thumb: if a human can understand the purpose of a tool and how to use it,
chances are that the LLM can too.
:::

LLMs are specifically fine-tuned to detect when to call tools and how to call them.
Some models can even call multiple tools at once, for example,
[OpenAI](https://platform.openai.com/docs/guides/function-calling/parallel-function-calling).

:::note
Please note that not all models support tools.
To see which models support tools, refer to the "Tools" column on [this](https://docs.langchain4j.dev/integrations/language-models/) page.
:::

:::note
Please note that tools/function calling is not the same as [JSON mode](/tutorials/ai-services#json-mode).
:::

# 2 levels of abstraction

LangChain4j provides two levels of abstraction for using tools:
- Low-level, using the `ChatModel` and `ToolSpecification` APIs
- High-level, using [AI Services](/tutorials/ai-services) and `@Tool`-annotated Java methods

## Low Level Tool API

At the low level, you can use the `chat(ChatRequest)` method
of the `ChatModel`. A similar method is also present in the `StreamingChatModel`.

You can specify one or more `ToolSpecification`s when creating the `ChatRequest`.

`ToolSpecification` is an object that contains all the information about the tool:
- The `name` of the tool
- The `description` of the tool
- The `parameters` of the tool and their descriptions
- The `metadata` of the tool.
By default, it is not sent to the LLM provider, you must explicitly specify which metadata keys should be sent
when creating a `ChatModel`.
Currently, tool metadata is supported only by the `langchain4j-anthropic` module.
When tools are provided by the [McpToolProvider](/tutorials/mcp#mcp-tool-provider),
`metadata` can contain MCP-specific entries.

It is recommended to provide as much information about the tool as possible:
a clear name, a comprehensive description, and a description for each parameter, etc.

### Creating Tool Specification

There are two ways to create a `ToolSpecification`:

1. Manually
```java
ToolSpecification toolSpecification = ToolSpecification.builder()
    .name("getWeather")
    .description("Returns the weather forecast for a given city")
    .parameters(JsonObjectSchema.builder()
        .addStringProperty("city", "The city for which the weather forecast should be returned")
        .addEnumProperty("temperatureUnit", List.of("CELSIUS", "FAHRENHEIT"))
        .required("city") // the required properties should be specified explicitly
        .build())
    .build();
```

You can find more information on `JsonObjectSchema` [here](/tutorials/structured-outputs#jsonobjectschema).

2. Using helper methods:
- `ToolSpecifications.toolSpecificationsFrom(Class)`
- `ToolSpecifications.toolSpecificationsFrom(Object)`
- `ToolSpecifications.toolSpecificationFrom(Method)`

```java
class WeatherTools { 
  
    @Tool("Returns the weather forecast for a given city")
    String getWeather(
            @P("The city for which the weather forecast should be returned") String city,
            TemperatureUnit temperatureUnit
    ) {
        ...
    }
}

List<ToolSpecification> toolSpecifications = ToolSpecifications.toolSpecificationsFrom(WeatherTools.class);
```

### JSON Serialization

`ToolSpecification` can be serialized to JSON and deserialized back using `toJson()` and `fromJson()` methods.
This can be useful, for example, when you want to store tool specifications in a database or transfer them over the network.

```java
String json = toolSpecification.toJson();

ToolSpecification deserialized = ToolSpecification.fromJson(json);
```

By default, a dedicated Jackson `ObjectMapper` is used for the JSON conversion.
You can provide your own implementation of `ToolSpecificationJsonCodec` via SPI
by implementing `ToolSpecificationJsonCodecFactory` and registering it
in `META-INF/services/dev.langchain4j.spi.agent.tool.ToolSpecificationJsonCodecFactory`.

### Using `ChatModel`

Once you have a `List<ToolSpecification>`, you can call the model:
```java
ChatRequest request = ChatRequest.builder()
    .messages(UserMessage.from("What will the weather be like in London tomorrow?"))
    .toolSpecifications(toolSpecifications)
    .build();
ChatResponse response = model.chat(request);
AiMessage aiMessage = response.aiMessage();
```

If the LLM decides to call the tool, the returned `AiMessage` will contain data
in the `toolExecutionRequests` field.
In this case, `AiMessage.hasToolExecutionRequests()` will return `true`.
Depending on the LLM, it can contain one or multiple `ToolExecutionRequest` objects
(some LLMs support calling multiple tools in parallel).

Each `ToolExecutionRequest` should contain:
- The `id` of the tool call. Please note that some LLM providers (e.g., Google, Ollama) may omit this ID.
- The `name` of the tool to be called, for example: `getWeather`
- The `arguments`, for example: `{ "city": "London", "temperatureUnit": "CELSIUS" }`

You'll need to manually execute the tool(s) using information from the `ToolExecutionRequest`(s).

If you want to send the result of the tool execution back to the LLM,
you need to create a `ToolExecutionResultMessage` (one for each `ToolExecutionRequest`)
and send it along with all previous messages:
```java

String result = "It is expected to rain in London tomorrow.";
ToolExecutionResultMessage toolExecutionResultMessage = ToolExecutionResultMessage.from(toolExecutionRequest, result);
ChatRequest request2 = ChatRequest.builder()
        .messages(List.of(userMessage, aiMessage, toolExecutionResultMessage))
        .toolSpecifications(toolSpecifications)
        .build();
ChatResponse response2 = model.chat(request2);
```

#### Multimodal Tool Results
`ToolExecutionResultMessage` can also carry non-text content such as images.
Instead of using `text()`, you can use the builder with `contents()`:

```java
ToolExecutionResultMessage toolExecutionResultMessage = ToolExecutionResultMessage.builder()
        .id(toolExecutionRequest.id())
        .toolName(toolExecutionRequest.name())
        .contents(
                TextContent.from("Here is the photo"),
                ImageContent.from(Image.builder()
                        .base64Data(base64Data)
                        .mimeType("image/png")
                        .build())
        )
        .build();
```

:::note
Not all LLM providers support multimodal tool results.
See [Returning Images and Multimodal Content](/tutorials/tools#returning-images-and-multimodal-content) for details on provider support.
:::

### Using `StreamingChatModel`

Once you have a `List<ToolSpecification>`, you can call the model:
```java
ChatRequest request = ChatRequest.builder()
    .messages(UserMessage.from("What will the weather be like in London tomorrow?"))
    .toolSpecifications(toolSpecifications)
    .build();

model.chat(request, new StreamingChatResponseHandler() {

    @Override
    public void onPartialResponse(String partialResponse) {
        System.out.println("onPartialResponse: " + partialResponse);
    }

    @Override
    public void onPartialToolCall(PartialToolCall partialToolCall) {
        System.out.println("onPartialToolCall: " + partialToolCall);
    }

    @Override
    public void onCompleteToolCall(CompleteToolCall completeToolCall) {
        System.out.println("onCompleteToolCall: " + completeToolCall);
    }

    @Override
    public void onCompleteResponse(ChatResponse completeResponse) {
        System.out.println("onCompleteResponse: " + completeResponse);
    }

    @Override
    public void onError(Throwable error) {
        error.printStackTrace();
    }
});
```

If the LLM decides to call the tool, the `onPartialToolCall(PartialToolCall)` callback
will typically be called multiple times before an `onCompleteToolCall(CompleteToolCall)` callback
is eventually called, indicating that streaming for that tool call is finished.

:::note
Please note that not all LLM providers stream partial tool calls.
Some providers (e.g., Bedrock, Google, Mistral, Ollama) return only complete tool calls.
In those cases, `onPartialToolCall` callback won't be invoked - only `onCompleteToolCall` will be called.
:::

Here's an example of what streaming of a single tool call might look like:
```
onPartialToolCall(index = 0, id = "call_abc", name = "get_weather", partialArguments = "{\"")
onPartialToolCall(index = 0, id = "call_abc", name = "get_weather", partialArguments = "city")
onPartialToolCall(index = 0, id = "call_abc", name = "get_weather", partialArguments = ""\":\"")
onPartialToolCall(index = 0, id = "call_abc", name = "get_weather", partialArguments = "London")
onPartialToolCall(index = 0, id = "call_abc", name = "get_weather", partialArguments = "\"}")
onCompleteToolCall(index = 0, id = "call_abc", name = "get_weather", arguments = "{\"city\":\"London\"}")
```

If the LLM initiates multiple tool calls, the `index` will increment, allowing you to correlate
the different `PartialToolCall`s with each other and with the final `CompleteToolCall`.

When complete response streaming is over and `onCompleteResponse(ChatResponse)` is invoked,
the `AiMessage` inside the `ChatResponse` will contain all the tool calls that occurred during streaming.

## High Level Tool API
At a high level of abstraction, you can annotate any Java method with the `@Tool` annotation
and specify them when creating [AI Service](/tutorials/ai-services#tools-function-calling).

AI Service will automatically convert such methods into `ToolSpecification`s
and include them in the request for each interaction with the LLM.
When the LLM decides to call the tool, the AI Service will automatically execute the appropriate method,
and the return value of the method (if any) will be sent back to the LLM.
You can find implementation details in `DefaultToolExecutor`.

A few tool examples:
```java
@Tool("Searches Google for relevant URLs, given the query")
public List<String> searchGoogle(@P("search query") String query) {
    return googleSearchService.search(query);
}

@Tool("Returns the content of a web page, given the URL")
public String getWebPageContent(@P("URL of the page") String url) {
    Document jsoupDocument = Jsoup.connect(url).get();
    return jsoupDocument.body().text();
}
```

### Tool Method Limitations
Methods annotated with `@Tool`:
- can be either static or non-static
- can have any visibility (public, private, etc.).

### Tool Method Parameters
Methods annotated with `@Tool` can accept any number of parameters of various types:
- Primitive types: `int`, `double`, etc
- Object types: `String`, `Integer`, `Double`, etc
- Custom POJOs (can contain nested POJOs)
- `enum`s
- Polymorphic types (sealed interfaces/classes or types annotated with Jackson
  `@JsonSubTypes` / `@JsonTypeInfo`) — see [Polymorphic Tool Parameters](#polymorphic-tool-parameters)
- `List<T>`/`Set<T>` where `T` is one of the above-mentioned types
- `Map<K,V>` (you need to manually specify the types of `K` and `V` in the parameter description with `@P`)

Methods without parameters are supported as well.

#### Parameter Name

By default, if the `name` attribute of `@P` is not specified, the parameter name is obtained via reflection.
However, without the `-parameters` javac option, reflection returns generic names like `arg0`, `arg1`, etc.
The semantic meaning of the parameter is lost, which may confuse the LLM.

Setting `name` in `@P` is useful in two cases:

1. **Missing `-parameters` javac option** — to avoid generic `arg0`/`arg1` names that the LLM would otherwise see.
   Note that frameworks like Quarkus and Spring enable `-parameters` by default,
   so the actual method parameter names are preserved and you typically do not need to set `name` when using those frameworks.
2. **Custom name for the LLM** — when you want the LLM to see a different parameter name than the one in the source code
   (for example, to match a specific API contract or to provide a more descriptive name).

**Example:**

```java
@Tool
void getTemperature(
        @P("Temperature value") double value,
        @P("Unit of temperature") Optional<String> unit) {
    ...
}
```

#### Required and Optional

By default, all tool method parameters are considered **_required_**.
This means the parameter is listed in the JSON schema's `required` array sent to the LLM,
instructing it to produce a value.
A parameter can be made optional by annotating it with `@P(required = false)`:
```java
@Tool
String getTemperature(String location, @P(required = false) Unit unit) {
    ...
}
```

Alternatively, you can declare the parameter as `Optional<T>`:
```java
@Tool
String getTemperature(String location, Optional<Unit> unit) {
    ...
}
```

Fields and sub-fields of complex parameters are also considered **_required_** by default.
You can make a field optional by annotating it with `@JsonProperty(required = false)`:
```java
record User(String name, @JsonProperty(required = false) String email) {}

@Tool
void add(User user) {
    ...
}
```

:::note
Please note that when used with [structured outputs](/tutorials/structured-outputs),
all fields and sub-fields are considered **_optional_** by default.
:::

:::caution Required is advisory: the LLM can still omit arguments
The `required` flag controls the JSON schema sent to the LLM (required parameters are listed
in the schema's `required` array). The LLM is expected to honor this, but in practice it can
disregard the schema and omit an argument anyway.

In LangChain4j 1.x, this is detected only for **primitive** parameters (`int`, `long`, `boolean`, …) — a
missing primitive triggers the `ToolArgumentsErrorHandler` (see [Error Handling](#handling-tool-arguments-errors)
below). **Missing object parameters are not validated**: `null` is passed to the
`@Tool`-annotated method even though the schema marked the parameter as required.

We are planning to remove this asymmetry in LangChain4j 2.0 so that all required arguments are
validated uniformly. If this planned change would affect your use case, please
[open an issue](https://github.com/langchain4j/langchain4j/issues) so we can hear your feedback before it lands.

If you want a real fallback instead of `null` (or instead of an error for primitive parameters), use
[`@P(defaultValue = ...)`](#default-parameter-values).
:::

#### Default Parameter Values

`@P(defaultValue = "...")` declares a value that LangChain4j substitutes when the LLM
omits the argument. It's the simplest way to make a parameter optional and supply a
sensible fallback that your tool method receives.

```java
enum SortBy { RELEVANCE, DATE, RATING }

@Tool
List<Article> searchArticles(
    String query,
    @P(defaultValue = "10") int limit,
    @P(defaultValue = "[\"en\"]") List<String> languages,
    @P(defaultValue = "RELEVANCE") SortBy sortBy
) {
    // When the LLM omits them:
    //   'limit'     -> 10
    //   'languages' -> ["en"]
    //   'sortBy'    -> SortBy.RELEVANCE
}
```

**Setting `defaultValue` implies optional in the JSON schema** — the parameter is *not*
listed in the schema's `required` array, regardless of `@P(required)`. The LLM is told
it may omit the argument; if it does, LangChain4j fills in the default before invoking
your method.

**Supported types:**

| Type                         | Format                   | Example                                  |
|------------------------------|--------------------------|------------------------------------------|
| `String`                     | used verbatim            | `defaultValue = "USD"`                   |
| Primitive / boxed primitive  | type-specific conversion | `"10"`, `"3.14"`, `"true"`               |
| `enum`                       | enum constant name       | `defaultValue = "EUR"`                   |
| `UUID`                       | `UUID.fromString`        | `"550e8400-e29b-41d4-a716-446655440000"` |
| `BigDecimal`, `BigInteger`   | numeric literal          | `"1.5"`, `"100"`                         |
| `List<T>` / `Set<T>` / array | JSON array               | `"[\"a\",\"b\"]"`, `"[1,2,3]"`           |
| `Map<K,V>`                   | JSON object              | `"{\"a\":1,\"b\":2}"`                    |
| POJOs (including nested)     | JSON object              | `"{\"name\":\"Klaus\",\"age\":42}"`      |

The default value string is parsed at AI Service registration time. If it cannot be
converted into the parameter's type, AI Service construction fails immediately with
`IllegalConfigurationException`, naming the offending parameter — typos are caught at
startup rather than on the first LLM call.

**Defaults only apply to absence, not to wrong values.** If the LLM provides an argument
that fails type coercion (e.g. `"banana"` for an `int`), the coercion error propagates as
usual — the default is *not* used as a fallback.

**Defaults are re-parsed on every invocation,** so a tool that mutates a defaulted
`List`/`Map`/POJO does not contaminate later calls:

```java
@Tool
void process(@P(defaultValue = "[\"a\",\"b\"]") List<String> tags) {
    tags.add("processed"); // safe — next invocation still receives ["a","b"]
}
```

**Restrictions** (rejected with `IllegalConfigurationException` at registration time):

- `defaultValue` cannot be combined with `Optional<T>` — `Optional` already encodes
  "absent"; pick one mechanism.
- `defaultValue` cannot be set on LangChain4j-injected parameters (`@ToolMemoryId`,
  `InvocationContext`, etc.) — they don't come from the LLM.

#### Polymorphic Tool Parameters

A tool parameter can be a polymorphic type — a base type whose concrete subtype is decided
by the LLM at call time. Sealed interfaces and sealed classes work without annotations;
plain abstract classes and interfaces must declare their subtypes with Jackson's
`@JsonSubTypes`.
The schema sent to the LLM contains an `anyOf` over the permitted subtypes, each with a
discriminator property (defaulting to `"type"`) so the LLM can communicate which concrete
type it produced; LangChain4j deserializes the LLM's argument into the right subtype
before invoking your tool method.

This works for the polymorphic type as a parameter, for `List<T>` / `Set<T>` of polymorphic
types, and for polymorphic types nested inside another POJO parameter.

**Sealed interfaces and classes — no annotations needed:**

```java
sealed interface Animal permits Dog, Cat {}

record Dog(String name, String breed) implements Animal {}

record Cat(String name, boolean indoor) implements Animal {}

class AnimalRegistry {

    @Tool("Registers a single animal")
    void registerAnimal(Animal animal) { /* dispatched to Dog or Cat */ }

    @Tool("Registers a batch of animals")
    void registerAnimals(List<Animal> animals) { /* mixed Dog / Cat */ }

    @Tool("Registers an owner with their pet")
    void registerOwner(Owner owner) { /* Owner.pet is dispatched */ }
}

record Owner(String name, Animal pet) {}
```

**Jackson `@JsonSubTypes` / `@JsonTypeInfo`** are also supported and let you decouple wire
names from Java class names:

```java
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "kind")
@JsonSubTypes({
    @JsonSubTypes.Type(value = Square.class, name = "square"),
    @JsonSubTypes.Type(value = Circle.class, name = "circle")
})
interface Shape {}

class ShapeRegistry {

    @Tool("Registers a shape")
    void registerShape(Shape shape) { /* dispatched to Square or Circle */ }
}
```

The set of supported `@JsonTypeInfo` options, the discriminator-name resolution order,
`defaultImpl` behavior, the `visible` flag, and field-collision detection are described in
detail in [Polymorphic Types](/tutorials/structured-outputs#polymorphic-types) under
Structured Outputs — they apply identically to tool parameters.

#### Recursive Parameters

Recursive parameters (e.g., a `Person` class having a `Set<Person> children` field)
are currently supported only by OpenAI.

### Tool Method Return Types
Methods annotated with `@Tool` can return any type, including `void`.
If the method has a `void` return type, "Success" string is sent to the LLM if the method returns successfully.

If the method has a `String` return type, the returned value is sent to the LLM as is, without any conversions.

For other return types, the returned value is converted into a JSON string before being sent to the LLM.

#### Returning Images and Multimodal Content

Tools can also return images and other non-text content. When a tool returns one of the following types,
the result is sent to the LLM as multimodal content (e.g., image) instead of being serialized to JSON text:

- `Image` — sent as a single image
- `ImageContent` — sent as a single image content
- `Content` — sent as a single content element (e.g., `TextContent`, `ImageContent`)
- `List<Content>` — sent as multiple content elements
- `Content[]` — sent as multiple content elements

For example, a tool that takes a photo and returns an image:
```java
@Tool("Takes a photo and returns it")
Image takePhoto() {
    byte[] imageBytes = camera.capture();
    return Image.builder()
            .base64Data(Base64.getEncoder().encodeToString(imageBytes))
            .mimeType("image/png")
            .build();
}
```

Or a tool that returns both text and an image:
```java
@Tool("Takes a photo and returns it with a description")
List<Content> takePhoto() {
    Image image = camera.capture();
    return List.of(
            TextContent.from("Photo taken at " + LocalDateTime.now()),
            ImageContent.from(image)
    );
}
```

:::note
Not all LLM providers support multimodal tool results.
Providers that currently support images in tool results include Anthropic, Amazon Bedrock, and Google AI Gemini.
Other providers will throw an `UnsupportedFeatureException` if a tool returns non-text content.
:::

### AI services as tools for other AI services

AI services can also be used as tools for other AI services. This can be useful in many agentic use cases, where one AI service can ask the help of another, more specialized, AI service to perform a specific task. For instance, having defined the following AI services:

```java
    interface RouterAgent {

        @dev.langchain4j.service.UserMessage("""
            Analyze the following user request and categorize it as 'legal', 'medical' or 'technical',
            then forward the request as it is to the corresponding expert provided as a tool.
            Finally return the answer that you received from the expert without any modification.

            The user request is: '{{it}}'.
            """)
        String askToExpert(String request);
    }

    interface MedicalExpert {

        @dev.langchain4j.service.UserMessage("""
            You are a medical expert.
            Analyze the following user request under a medical point of view and provide the best possible answer.
            The user request is {{it}}.
            """)
        @Tool("A medical expert")
        String medicalRequest(String request);
    }

    interface LegalExpert {

        @dev.langchain4j.service.UserMessage("""
            You are a legal expert.
            Analyze the following user request under a legal point of view and provide the best possible answer.
            The user request is {{it}}.
            """)
        @Tool("A legal expert")
        String legalRequest(String request);
    }

    interface TechnicalExpert {

        @dev.langchain4j.service.UserMessage("""
            You are a technical expert.
            Analyze the following user request under a technical point of view and provide the best possible answer.
            The user request is {{it}}.
            """)
        @Tool("A technical expert")
        String technicalRequest(String request);
    }
```

The `RouterAgent` can be configured to use as tools the 3 other AI services, experts in specific fields, routing the user request to one of them.

```java
MedicalExpert medicalExpert = AiServices.builder(MedicalExpert.class)
        .chatModel(model)
        .build();
LegalExpert legalExpert = AiServices.builder(LegalExpert.class)
        .chatModel(model)
        .build();
TechnicalExpert technicalExpert = AiServices.builder(TechnicalExpert.class)
        .chatModel(model)
        .build();

RouterAgent routerAgent = AiServices.builder(RouterAgent.class)
        .chatModel(model)
        .tools(medicalExpert, legalExpert, technicalExpert)
        .build();

routerAgent.askToExpert("I broke my leg what should I do");
```

:::note
Using AI services as tools for other AI services is a powerful feature that enables to build complex agentic systems. However, this approach also comes with a few relevant drawbacks that are important to be aware of:
- This implementation requires the LLM to copy-paste the user request without modifications as a tool call and this could be an error-prone operation.
- The LLM calling the other LLM as a tool has to reprocess its response, as it happens for any other tool invocation, and this could be a wasteful computation in terms of both time and consumed tokens.
- The agent-tool, being a totally separated AI service, has no access to the chat memory of the agent calling it, so it cannot use the chat memory to provide a more informed answer.
:::


### `@Tool`
Any Java method annotated with `@Tool`
and _explicitly_ specified during the build of an AI Service can be executed by the LLM:
```java
interface MathGenius {
    
    String ask(String question);
}

class Calculator {
    
    @Tool
    double add(int a, int b) {
        return a + b;
    }

    @Tool
    double squareRoot(double x) {
        return Math.sqrt(x);
    }
}

MathGenius mathGenius = AiServices.builder(MathGenius.class)
    .chatModel(model)
    .tools(new Calculator())
    .build();

String answer = mathGenius.ask("What is the square root of 475695037565?");

System.out.println(answer); // The square root of 475695037565 is 689706.486532.
```

When the `ask` method is called, 2 interactions with the LLM occur, as described in the earlier section.
In between those interactions, the `squareRoot` method is called automatically.

The `@Tool` annotation has these fields:
- `name`: the tool's name. If this is not provided, the method's name will serve as the tool's name.
- `value`: the tool's description.
- `returnBehavior`: see [this](/tutorials/tools#returning-immediately-the-result-of-a-tool-execution-request) for more details
- `metadata`: a valid JSON string that contains LLM-provider-specific tool metadata entries.
By default, it is not sent to the LLM provider, you must explicitly specify which metadata keys should be sent
when creating a `ChatModel`.
Currently, tool metadata is supported only by the `langchain4j-anthropic` module.

Depending on the tool, the LLM might understand it well even without any description
(for example, `add(a, b)` is obvious),
but it is usually better to provide clear and meaningful names and descriptions.
This way, the LLM has more information to decide whether or not to call the given tool, and how to do so.

### `@P`
Method parameters can optionally be annotated with `@P`.

The `@P` annotation has the following optional fields:

- `name`: name of the parameter as seen by the LLM. If not specified, the actual method parameter name is used.
- `description`: description of the parameter (alias of `value`). Empty by default.
- `value`: description of the parameter (alias of `description`). Empty by default.
- `required`: whether the parameter is required, default is `true`.

#### Parameter Name

The `name` attribute overrides the parameter name that the LLM will see.
Setting `name` is useful in two cases:

1. **Missing `-parameters` javac option.**
   Without the `-parameters` javac option, Java reflection returns generic names such as `arg0`, `arg1`, etc.
   The semantic meaning of the parameter is lost, which may confuse the LLM.
   Setting `name` restores a meaningful name.
   Note that frameworks like Quarkus and Spring enable `-parameters` by default,
   so the actual method parameter names are preserved and you typically do not need to set `name` when using those frameworks.

2. **Custom name for the LLM.**
   When you want the LLM to see a different parameter name than the one the developer uses in the source code
   (for example, to match a specific API contract or to provide a more descriptive name).


#### Parameter Description

`description` and `value` are interchangeable — they both set the parameter's description that the LLM will see.
When only a description is needed, use the shorthand `value` form:
```java
@Tool
void getWeather(@P("The city name") String city) { ... }
```

When both a name and a description are needed, use named attributes:
```java
@Tool
void getWeather(@P(name = "city", description = "The city name") String city) { ... }
```

### `@Description`
The description of classes and fields can be specified using the `@Description` annotation:

```java
@Description("Query to execute")
class Query {

  @Description("Fields to select")
  private List<String> select;

  @Description("Conditions to filter on")
  private List<Condition> where;
}

@Tool
Result executeQuery(Query query) {
  ...
}
```

:::note
Please note that `@Description` placed on an `enum` value has **_no effect_** and **_is not_** included
in the generated JSON schema:
```java
enum Priority {

    @Description("Critical issues such as payment gateway failures or security breaches.") // this is ignored
    CRITICAL,
    
    @Description("High-priority issues like major feature malfunctions or widespread outages.") // this is ignored
    HIGH,
    
    @Description("Low-priority issues such as minor bugs or cosmetic problems.") // this is ignored
    LOW
}
```
:::

### `InvocationParameters`
If you wish to pass extra data into the tool when invoking AI Service, you can do it with
`InvocationParameters`:
```java

interface Assistant {
    String chat(@UserMessage String userMessage, InvocationParameters parameters);
}

class Tools {
    @Tool
    String getWeather(String city, InvocationParameters parameters) {
        String userId = parameters.get("userId");
        UserPreferences preferences = getUserPreferences(userId);
        return weatherService.getWeather(city, preferences.temperatureUnits());
    }
}

InvocationParameters parameters = InvocationParameters.from(Map.of("userId", "12345"));
String response = assistant.chat("What is the weather in London?", parameters);
```

In this case, the LLM is not aware of these parameters;
they are only visible to LangChain4j and user code.

`InvocationParameters` can also be accessed within other AI Service components, such as:
- [`ToolProvider`](/tutorials/tools#specifying-tools-dynamically): inside the `ToolProviderRequest`
- [`ToolArgumentsErrorHandler`](/tutorials/tools#handling-tool-arguments-errors)
and [`ToolExecutionErrorHandler`](https://docs.langchain4j.dev/tutorials/tools#handling-tool-execution-errors):
inside the `ToolErrorContext`
- [RAG components](/tutorials/rag/): inside the `Query` -> `Metadata`

Parameters are stored in a mutable, thread safe `Map`.

Data can be passed between AI Service components inside the `InvocationParameters`
(for example, from one tool to another or from a RAG component to a tool)
during a single invocation of the AI Service.

### `InvocationContext`

Similarly to `InvocationParameters`, `@Tool`-annotated methods
can accept `InvocationContext` parameter to get access to the information
about AI Service invocation.

```java
class Tools {
    @Tool
    String getWeather(String city, InvocationContext context) {
        UUID invocationId = context.invocationId();
        String aiServiceInterfaceName = context.interfaceName();
        ...
    }
}
```

In this case, the LLM is not aware of these parameters;
they are only visible to LangChain4j and user code.

### `@ToolMemoryId`
If your AI Service method has a parameter annotated with `@MemoryId`,
you can also annotate a parameter of a `@Tool` method with `@ToolMemoryId`:

```java
interface Assistant{
    String chat(@UserMessage String userMessage, @MemoryId memoryId);
}

class Tools {
    @Tool
    String addCalendarEvent(CalendarEvent event, @ToolMemoryId memoryId) {
        ...
    }
}

String answer = assistant.chat("Tomorrow I will have a meeting with Klaus at 14:00", "12345");
```

The value provided to the AI Service method will be automatically passed to the `@Tool` method.
This feature is useful if you have multiple users and/or multiple chats/memories per user
and wish to distinguish between them inside the `@Tool` method.

### Executing Tools Concurrently

By default, when the LLM calls **_multiple_** tools at once (also known as parallel tool calling),
the AI Service executes them sequentially. If you want the tools to be executed concurrently,
you can call `executeToolsConcurrently()` or `executeToolsConcurrently(Executor)` when building the AI Service.
If you enable one of these options, the tools will be executed concurrently (with one exception - see below),
using either the default or the specified `Executor`.

#### When using `ChatModel`:
- When the LLM calls multiple tools, they are executed concurrently in separate threads
using the `Executor`.
- When the LLM calls a single tool, it is executed in the same (caller) thread,
the `Executor` is **_not_** used to avoid wasting resources.

#### When using `StreamingChatModel`:
- When the LLM calls multiple tools, they are executed concurrently in separate threads
using the `Executor`.
Each tool is executed as soon as `StreamingChatResponseHandler.onCompleteToolCall(CompleteToolCall)`
is called, without waiting for other tools or for the response streaming to complete.
- When the LLM calls a single tool, it is executed in a separate thread using the `Executor`.
We cannot execute it in the same thread because, at that point,
we do not yet know how many tools the LLM will call.

### Accessing Executed Tools
If you wish to access tools executed during the invocation of an AI Service,
you can easily do so by wrapping the return type in the `Result` class:
```java
interface Assistant {

    Result<String> chat(String userMessage);
}

Result<String> result = assistant.chat("Cancel my booking 123-456");

String answer = result.content();
List<ToolExecution> toolExecutions = result.toolExecutions();

ToolExecution toolExecution = toolExecutions.get(0);
ToolExecutionRequest request = toolExecution.request();
String result = toolExecution.result(); // tool execution result as text
List<Content> resultContents = toolExecution.resultContents(); // tool execution result as content list (may include images)
Object resultObject = toolExecution.resultObject(); // actual value returned by the tool
```

In streaming mode, you can do so by specifying `onToolExecuted` callback:
```java
interface Assistant {

    TokenStream chat(String message);
}

TokenStream tokenStream = assistant.chat("Cancel my booking");

tokenStream
    .onToolExecuted((ToolExecution toolExecution) -> System.out.println(toolExecution))
    .onPartialResponse(...)
    .onCompleteResponse(...)
    .onError(...)
    .start();
```

### Specifying Tools Programmatically

When using AI Services, tools can also be specified programmatically.
This approach offers a lot of flexibility, as tools can be loaded
from external sources such as databases and configuration files.

Tool names, descriptions, parameter names, and descriptions
can all be configured using `ToolSpecification`:
```java
ToolSpecification toolSpecification = ToolSpecification.builder()
        .name("get_booking_details")
        .description("Returns booking details")
        .parameters(JsonObjectSchema.builder()
                .properties(Map.of(
                        "bookingNumber", JsonStringSchema.builder()
                                .description("Booking number in B-12345 format")
                                .build()
                ))
                .build())
        .build();
```

For each `ToolSpecification`, one needs to provide a `ToolExecutor` implementation
that will be handling tool execution requests generated by the LLM:
```java
ToolExecutor toolExecutor = (toolExecutionRequest, memoryId) -> {
    Map<String, Object> arguments = fromJson(toolExecutionRequest.arguments());
    String bookingNumber = arguments.get("bookingNumber").toString();
    Booking booking = getBooking(bookingNumber);
    return booking.toString();
};
```

LangChain4j also provides the `DefaultToolExecutor`, which can automatically invoke methods on Java objects and handle
parameter mapping:
```java
class BookingTools {
    String getBookingDetails(String bookingNumber) {
        Booking booking = loadBookingFromDatabase(bookingNumber);
        return booking.toString();
    }
}

BookingTools tools = new BookingTools();
Method method = BookingTools.class.getMethod("getBookingDetails", String.class);
ToolExecutor toolExecutor = new DefaultToolExecutor(tools, method);
```

Once we have one or multiple (`ToolSpecification`, `ToolExecutor`) pairs,
we wrap each pair in an `AiServiceTool` and pass the list to the AI Service:
```java
AiServiceTool tool = AiServiceTool.builder()
        .toolSpecification(toolSpecification)
        .toolExecutor(toolExecutor)
        .build();

Assistant assistant = AiServices.builder(Assistant.class)
    .chatModel(chatModel)
    .tools(List.of(tool))
    .build();
```

#### Configuring Immediate Return for Programmatic Tools

In case you need to specify
[immediate return behavior](/tutorials/tools#returning-immediately-the-result-of-a-tool-execution-request)
for a tool, set its `ReturnBehavior` on the `AiServiceTool` builder:

```java
AiServiceTool bookingTool = AiServiceTool.builder()
        .toolSpecification(bookingToolSpec)
        .toolExecutor(bookingExecutor)
        .returnBehavior(IMMEDIATE)
        .build();

AiServiceTool closeTool = AiServiceTool.builder()
        .toolSpecification(closeToolSpec)
        .toolExecutor(closeExecutor)
        .returnBehavior(IMMEDIATE_IF_LAST)
        .build();

AiServiceTool weatherTool = AiServiceTool.builder()
        .toolSpecification(weatherToolSpec)
        .toolExecutor(weatherExecutor)
        // ReturnBehavior.TO_LLM by default
        .build();

Assistant assistant = AiServices.builder(Assistant.class)
    .chatModel(chatModel)
    .tools(List.of(bookingTool, closeTool, weatherTool))
    .build();
```

### Specifying Tools Dynamically

When using AI services, tools can also be specified dynamically for each invocation.
One can configure a `ToolProvider` that will be called each time the AI service is invoked
and will provide the tools that should be included in the current request to the LLM.
The `ToolProvider` accepts a `ToolProviderRequest`
(that contains the `UserMessage`, chat memory ID and [`InvocationParameters`](/tutorials/tools#invocationparameters))
and returns a `ToolProviderResult` that contains tools for the current AI Service invocation.

Here is an example of how to add the `get_booking_details` tool only when the user's message contains the word "booking":
```java
ToolProvider toolProvider = (toolProviderRequest) -> {
    if (toolProviderRequest.userMessage().singleText().contains("booking")) {
        ToolSpecification toolSpecification = ToolSpecification.builder()
            .name("get_booking_details")
            .description("Returns booking details")
            .parameters(JsonObjectSchema.builder()
                .addStringProperty("bookingNumber")
                .build())
            .build();
        return ToolProviderResult.builder()
            .add(toolSpecification, toolExecutor)
            .build();
    } else {
        return null;
    }
};

Assistant assistant = AiServices.builder(Assistant.class)
    .chatModel(model)
    .toolProvider(toolProvider)
    .build();
```

It is possible to mix statically (both `@Tool`-annotated methods and tools configured programmatically)
and dynamically specified tools in the same AI Service invocation.
In this case all static and dynamic tools are merged together.

#### Configuring Immediate Return for Dynamic Tools

When building `ToolProviderResult`, you can mark tools for
[immediate return](/tutorials/tools#returning-immediately-the-result-of-a-tool-execution-request)
using `ToolProviderResult.builder()`. The `add(ToolSpecification, ToolExecutor, ReturnBehavior)` overload
accepts any of `TO_LLM`, `IMMEDIATE`, or `IMMEDIATE_IF_LAST`:

```java
ToolProvider toolProvider = (toolProviderRequest) -> {
    return ToolProviderResult.builder()
        .add(bookingToolSpec, bookingExecutor, ReturnBehavior.IMMEDIATE)
        .add(closeToolSpec, closeExecutor, ReturnBehavior.IMMEDIATE_IF_LAST)
        .add(weatherToolSpec, weatherExecutor) // ReturnBehavior.TO_LLM by default
        .build();
};
```

### Tool Search

When working with a large number of tools,
sending all tools on every request can significantly increase token usage and reduce model performance.
To address this, LangChain4j provides a tool search mechanism
that allows tools to be discovered dynamically by the LLM itself,
instead of being exposed upfront.

The core idea is simple:
- Initially, the LLM is exposed to one or more special tool-search tools
- The LLM can call these tools to search for relevant tools
- Once relevant tools are found, they are included in subsequent requests to the LLM

This enables scalable, token-efficient, and model-driven tool discovery.

#### How Tool Search Works

A tool search flow typically looks like this:
1. Initial request:
   - The LLM sees only tool-search tools (not the full tool set)
2. Tool search
   - The LLM calls a tool-search tool, describing what kind of tool it needs
   - The tool-search strategy matches the request against available tools
4. Tool exposure
   - Matching tools are added to the next request to the LLM
5. Tool execution
   - The LLM can now call the found tools normally

Previously found tools are accumulated across multiple tool-search calls.
Each time the LLM invokes the tool-search tool,
the newly matched tools are added to the existing set of tools visible to the LLM (they are merged, not replaced).
This means the list of tools visible to the LLM can grow over time.
Found tools remain visible to the LLM until their corresponding `ToolExecutionResultMessage`
is evicted from the `ChatMemory`, and at least until the end of the AI Service invocation.

If `ChatMemory` is not configured, the found tools remain visible to the LLM
only until the end of the AI service invocation.

#### ToolSearchStrategy

Tool search is implemented via the `ToolSearchStrategy` interface:

```java
@Experimental
public interface ToolSearchStrategy {

    List<ToolSpecification> getToolSearchTools(InvocationContext invocationContext);

    ToolSearchResult search(ToolSearchRequest toolSearchRequest);
}
```

A `ToolSearchStrategy` is responsible for:
- Exposing tool-search tools to the LLM
- Executing tool search requests generated by the LLM
- Returning matching tool names, which will then be resolved and exposed

LangChain4j currently provides 2 out-of-the-box implementations:
- `SimpleToolSearchStrategy` – keyword-based matching
- `VectorToolSearchStrategy` – semantic search using embeddings

See Javadoc of these classes for more details.

You can also implement custom strategies.

#### Configuring Tool Search in AI Services

Tool search is configured at the AI Service level:

```java
Assistant assistant = AiServices.builder(Assistant.class)
    .chatModel(chatModel)
    .chatMemory(chatMemory)
    .tools(tools) // tool search works for static tools
    .toolProvider(mcpToolProvider) // tool search works for tools provided dynamically (e.g., MCP)
    .toolSearchStrategy(new SimpleToolSearchStrategy())
    .build();
```

Once configured:
- The LLM no longer sees all tools upfront
- Tool discovery becomes an explicit, model-driven step
- Token usage is reduced, especially with large tool sets

#### When to Use Tool Search

Tool search is especially useful when:
- You have many tools (dozens or hundreds)
- Tools are domain-specific or rarely used
- Tool availability depends on context, user, or permissions
- You want the LLM to reason about which tools it needs, instead of guessing from a long list

If you have only a small number of tools, or all tools are always relevant,
using regular approach may be simpler.

#### Always Visible Tools

When tool search is enabled, tools are normally hidden from the LLM until they are discovered via a tool-search call.
However, in some cases you may want certain tools to always be visible to the LLM.

Typical use cases:
- Core tools that should always be accessible
- Frequently used tools where search overhead is unnecessary
- Utility tools

LangChain4j supports this via the `ALWAYS_VISIBLE` tool search behavior.

##### How It Works

When a tool is marked as `ALWAYS_VISIBLE`:
- It is exposed to the LLM in the very first request
- It does not require discovery via tool search
- It remains visible throughout the AI Service invocation
- It is not included in searchable tool candidates

All other tools continue to follow the normal tool-search flow.

##### Using `@Tool` Annotation

You can mark a tool as always visible via the @Tool annotation:
```java
@Tool(searchBehavior = ALWAYS_VISIBLE)
String getWeather(String city) {
    return weatherService.getWeather(city);
}
```

##### Using `McpToolProvider`

When using MCP tools (via `McpToolProvider`), always-visible tools can be configured via `alwaysVisibleToolNames`:

```java
McpToolProvider.builder()
    .mcpClients(mcpClient)
    .alwaysVisibleToolNames("getWeather")
    .build();
```

##### Using `ToolSpecification`

If you configure tools programmatically, you can mark them as always visible using `metadata`:
```java
ToolSpecification toolSpecification = ToolSpecification.builder()
    .name("getWeather")
    .parameters(JsonObjectSchema.builder()
        .addStringProperty("city")
        .required("city")
        .build())
    .metadata(Map.of(ToolSpecification.METADATA_SEARCH_BEHAVIOR, SearchBehavior.ALWAYS_VISIBLE))
    .build();
```

#### Notes and Limitations

:::note
Tool search relies on the LLM’s ability to understand when and how to search for tools.
The effectiveness of this feature depends heavily on the chosen model.
:::

:::note
Tool search is currently marked as experimental and may evolve in future releases.
:::

### Returning immediately the result of a tool execution request

By default, the result of a tool execution request is sent back to the LLM
that uses this result and further reprocesses it. However, in some circumstances,
the result produced by that tool execution request already represents the expected result
of the AI service invocation. In this case it is possible to configure the tool to
immediately/directly return its result, skipping a wasteful and resource consuming
reprocessing by the LLM. This can be done by configuring the `returnBehavior` field of the
`@Tool` annotation as in the following example:

```java
class CalculatorWithImmediateReturn {
    
    @Tool(returnBehavior = ReturnBehavior.IMMEDIATE)
    double add(int a, int b) {
        return a + b;
    }
}
```

:::note
This feature is supported only on AI Services having a `Result<T>` return type.
Attempting to use it on AI Service with a different return type will produce an `IllegalConfigurationException`.
See [Return Types](/tutorials/ai-services#return-types) for more information about `Result<T>`.
:::

In this way, an `Assistant` service like the following

```java
interface Assistant {
    Result<String> chat(String userMessage);
}
```

configured to use the above `CalculatorWithImmediateReturn` tool

```java
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(model)
        .tools(new CalculatorWithImmediateReturn())
        .build();
```

will return a response directly from the tool invocation. For instance, prompting the assistant with

```java
Result<String> result = assistant.chat("How much is 37 plus 87?");
```

will produce a `Result` with a `Result.content() == null`,
while the actual response of `124` will have to be retrieved from the `result.toolExecutions()`.
Without the immediate return, the LLM would have to reprocess the result of the `add` tool execution request,
thus returning a response like: `The result of adding 37 and 87 is 124.`

#### Immediate-return rule with multiple tool calls in a single response

When the LLM returns multiple tool calls in a single response,
the loop returns immediately (without sending the results back to the LLM) only when **both** of the following hold:

1. **No tool errored.** Any error in any tool call forces reprocessing so the LLM can react to the error on the next turn.
2. **Either** every tool in the response is `IMMEDIATE` (no `TO_LLM` tools mixed in) **or** the last tool is `IMMEDIATE_IF_LAST` (see next section).

Examples (tools listed in the order returned by the LLM in a single response;
`(err)` marks a tool whose execution errored):

| Tool calls in the response                     | Outcome            | Why                                                          |
|------------------------------------------------|--------------------|--------------------------------------------------------------|
| `[IMMEDIATE]`                                  | return immediately | only tool is `IMMEDIATE`                                     |
| `[IMMEDIATE, IMMEDIATE]`                       | return immediately | every tool is `IMMEDIATE`/`IMMEDIATE_IF_LAST`                |
| `[IMMEDIATE, TO_LLM]`                          | reprocess          | `TO_LLM` disqualifies the all-immediate rule                 |
| `[IMMEDIATE_IF_LAST]`                          | return immediately | last tool is `IMMEDIATE_IF_LAST`                             |
| `[TO_LLM, IMMEDIATE_IF_LAST]`                  | return immediately | last tool is `IMMEDIATE_IF_LAST`                             |
| `[IMMEDIATE_IF_LAST, TO_LLM]`                  | reprocess          | not last, and `TO_LLM` disqualifies the all-immediate rule   |
| `[IMMEDIATE, IMMEDIATE_IF_LAST]`               | return immediately | last tool is `IMMEDIATE_IF_LAST`                             |
| `[IMMEDIATE_IF_LAST, IMMEDIATE]`               | return immediately | every tool is `IMMEDIATE`/`IMMEDIATE_IF_LAST`                |
| `[TO_LLM, IMMEDIATE_IF_LAST, IMMEDIATE]`       | reprocess          | not last, and `TO_LLM` disqualifies the all-immediate rule   |
| `[IMMEDIATE_IF_LAST(err)]`                     | reprocess          | any error disables immediate return                          |
| `[TO_LLM(err), IMMEDIATE_IF_LAST]`             | reprocess          | any error disables immediate return                          |

See the Javadoc of `ReturnBehavior` for the full immediate-return-vs-reprocess matrix.

#### `IMMEDIATE_IF_LAST` for tools that explicitly close an action sequence

`ReturnBehavior.IMMEDIATE_IF_LAST` is meant for tools the LLM uses to explicitly signal
the end of a multi-step action — for example, an `endExecutionAndGetFinalResult` tool
the LLM appendsafter a sequence of clicks, navigations, etc.

Without `IMMEDIATE_IF_LAST`, the LLM would typically need two turns to close out execution:
one turn that mixes work tools (`TO_LLM`) with the closing tool,
and a second turn where the LLM, after seeing all the results,
calls the closing tool alone. The loop only returns immediately on the second turn.

With `IMMEDIATE_IF_LAST`, the loop returns immediately as soon as the LLM places the tool last
in its response — saving one full LLM round trip per invocation.

```java
class ScreenAutomation {

    @Tool
    String leftMouseClick(int x, int y) { /* ... */ }

    @Tool
    String typeText(String text) { /* ... */ }

    @Tool(returnBehavior = ReturnBehavior.IMMEDIATE_IF_LAST)
    String endExecutionAndGetFinalResult(String summary) { return summary; }
}
```

For an LLM response of `[leftMouseClick, typeText, endExecutionAndGetFinalResult]`,
the loop returns immediately after executing all three tools.
If the LLM puts the closing tool anywhere other than last
(e.g. `[endExecutionAndGetFinalResult, leftMouseClick]`),
the loop continues and sends all results back to the LLM.

`IMMEDIATE_IF_LAST` also counts toward the all-immediate rule of `IMMEDIATE`:
a response made up only of `IMMEDIATE` and/or `IMMEDIATE_IF_LAST` tools returns immediately
regardless of which one is last (still subject to the no-errors rule).

Like `IMMEDIATE`, `IMMEDIATE_IF_LAST` is only allowed on AI services with a `Result<T>` return type.

### Error Handling

#### Handling Tool Name Errors

It may happen that an LLM hallucinates on tools invocation,
or in other words that it asks to use a tool with a name that doesn't exist.
In this case by default LangChain4j will throw an exception reporting the problem,
but it is possible to configure a different behavior providing the AI service
with a strategy to be used in this situation.

This strategy is an implementation of a `Function<ToolExecutionRequest, ToolExecutionResultMessage>` defining which `ToolExecutionResultMessage` should be produced as the result for a `ToolExecutionRequest` containing the request to invoke a tool that is not available. For instance, it could be possible to configure the AI service with a strategy that returns to the LLM a response that hopefully will push it to retry a different tool invocation, knowing that the formerly required tool doesn't exist, as in the following example:

```java
AssistantHallucinatedTool assistant = AiServices.builder(AssistantHallucinatedTool.class)
        .chatModel(chatModel)
        .tools(new HelloWorld())
        .hallucinatedToolNameStrategy(toolExecutionRequest -> ToolExecutionResultMessage.from(
                toolExecutionRequest, "Error: there is no tool called " + toolExecutionRequest.name()))
        .build();
```

#### Handling Tool Arguments Errors

By default, when something is wrong with tool arguments (e.g., the LLM generates an invalid JSON
or omits a required parameter), the AI Service will not be able to execute the tool, so it will
fail with an exception.

:::caution Recommended: feed argument errors back to the LLM
The current default (throw) is rarely what you want.
Argument errors usually come from the LLM, and LLMs can typically self-correct when given a clear
error message. Configure a `ToolArgumentsErrorHandler` that returns the error text so the LLM can
retry with corrected arguments.

We are planning to change the default to this behaviour in LangChain4j 2.0. If this planned change
would affect your use case, please [open an issue](https://github.com/langchain4j/langchain4j/issues)
so we can hear your feedback before it lands.
:::

You can customize this behaviour by configuring a `ToolArgumentsErrorHandler` on the AI Service:

```java
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .tools(tools)
        .toolArgumentsErrorHandler((error, errorContext) -> ...)
        .build();
```

Currently, there are two ways to handle errors inside the `ToolArgumentsErrorHandler`:

- Throw an exception: this will stop the AI service flow.
- Return a text message (e.g., an error description) that will be sent back to the LLM,
  allowing it to respond appropriately (for example, by correcting the error and retrying).

**Recommended (let the LLM retry):**

```java
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .tools(tools)
        .toolArgumentsErrorHandler((error, errorContext) -> ToolErrorHandlerResult.text(error.getMessage()))
        .build();
```

**Strict (stop the flow on any argument error):**

```java
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .tools(tools)
        .toolArgumentsErrorHandler((error, errorContext) -> { throw MyCustomException(error); })
        .build();

try {
    assistant.chat(...);
} catch (MyCustomException e) {
    // handle e
}
```

#### Handling Tool Execution Errors

By default, when a method annotated with `@Tool` throws an `Exception`,
the message of the `Exception` (`e.getMessage()`) will be sent to the LLM as the result of tool's execution.
This allows the LLM to correct its mistake and retry, if it considers it necessary.

:::warning Recommended: do not send raw exception messages to the LLM in production
The current default sends the raw exception message back to the LLM. In production this can leak
internal application data: stack traces, file paths, credentials embedded in error strings,
downstream API responses, PII from error messages, etc.
Once fed to the LLM, this content can flow into responses, chat history, observability pipelines,
and the LLM provider's logs.

Configure a `ToolExecutionErrorHandler` that returns either a generic message or a curated/sanitized
description of the failure, and rely on logs and observability events for the underlying detail.

We are planning to change the default to "Throw an exception and abort AI Service invocation" in
LangChain4j 2.0. If this planned change would affect your use case, please
[open an issue](https://github.com/langchain4j/langchain4j/issues) so we can hear your feedback
before it lands.
:::

You can customize this behaviour by configuring a `ToolExecutionErrorHandler` on the AI Service:

```java
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .tools(tools)
        .toolExecutionErrorHandler((error, errorContext) -> ToolErrorHandlerResult.text("Tool execution failed."))
        .build();
```

As with the `ToolArgumentsErrorHandler`, there are two ways to handle errors in `ToolExecutionErrorHandler`:
throw an exception or return a text message.

## Model Context Protocol (MCP)

You can also import [tools from MCP server](https://modelcontextprotocol.io/docs/concepts/tools).
More information on this can be found [here](/tutorials/mcp/#creating-an-mcp-tool-provider).

## Related Tutorials

- [Great guide on Tools](https://www.youtube.com/watch?v=cjI_6Siry-s)
  by [Tales from the jar side (Ken Kousen)](https://www.youtube.com/@talesfromthejarside)

## Examples

- [Example with Tools](https://github.com/langchain4j/langchain4j-examples/blob/main/other-examples/src/main/java/ServiceWithToolsExample.java)
- [Example with dynamic Tools](https://github.com/langchain4j/langchain4j-examples/blob/main/other-examples/src/main/java/ServiceWithDynamicToolsExample.java)
@
