# Step 03 - Streaming responses

LLM responses can be long.
Imagine asking the model to generate a story. It could potentially produce hundreds of lines of text.

In the current application, the entire response is accumulated before being sent to the client.
During that generation, the client is waiting for the response, and the server is waiting for the model to finish generating the response.
Sure there is the "..." bubble indicating that something is happening, but it is not the best user experience.

Streaming allows us to send the response in chunks as it is generated by the model.
The model sends the response in chunks (tokens) and the server sends these chunks to the client as they arrive.

The final code of this step is located in the `step-03` directory.
However, we recommend you to follow the instructions below to get there, and continue extending your current application.

## Asking the LLM to return chunks

The first step is to ask the LLM to return the response in chunks.
Initially, our AI service looked like this:

```java title="CustomerSupportAgent.java"
--8<-- "../../section-1/step-02/src/main/java/dev/langchain4j/quarkus/workshop/CustomerSupportAgent.java"
```

Note that the return type of the `chat` method is `String`.
==We will change it to `Multi<String>` to indicate that the response will be streamed instead of returned synchronously.==

```java title="CustomerSupportAgent.java"
--8<-- "../../section-1/step-03/src/main/java/dev/langchain4j/quarkus/workshop/CustomerSupportAgent.java"
```

A `Multi<String>` is a stream of strings.
`Multi` is a type from the [Mutiny](https://smallrye.io/smallrye-mutiny/latest/){target="_blank"} library that represents a stream of items, possibly infinite.
In this case, it will be a stream of strings representing the response from the LLM, and it will be finite (fortunately).
A `Multi` has other characteristics, such as the ability to handle back pressure, which we will not cover in this workshop.

## Serving streams from the websocket

Ok, now our AI Service returns a stream of strings.
But, we need to modify our websocket endpoint to handle this stream and send it to the client.

Currently, our websocket endpoint looks like this:

```java title="CustomerSupportAgentWebSocket.java"
--8<-- "../../section-1/step-02/src/main/java/dev/langchain4j/quarkus/workshop/CustomerSupportAgentWebSocket.java"
```

==Let's modify the `onTextMessage` method to send the response to the client as it arrives.==

```java hl_lines="22-25" title="CustomerSupportAgentWebSocket.java"
--8<-- "../../section-1/step-03/src/main/java/dev/langchain4j/quarkus/workshop/CustomerSupportAgentWebSocket.java"
```

That's it!
Now the response will be streamed to the client as it arrives.
This is because Quarkus understands that the return type is a `Multi` _natively_, and it knows how to handle it.

## Testing the streaming

To test the streaming, you can use the same chat interface as before.
The application should still be running. Go back to the browser, refresh the page, and start chatting.
If you ask simple questions, you may not notice the difference.

Ask something like

```
Tell me a story containing 500 words
```

and you will see the response being displayed as it arrives.

![type:video](../images/streaming.mp4){autoplay=true}

Let's now switch to the [next step](./step-04.md)!
