# LiteRT-LM model host for ADK agents

<div class="language-support-tag">
    <span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.1.0</span>
</div>

[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) is a C++ library to
efficiently run language models across edge platforms.
On desktop (Linux, macOS, and Windows), ADK integrates with LiteRT-LM-hosted
models through the LiteRT-LM server launched by the LiteRT-LM CLI `lit`.

## Get started

LiteRT-LM works with the `Gemini` class. You only have to set the `base_url` and
`model` parameters.

1.  Set `base_url` to the LiteRT-LM server URL, for example: `localhost:8001`.
2.  Set `model` to the LiteRT-LM model name, for example: `gemma3n-e2b`.

```py
from google.adk.agents import Agent
from google.adk.models import Gemini

root_agent = Agent(
    model=Gemini(
        model="gemma3n-e2b",
        base_url="http://localhost:8001",
    ),
    name="dice_agent",
    description=(
        "hello world agent that can roll a die of 8 sides and check prime"
        " numbers."
    ),
    instruction="""
      You roll dice and answer questions about the outcome of the dice rolls.
    """,
    tools=[
        roll_die,
        check_prime,
    ],
)
```

Then run the agent as usual:

```bash
adk web
```

## Running the LiteRT-LM Server

The LiteRT-LM server is a separate process that serves LiteRT-LM models. It is
started by the LiteRT-LM CLI tool `lit`.

### Download the `lit` CLI tool

Download the `lit` CLI tool by following these
[instructions](https://github.com/google-ai-edge/LiteRT-LM?tab=readme-ov-file#desktop-cli-lit)
in the LiteRT-LM GitHub repository.

### Download a model

Before you start the server, you need to download a model. You'll need a
*Hugging Face* user access token to download a LiteRT-LM model using `lit`. You
can get a token for your *Hugging Face* account
[here](https://huggingface.co/settings/tokens).

To see a list of models available for download, use the `lit list` command:

```bash
lit list --show_all
```

Download a model using the `lit pull` command:

```bash
export HUGGING_FACE_HUB_TOKEN="**your Hugging Face token**"
lit pull gemma3n-e2b
```

### Run the server

After downloading a model, start the LiteRT-LM server locally by running the
following command:

```bash
lit serve --port 8001
```

!!! tip "Local Server Port Number"

    You may choose any port number for the LiteRT-LM server as long as it matches the `base_url` you set in the `Gemini` class in your agent code.

### Debugging

To see incoming requests to the LiteRT-LM server and the exact input sent to the
model, use the `--verbose` flag:

```bash
lit serve --port 8001 --verbose
```
