.. _llm_router:

LLM Routing
==============================================================

With the rapid proliferation of large language models (LLMs) — each optimized for different strengths, style, or latency/cost profile — routing has become an essential technique to operationalize the use of different models. Plano provides three distinct routing approaches to meet different use cases: :ref:`Model-based routing <model_based_routing>`, :ref:`Alias-based routing <alias_based_routing>`, and :ref:`Preference-aligned routing <preference_aligned_routing>`. This enables optimal performance, cost efficiency, and response quality by matching requests with the most suitable model from your available LLM fleet.

.. note::
  For details on supported model providers, configuration options, and client libraries, see :ref:`LLM Providers <llm_providers>`.

Routing Methods
---------------

.. _model_based_routing:

Model-based routing
~~~~~~~~~~~~~~~~~~~

Direct routing allows you to specify exact provider and model combinations using the format ``provider/model-name``:

- Use provider-specific names like ``openai/gpt-5.2`` or ``anthropic/claude-sonnet-4-5``
- Provides full control and transparency over which model handles each request
- Ideal for production workloads where you want predictable routing behavior

Configuration
^^^^^^^^^^^^^

Configure your LLM providers with specific provider/model names:

.. code-block:: yaml
    :caption: Model-based Routing Configuration

    listeners:
      egress_traffic:
        address: 0.0.0.0
        port: 12000
        message_format: openai
        timeout: 30s

    llm_providers:
      - model: openai/gpt-5.2
        access_key: $OPENAI_API_KEY
        default: true

      - model: openai/gpt-5
        access_key: $OPENAI_API_KEY

      - model: anthropic/claude-sonnet-4-5
        access_key: $ANTHROPIC_API_KEY

Client usage
^^^^^^^^^^^^

Clients specify exact models:

.. code-block:: python

    # Direct provider/model specification
    response = client.chat.completions.create(
        model="openai/gpt-5.2",
        messages=[{"role": "user", "content": "Hello!"}]
    )

    response = client.chat.completions.create(
        model="anthropic/claude-sonnet-4-5",
        messages=[{"role": "user", "content": "Write a story"}]
    )

.. _alias_based_routing:

Alias-based routing
~~~~~~~~~~~~~~~~~~~

Alias-based routing lets you create semantic model names that decouple your application from specific providers:

- Use meaningful names like ``fast-model``, ``reasoning-model``, or ``plano.summarize.v1`` (see :ref:`model_aliases`)
- Maps semantic names to underlying provider models for easier experimentation and provider switching
- Ideal for applications that want abstraction from specific model names while maintaining control

Configuration
^^^^^^^^^^^^^

Configure semantic aliases that map to underlying models:

.. code-block:: yaml
    :caption: Alias-based Routing Configuration

    listeners:
      egress_traffic:
        address: 0.0.0.0
        port: 12000
        message_format: openai
        timeout: 30s

    llm_providers:
      - model: openai/gpt-5.2
        access_key: $OPENAI_API_KEY

      - model: openai/gpt-5
        access_key: $OPENAI_API_KEY

      - model: anthropic/claude-sonnet-4-5
        access_key: $ANTHROPIC_API_KEY

    model_aliases:
      # Model aliases - friendly names that map to actual provider names
      fast-model:
        target: gpt-5.2

      reasoning-model:
        target: gpt-5

      creative-model:
        target: claude-sonnet-4-5

Client usage
^^^^^^^^^^^^

Clients use semantic names:

.. code-block:: python

    # Using semantic aliases
    response = client.chat.completions.create(
        model="fast-model",  # Routes to best available fast model
        messages=[{"role": "user", "content": "Quick summary please"}]
    )

    response = client.chat.completions.create(
        model="reasoning-model",  # Routes to best reasoning model
        messages=[{"role": "user", "content": "Solve this complex problem"}]
    )

.. _preference_aligned_routing:

Preference-aligned routing (Plano-Orchestrator)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Preference-aligned routing uses the `Plano-Orchestrator <https://huggingface.co/katanemo/Plano-Orchestrator-30B-A3B>`_ model to pick the best LLM based on domain, action, and your configured preferences instead of hard-coding a model.

- **Domain**: High-level topic of the request (e.g., legal, healthcare, programming).
- **Action**: What the user wants to do (e.g., summarize, generate code, translate).
- **Routing preferences**: Your mapping from (domain, action) to preferred models.

Plano-Orchestrator analyzes each prompt to infer domain and action, then applies your preferences to select a model. This decouples **routing policy** (how to choose) from **model assignment** (what to run), making routing transparent, controllable, and easy to extend as you add or swap models.

Configuration
^^^^^^^^^^^^^

To configure preference-aligned dynamic routing, declare a top-level ``routing_preferences`` list and attach an ordered ``models`` candidate pool to each route. Starting in ``v0.4.0``, ``routing_preferences`` lives at the root of the config (not inline under ``model_providers``), which lets multiple models serve the same route — the first entry in ``models`` is primary, the rest are fallbacks that the client tries on ``429``/``5xx`` errors.

.. code-block:: yaml
    :caption: Preference-Aligned Dynamic Routing Configuration

    version: v0.4.0

    listeners:
      - name: egress_traffic
        type: model
        address: 0.0.0.0
        port: 12000
        timeout: 30s

    model_providers:
      - model: openai/gpt-5.2
        access_key: $OPENAI_API_KEY
        default: true

      - model: openai/gpt-5
        access_key: $OPENAI_API_KEY

      - model: anthropic/claude-sonnet-4-5
        access_key: $ANTHROPIC_API_KEY

    routing_preferences:
      - name: code understanding
        description: understand and explain existing code snippets, functions, or libraries
        models:
          - openai/gpt-5
          - anthropic/claude-sonnet-4-5
      - name: complex reasoning
        description: deep analysis, mathematical problem solving, and logical reasoning
        models:
          - openai/gpt-5
      - name: creative writing
        description: creative content generation, storytelling, and writing assistance
        models:
          - anthropic/claude-sonnet-4-5
      - name: code generation
        description: generating new code snippets, functions, or boilerplate based on user prompts
        models:
          - anthropic/claude-sonnet-4-5
          - openai/gpt-5

.. note::
   Configs still using the ``v0.3.0`` inline style (``routing_preferences`` nested under each ``model_provider``) are auto-migrated to this top-level shape by the Plano CLI at compile time, with a deprecation warning. Update your config to the form above to silence the warning.

Client usage
^^^^^^^^^^^^

Clients can let the router decide or still specify aliases:

.. code-block:: python

    # Let Plano-Orchestrator choose based on content
    response = client.chat.completions.create(
        messages=[{"role": "user", "content": "Write a creative story about space exploration"}]
        # No model specified - router will analyze and choose claude-sonnet-4-5
    )


.. _cost_latency_aware_selection:

Cost- and latency-aware selection
---------------------------------

When a route lists more than one candidate model, you can let Plano reorder that
candidate pool using **live cost or latency data** instead of relying solely on the
order you wrote them in. This is controlled per route with ``selection_policy`` and
backed by one or more ``model_metrics_sources``.

This is useful when several models are equally capable for a route and you want Plano
to always reach for the cheapest (or fastest) option first, with the others kept as
fallbacks.

Selection policy
~~~~~~~~~~~~~~~~~

Attach an optional ``selection_policy`` to any entry in ``routing_preferences``:

.. code-block:: yaml
    :caption: Per-route selection policy

    routing_preferences:
      - name: code review
        description: reviewing, analyzing, and suggesting improvements to existing code
        models:
          - anthropic/claude-sonnet-4-5
          - groq/llama-3.3-70b-versatile
        selection_policy:
          prefer: cheapest   # cheapest | fastest | none

``prefer`` accepts:

- ``cheapest`` — order candidates by total price (input + output rate) ascending, using a ``cost`` metrics source.
- ``fastest`` — order candidates by observed latency ascending, using a ``latency`` metrics source.
- ``none`` (default) — keep the order you declared; no reordering.

Models that have no data in the selected source are ranked **last**, in their original
order, so routing always degrades gracefully rather than dropping a candidate.

Configuring the pricing source
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``cheapest`` routing needs a price catalog. Plano's **default pricing provider is
DigitalOcean** — its GenAI model catalog is public (no API key, no signup), so cost data
is available out of the box and is what ``planoai obs`` uses if you don't configure
anything. The pricing source is fully swappable: point Plano at `models.dev <https://models.dev/>`_,
or at **any endpoint that exposes a supported pricing structure**.

The ``provider`` field selects which response schema Plano expects (and therefore how it
parses the catalog); the optional ``url`` lets you override the endpoint — for example to
use a mirror, a cached copy, or an internal catalog service that returns the same shape.

.. list-table::
   :header-rows: 1
   :widths: 18 34 28 20

   * - ``provider``
     - Default catalog URL
     - Key format
     - Expected structure
   * - ``digitalocean`` *(default)*
     - DigitalOcean GenAI model catalog
     - ``lowercase(creator)/model_id``
     - ``{ data: [ { model_id, pricing: { input_price_per_million, output_price_per_million } } ] }``
   * - ``models.dev``
     - ``https://models.dev/api.json``
     - ``creator/model`` (e.g. ``anthropic/claude-sonnet-4-5``)
     - ``{ <provider>: { models: { <model>: { cost: { input, output } } } } }``

Because the source is selected per ``provider``, switching is a one-line change. To stay
on the default DigitalOcean catalog you can omit ``model_metrics_sources`` entirely for
``planoai obs``, or declare it explicitly for routing:

.. code-block:: yaml
    :caption: Default cost source (DigitalOcean)

    model_metrics_sources:
      - type: cost
        provider: digitalocean   # default; uses the public DO GenAI catalog

To switch to models.dev — an open, community-maintained catalog covering a broad range of
providers and models — change the ``provider`` (and optionally ``url``):

.. code-block:: yaml
    :caption: Cost source backed by models.dev

    model_metrics_sources:
      - type: cost
        provider: models.dev               # models.dev | digitalocean
        url: https://models.dev/api.json    # optional; defaults per provider
        refresh_interval: 3600              # optional, seconds; refetch on this interval
        model_aliases:                      # optional; see below
          openai/gpt-oss-120b: openai/gpt-4o

To use your own endpoint, pick the ``provider`` whose structure your endpoint matches and
override ``url`` — Plano parses the response with that provider's schema:

.. code-block:: yaml
    :caption: Custom endpoint exposing the DigitalOcean catalog structure

    model_metrics_sources:
      - type: cost
        provider: digitalocean                       # selects the DO response schema
        url: https://catalog.internal.example.com/pricing

.. note::
   The cost metric used for ranking is the sum of the input and output per-million-token
   rates — a relative signal for ordering candidates, not a per-request bill. For actual
   per-request cost, see the observability console below.

Matching catalog keys to your models
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The router looks up each candidate model by the exact name you use in
``routing_preferences`` (e.g. ``anthropic/claude-sonnet-4-5``). models.dev keys models as
``creator/model``, which lines up with Plano's ``provider/model`` naming, so most models
match automatically.

When a catalog key does not match your model name — for example a version skew, or an
open-weight model you serve under a different provider — use ``model_aliases`` to map the
**catalog key** to the **Plano model name** used in your routing preferences:

.. code-block:: yaml

    model_metrics_sources:
      - type: cost
        provider: models.dev
        model_aliases:
          # catalog key            : plano model name
          openai/gpt-oss-120b: openai/gpt-4o

Latency source
~~~~~~~~~~~~~~~

``fastest`` routing reads observed latency from a Prometheus instance. Provide the query
that returns a per-model latency value (lower is faster), labelled by ``model_name``:

.. code-block:: yaml
    :caption: Latency source backed by Prometheus

    model_metrics_sources:
      - type: latency
        provider: prometheus
        url: http://prometheus:9090
        query: avg by (model_name) (rate(plano_llm_latency_seconds_sum[5m]))
        refresh_interval: 60

You can declare both a ``cost`` and a ``latency`` source at the same time; each route
picks whichever it needs based on its ``selection_policy``.

Cost in the observability console
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``planoai obs`` displays a per-request USD cost column derived from the same pricing
catalog. By default it reads the ``cost`` source from your config (the first
``type: cost`` entry under ``model_metrics_sources``); you can also override it on the
command line:

.. code-block:: bash

    # Use the cost source from ./config.yaml (default)
    planoai obs

    # Or override the provider / endpoint explicitly
    planoai obs --pricing-provider models.dev
    planoai obs --pricing-url https://models.dev/api.json

If no source is configured and no override is given, ``planoai obs`` falls back to the
DigitalOcean catalog so the cost column still populates out of the box.


Plano-Orchestrator
-------------------
Plano-Orchestrator is a **preference-based routing model** specifically designed to address the limitations of traditional LLM routing. It delivers production-ready performance with low latency and high accuracy while solving key routing challenges.

**Addressing Traditional Routing Limitations:**

**Human Preference Alignment**
Unlike benchmark-driven approaches, Plano-Orchestrator learns to match queries with human preferences by using domain-action mappings that capture subjective evaluation criteria, ensuring routing decisions align with real-world user needs.

**Flexible Model Integration**
The system supports seamlessly adding new models for routing without requiring retraining or architectural modifications, enabling dynamic adaptation to evolving model landscapes.

**Preference-Encoded Routing**
Provides a practical mechanism to encode user preferences through domain-action mappings, offering transparent and controllable routing decisions that can be customized for specific use cases.

To support effective routing, Plano-Orchestrator introduces two key concepts:

- **Domain** – the high-level thematic category or subject matter of a request (e.g., legal, healthcare, programming).

- **Action** – the specific type of operation the user wants performed (e.g., summarization, code generation, booking appointment, translation).

Both domain and action configs are associated with preferred models or model variants. At inference time, Plano-Orchestrator analyzes the incoming prompt to infer its domain and action using semantic similarity, task indicators, and contextual cues. It then applies the user-defined routing preferences to select the model best suited to handle the request.

In summary, Plano-Orchestrator demonstrates:

- **Structured Preference Routing**: Aligns prompt request with model strengths using explicit domain–action mappings.

- **Transparent and Controllable**: Makes routing decisions transparent and configurable, empowering users to customize system behavior.

- **Flexible and Adaptive**: Supports evolving user needs, model updates, and new domains/actions without retraining the router.

- **Production-Ready Performance**: Optimized for low-latency, high-throughput applications in multi-model environments.


Self-hosting Plano-Orchestrator
-------------------------------

By default, Plano uses a hosted Plano-Orchestrator endpoint. To run Plano-Orchestrator locally, you can serve the model yourself using either **Ollama** or **vLLM**.

Using Ollama (recommended for local development)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. **Install Ollama**

   Download and install from `ollama.ai <https://ollama.ai>`_.

2. **Pull and serve the routing model**

   .. code-block:: bash

       ollama pull hf.co/katanemo/Arch-Router-1.5B.gguf:Q4_K_M
       ollama serve

   This downloads the quantized GGUF model from HuggingFace and starts serving on ``http://localhost:11434``.

3. **Configure Plano to use local routing model**

   .. code-block:: yaml

       version: v0.4.0

       overrides:
         llm_routing_model: plano/hf.co/katanemo/Arch-Router-1.5B.gguf:Q4_K_M

       model_providers:
         - model: plano/hf.co/katanemo/Arch-Router-1.5B.gguf:Q4_K_M
           base_url: http://localhost:11434

         - model: openai/gpt-5.2
           access_key: $OPENAI_API_KEY
           default: true

         - model: anthropic/claude-sonnet-4-5
           access_key: $ANTHROPIC_API_KEY

       routing_preferences:
         - name: creative writing
           description: creative content generation, storytelling, and writing assistance
           models:
             - anthropic/claude-sonnet-4-5

4. **Verify the model is running**

   .. code-block:: bash

       curl http://localhost:11434/v1/models

   You should see ``Arch-Router-1.5B`` listed in the response.

Using vLLM (recommended for production / EC2)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vLLM provides higher throughput and GPU optimizations suitable for production deployments.

1. **Install vLLM**

   .. code-block:: bash

       pip install vllm

2. **Download the model weights**

   The GGUF weights are downloaded automatically from HuggingFace on first use. To pre-download:

   .. code-block:: bash

       pip install huggingface_hub
       huggingface-cli download katanemo/Arch-Router-1.5B.gguf

3. **Start the vLLM server**

   After downloading, find the GGUF file and Jinja template in the HuggingFace cache:

   .. code-block:: bash

       # Find the downloaded files
       SNAPSHOT_DIR=$(ls -d ~/.cache/huggingface/hub/models--katanemo--Arch-Router-1.5B.gguf/snapshots/*/ | head -1)

       vllm serve ${SNAPSHOT_DIR}Arch-Router-1.5B-Q4_K_M.gguf \
           --host 0.0.0.0 \
           --port 10000 \
           --load-format gguf \
           --chat-template ${SNAPSHOT_DIR}template.jinja \
           --tokenizer katanemo/Arch-Router-1.5B \
           --served-model-name Plano-Orchestrator \
           --gpu-memory-utilization 0.3 \
           --tensor-parallel-size 1 \
           --enable-prefix-caching

4. **Configure Plano to use the vLLM endpoint**

   .. code-block:: yaml

       version: v0.4.0

       overrides:
         llm_routing_model: plano/Plano-Orchestrator

       model_providers:
         - model: plano/Plano-Orchestrator
           base_url: http://<your-server-ip>:10000

         - model: openai/gpt-5.2
           access_key: $OPENAI_API_KEY
           default: true

         - model: anthropic/claude-sonnet-4-5
           access_key: $ANTHROPIC_API_KEY

       routing_preferences:
         - name: creative writing
           description: creative content generation, storytelling, and writing assistance
           models:
             - anthropic/claude-sonnet-4-5

5. **Verify the server is running**

   .. code-block:: bash

       curl http://localhost:10000/health
       curl http://localhost:10000/v1/models


Using vLLM on Kubernetes (GPU nodes)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For teams running Kubernetes, Plano-Orchestrator and Plano can be deployed as in-cluster services.
The ``demos/llm_routing/model_routing_service/`` directory includes ready-to-use manifests:

- ``vllm-deployment.yaml`` — Plano-Orchestrator served by vLLM, with an init container to download
  the model from HuggingFace
- ``plano-deployment.yaml`` — Plano proxy configured to use the in-cluster Plano-Orchestrator
- ``config_k8s.yaml`` — Plano config with ``llm_routing_model`` pointing at
  ``http://plano-orchestrator:10000`` instead of the default hosted endpoint

Key things to know before deploying:

- GPU nodes commonly have a ``nvidia.com/gpu:NoSchedule`` taint — the ``vllm-deployment.yaml``
  includes a matching toleration. The ``nvidia.com/gpu: "1"`` resource request is sufficient
  for scheduling in most clusters; a ``nodeSelector`` is optional and commented out in the
  manifest for cases where you need to pin to a specific GPU node pool.
- Model download takes ~1 minute; vLLM loads the model in ~1-2 minutes after that. The
  ``livenessProbe`` has a 180-second ``initialDelaySeconds`` to avoid premature restarts.
- The Plano config ConfigMap must use ``--from-file=plano_config.yaml=config_k8s.yaml`` with
  ``subPath`` in the Deployment — omitting ``subPath`` causes Kubernetes to mount a directory
  instead of a file.

For the canonical Plano Kubernetes deployment (ConfigMap, Secrets, Deployment YAML), see
:ref:`deployment`. For full step-by-step commands specific to this demo, see the
`demo README <https://github.com/katanemo/plano/tree/main/demos/llm_routing/model_routing_service/README.md>`_.


.. _model_affinity:

Model Affinity
--------------

In agentic loops — where a single user request triggers multiple LLM calls through tool use — Plano's router classifies each turn independently. Because successive prompts differ in intent (tool selection looks like code generation, reasoning about results looks like analysis), the router may select different models mid-session. This causes behavioral inconsistency and invalidates provider-side KV caches, increasing both latency and cost.

**Model affinity** pins the routing decision for the duration of a session. Send an ``X-Model-Affinity`` header with any string identifier (typically a UUID). The first request routes normally and caches the result. All subsequent requests with the same affinity ID skip routing and reuse the cached model.

.. code-block:: python

    import uuid
    from openai import OpenAI

    client = OpenAI(base_url="http://localhost:12000/v1", api_key="EMPTY")
    affinity_id = str(uuid.uuid4())

    # Every call in the loop uses the same header
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        tools=tools,
        extra_headers={"X-Model-Affinity": affinity_id},
    )

Without the header, routing runs fresh on every request — no behavior change for existing clients.

**Configuration:**

.. code-block:: yaml

    routing:
      session_ttl_seconds: 600    # How long affinity lasts (default: 10 min)
      session_max_entries: 10000  # Max cached sessions (upper limit: 10000)

To start a new routing decision (e.g., when the agent's task changes), generate a new affinity ID.

Session Cache Backends
~~~~~~~~~~~~~~~~~~~~~~

By default, Plano stores session affinity state in an in-process LRU cache. This works well for single-instance deployments, but sessions are not shared across replicas — each instance has its own independent cache.

For deployments with multiple Plano replicas (Kubernetes, Docker Compose with ``scale``, or any load-balanced setup), use Redis as the session cache backend. All replicas connect to the same Redis instance, so an affinity decision made by one replica is honoured by every other replica in the pool.

**In-memory (default)**

No configuration required. Sessions live only for the lifetime of the process and are lost on restart.

.. code-block:: yaml

    routing:
      session_ttl_seconds: 600    # How long affinity lasts (default: 10 min)
      session_max_entries: 10000  # LRU capacity (upper limit: 10000)

**Redis**

Requires a reachable Redis instance. The ``url`` field supports standard Redis URI syntax, including authentication (``redis://:password@host:6379``) and TLS (``rediss://host:6380``). Redis handles TTL expiry natively, so no periodic cleanup is needed.

.. code-block:: yaml

    routing:
      session_ttl_seconds: 600
      session_cache:
        type: redis
        url: redis://localhost:6379

.. note::
   When using Redis in a multi-tenant environment, construct the ``X-Model-Affinity`` header value to include a tenant identifier, for example ``{tenant_id}:{session_id}``. Plano stores each key under the internal namespace ``plano:affinity:{key}``, so tenant-scoped values avoid cross-tenant collisions without any additional configuration.

**Example: Kubernetes multi-replica deployment**

Deploy a Redis instance alongside your Plano pods and point all replicas at it:

.. code-block:: yaml

    routing:
      session_ttl_seconds: 600
      session_cache:
        type: redis
        url: redis://redis.plano.svc.cluster.local:6379

With this configuration, any replica that first receives a request for affinity ID ``abc-123`` caches the routing decision in Redis. Subsequent requests for ``abc-123`` — regardless of which replica they land on — retrieve the same pinned model.


Combining Routing Methods
-------------------------

You can combine static model selection with dynamic routing preferences for maximum flexibility:

.. code-block:: yaml
    :caption: Hybrid Routing Configuration

    version: v0.4.0

    model_providers:
      - model: openai/gpt-5.2
        access_key: $OPENAI_API_KEY
        default: true

      - model: openai/gpt-5
        access_key: $OPENAI_API_KEY

      - model: anthropic/claude-sonnet-4-5
        access_key: $ANTHROPIC_API_KEY

    routing_preferences:
      - name: complex_reasoning
        description: deep analysis and complex problem solving
        models:
          - openai/gpt-5
          - anthropic/claude-sonnet-4-5
      - name: creative_tasks
        description: creative writing and content generation
        models:
          - anthropic/claude-sonnet-4-5
          - openai/gpt-5

    model_aliases:
      # Model aliases - friendly names that map to actual provider names
      fast-model:
        target: gpt-5.2

      reasoning-model:
        target: gpt-5

      # Aliases that can also participate in dynamic routing
      creative-model:
        target: claude-sonnet-4-5

This configuration allows clients to:

1. **Use direct model selection**: ``model="fast-model"``
2. **Let the router decide**: No model specified, router analyzes content

Example Use Cases
-----------------
Here are common scenarios where Plano-Orchestrator excels:

- **Coding Tasks**: Distinguish between code generation requests ("write a Python function"), debugging needs ("fix this error"), and code optimization ("make this faster"), routing each to appropriately specialized models.

- **Content Processing Workflows**: Classify requests as summarization ("summarize this document"), translation ("translate to Spanish"), or analysis ("what are the key themes"), enabling targeted model selection.

- **Multi-Domain Applications**: Accurately identify whether requests fall into legal, healthcare, technical, or general domains, even when the subject matter isn't explicitly stated in the prompt.

- **Conversational Routing**: Track conversation context to identify when topics shift between domains or when the type of assistance needed changes mid-conversation.

Best practices
--------------
- **💡Consistent Naming:**  Route names should align with their descriptions.

  - ❌ Bad:
    ```
    {"name": "math", "description": "handle solving quadratic equations"}
    ```
  - ✅ Good:
    ```
    {"name": "quadratic_equation", "description": "solving quadratic equations"}
    ```

- **💡 Clear Usage Description:**  Make your route names and descriptions specific, unambiguous, and minimizing overlap between routes. The Router performs better when it can clearly distinguish between different types of requests.

  - ❌ Bad:
    ```
    {"name": "math", "description": "anything closely related to mathematics"}
    ```
  - ✅ Good:
    ```
    {"name": "math", "description": "solving, explaining math problems, concepts"}
    ```

- **💡Nouns Descriptor:** Preference-based routers perform better with noun-centric descriptors, as they offer more stable and semantically rich signals for matching.

- **💡Domain Inclusion:** for best user experience, you should always include a domain route. This helps the router fall back to domain when action is not confidently inferred.

Unsupported Features
--------------------

The following features are **not supported** by the Plano-Orchestrator routing model:

- **Multi-modality**: The model is not trained to process raw image or audio inputs. It can handle textual queries *about* these modalities (e.g., "generate an image of a cat"), but cannot interpret encoded multimedia data directly.

- **Function calling**: Plano-Orchestrator is designed for **semantic preference matching**, not exact intent classification or tool execution. For structured function invocation, use models in the Plano Function Calling collection instead.

- **System prompt dependency**: Plano-Orchestrator routes based solely on the user’s conversation history. It does not use or rely on system prompts for routing decisions.
