Skip to main content

# Prompt context

Prompt contexts let you inject pre-messages and post-messages around the consumer's messages. This is the primary way to add system prompts, instructions, or framing information to LLM interactions without modifying the consumer's request.

## How it works​

The consumer sends messages like:

    [
      {"role": "user", "content": "What is the weather in Paris?"}
    ]

With a prompt context that has a system prompt as a pre-message, the LLM actually receives:

    [
      {"role": "system", "content": "You are an english butler. Always respond politely and formally."},
      {"role": "user", "content": "What is the weather in Paris?"},
      {"role": "system", "content": "Remember to be concise."}
    ]

## Entity structure​

    {
      "id": "context_xxx",
      "name": "English Butler",
      "description": "Makes the LLM respond like an english butler",
      "pre_messages": [
        {
          "role": "system",
          "content": "You are an english butler. Always respond politely and formally."
        }
      ],
      "post_messages": [
        {
          "role": "system",
          "content": "Remember to be concise."
        }
      ]
    }

Field | Type | Description
--- | --- | ---
`pre_messages` | array | Messages prepended before the consumer's messages
`post_messages` | array | Messages appended after the consumer's messages

Each message has:

Field | Type | Description
--- | --- | ---
`role` | string | Message role (`system`, `user`, `assistant`)
`content` | string | Message content

## Two ways to use contexts​

### 1. Plugin-based (route level)​

Add the Cloud APIM - LLM Proxy - prompt context plugin to your route, before the LLM proxy plugin:

    {
      "enabled": true,
      "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.AiPromptContext",
      "config": {
        "ref": "context-entity-id"
      }
    }

Parameter | Type | Description
--- | --- | ---
`ref` | string | Reference to a PromptContext entity ID

Multiple context plugins can be stacked on the same route. Their pre/post messages accumulate.

### 2. Provider-level (provider settings)​

Each LLM provider entity has a `context` setting that supports a default context and a whitelist of selectable contexts:

    {
      "context": {
        "default": "context-entity-id",
        "contexts": ["context-entity-id-1", "context-entity-id-2"]
      }
    }

Parameter | Type | Default | Description
--- | --- | --- | ---
`default` | string | `null` | Default context applied automatically when no context is specified
`contexts` | array | `[]` | Whitelist of contexts that consumers can select from

Consumers can select a context by including `"context"` in the request body:

    curl --request POST \
      --url http://myroute.oto.tools:8080/v1/chat/completions \
      --header 'content-type: application/json' \
      --data '{
      "messages": [
        {"role": "user", "content": "Hello!"}
      ],
      "context": "context-entity-id-1"
    }'

The context can be referenced by ID or name. Only contexts listed in the provider's whitelist are accepted. The `context` field is stripped from the body before forwarding to the LLM.

## Plugin vs. provider-level context​

Aspect | Plugin-based | Provider-level
--- | --- | ---
Configured on | Route | Provider entity
Consumer can choose | No | Yes (from whitelist)
Default context | Always applied | Applied when consumer doesn't specify one
Stackable | Yes (multiple plugins) | No (one context per request)
Use case | Fixed context per route | Multi-persona or multi-tenant setup

## Example: multi-persona provider​

Configure a provider with multiple available contexts:

    {
      "context": {
        "default": "context_butler",
        "contexts": ["context_butler", "context_engineer", "context_teacher"]
      }
    }

Consumers select the persona in their request:

    {"messages": [{"role": "user", "content": "Explain quantum computing"}], "context": "context_teacher"}

## Testing contexts​

The admin UI includes a built-in context tester. Provide a user message and test the context injection + LLM call directly from the Otoroshi back-office.