Skip to main content

# Overview

The Otoroshi LLM extension provides a complete prompt engineering toolkit to control, optimize, and standardize interactions with LLMs. You can store reusable prompts, create message templates with dynamic expressions, and inject contextual information around user messages.

## Features​

Feature | Description
--- | ---
Prompt library | Store and reuse prompt texts across providers, plugins, and guardrails
Prompt templating | Define message templates with `@{path}` expressions that dynamically inject request data
Prompt context | Inject pre/post messages (system prompts, instructions) around user messages
Prompt validator | Validate prompt content against allow/deny regex patterns

## How they work together​

These features can be combined in a single route to build powerful prompt pipelines:

1. Prompt validator (access validation step) — checks that the consumer's messages comply with allow/deny patterns
2. Prompt template (request transformation step) — optionally replaces the consumer's messages with a template-rendered message set
3. Prompt context (request transformation step) — injects pre/post messages around the (possibly templated) messages
4. Provider-level context (decorator) — the provider can also inject its own default context

Multiple prompt context plugins can be stacked on a single route — their pre/post messages accumulate.

## Provider-level context settings​

In addition to the plugin-based context injection, each LLM provider entity has its own context settings:

    {
      "context": {
        "default": "context-entity-id",
        "contexts": ["context-entity-id-1", "context-entity-id-2"]
      }
    }

Parameter | Type | Default | Description
--- | --- | --- | ---
`default` | string | `null` | Default PromptContext ID applied automatically when no context is specified in the request
`contexts` | array | `[]` | Whitelist of allowed PromptContext IDs that consumers can select from

Consumers can select a context by including `"context": "<id-or-name>"` in the request body. Only contexts listed in the provider's whitelist are accepted. If no context is specified, the `default` context is used.

## Admin API​

Entity | API path
--- | ---
Prompts | `ai-gateway.extensions.cloud-apim.com/v1/prompts`
Prompt Templates | `ai-gateway.extensions.cloud-apim.com/v1/prompt-templates`
Prompt Contexts | `ai-gateway.extensions.cloud-apim.com/v1/prompt-contexts`