GAI

📋 Requirements

• Go 1.26.x or newer
• API credentials for whichever provider you use

📦 Installation

go get github.com/lace-ai/gai

🧱 PromptBuilder

</summary>

PromptBuilder composes the full ai.Prompt from structured parts:

prompt := gaictx.NewPromptBuilder().
    Budget(gaictx.PromptBudget{
        Tokenizer:            model.Tokenizer(),
        ContextWindowTokens:  128000,
        ReservedOutputTokens: 4096,
    }).
    System("base-system", "Follow the system policy.", gaictx.Required()).
    Source(gaictx.SectionContext, "history", gaictx.History(store, sessionID), gaictx.Required(), gaictx.SourceTokenCap(1000)).
    Source(gaictx.SectionContext, "rag", ragSource, gaictx.Optional(), gaictx.SourceTokenCap(2000)).
    User("request", "Summarize the project status.", gaictx.Required())

Entries use stable IDs and are rendered in fixed section order (system, context, user). Inside each section, required entries render before optional entries; configured order is preserved within the required group and within the optional group. This lets required sources receive budget before optional context can consume it.

With Budget(...) configured, the builder counts the rendered prompt with the configured tokenizer. ContextWindowTokens - ReservedOutputTokens is the prompt budget, and ConversationReserveTokens reserves space for loop messages appended by incremental prompt sessions. Required entries must fit or BuildPrompt returns ErrPromptBudget. Optional entries are best-effort: the builder includes them when they fit, asks the configured Summarizer to compact optional static parts when they do not fit, and drops them when there is no fitting summary. Optional source failures are skipped, and optional source overflow is dropped.

Dynamic sources implement:

type Source interface {
    BuildParts(ctx context.Context, view PromptView, budget SourceBudget) ([]Part, error)
}

PromptView exposes the current conversation and a read-only view of the whole configured prompt plan, so sources can inspect planned entries by ID or section before emitting parts. SourceBudget provides the source's cap, remaining prompt budget, tokenizer, render-overhead reserve ratio, required flag, and optional summarizer. SourceTokenCap(...) limits one source's budget; uncapped sources receive the remaining prompt budget.

The default renderer is XML-like and can be replaced with a custom renderer. Grouped parts render as one outer part with child items, which lets sources like RAG budget individual documents without adding a full XML wrapper around every document. LastTrace() returns the most recent build trace, including emitted, skipped, dropped, summarized, token counts, available tokens, and reasons. Debug(gai.DebugSink) emits the same prompt-build decisions. Rendered part text is only included in debug events when the sink allows sensitive data.

</details>

<details>

<summary>

🚀 Quick Start

🧭 Usage

🛠️ Creating Providers and Models

To start, first create a provider. For example, for Gemini:

geminiProvider := gemini.New("your_api_key", nil)

<details>

<summary>🗂️ Use the Model Repository to manage multiple providers and dynamic model selection</summary>

You can use a ModelRepository to register multiple providers and look up models by name across providers.

modelRepo := ai.NewModelRepository(nil)
err := modelRepo.RegisterProvider(context.Background(), geminiProvider)
if err != nil {
    // handle error
}

To get a model from the repo just use the provider name and the model name:

model, err := modelRepo.GetModel(context.Background(), "gemini", "gemini-3-flash-preview")
if err != nil {
    // handle error
}

</details>

💬 Generate Text

Now you can access models from that provider, and generate text:

model, err := geminiProvider.Model("gemini-3-flash-preview")
if err != nil {
    // handle error
}
response, err := model.Generate(context.Background(), ai.AIRequest{
    Prompt: ai.Prompt{
        System: "You are a helpful assistant.",
        Prompt: "What is the capital of France?",
    },
    MaxTokens: 100,
})

<details>

<summary>🔌 Implement Your Own Provider</summary>

Currently, the library includes Gemini and Mistral implementations. Gemini uses the official go-genai library, and Mistral uses direct HTTP calls to the Mistral API.

But you can implement your own provider by implementing the Provider, Model, and Tokenizer interfaces defined in the ai package.

Example:

Provider Implementation:

type MyProvider struct {
    // any configuration fields you need, e.g. API key
}

func (p *MyProvider) Name() string {
    return "myprovider"
}

func (p *MyProvider) Model(name string) (ai.Model, error) {
    // return a model implementation based on the name
}

func (p *MyProvider) ListModels() ([]string, error) {
    // return a list of available model names
}

func (p *MyProvider) Validate() error {
    // validate the provider configuration, e.g. check API key is set
}

Model Implementation:

type MyModel struct {
    // any configuration fields you need, e.g. model name, provider reference
    name string
}

func (m *MyModel) Name() string {
    return m.name
}

func (m *MyModel) Generate(ctx context.Context, req ai.AIRequest) (*ai.AIResponse, error) {
    // implement the logic to call your model API and return the response
}

func (m *MyModel) GenerateStream(ctx context.Context, req ai.AIRequest) <-chan ai.Token {
    // implement streaming token generation
}

func (m *MyModel) Close() error {
    // clean up any resources if needed
}

func (m *MyModel) Tokenizer() ai.Tokenizer {
    // return a tokenizer implementation for your model
}

Tokenizer Implementation:

type MyTokenizer struct{}

func (t MyTokenizer) ID() string {
    return "myprovider.my-model"
}

func (t MyTokenizer) Tokenize(ctx context.Context, text string) ([]string, error) {
    // split text into model-specific tokens
}

func (t MyTokenizer) CountTokens(ctx context.Context, text string) (int, error) {
    // return the token count for text
}

Now you can use your custom provider just like the built-in ones

</details>

🧰 Tool Interface

</summary>

Tools must implement:

type Tool interface {
    Name() string
    Description() string
    Params() string
    Function(req *ai.ToolCall) *ToolResponse
}

Tool calls are expected to arrive as JSON with this shape:

{
  "type": "function",
  "name": "tool_name",
  "arguments": {
    "some": "value"
  }
}

Tool call IDs are generated internally by the runtime and are not model-controlled.

</details>

<details>

<summary>

🧱 Package Layout

agent/       Reusable loop-backed agent definitions and built-in agents such as summary.
ai/          Core abstractions: Provider, Model, Tokenizer, AIRequest, AIResponse, ModelRepository. With implementations for Gemini and Mistral.
context/     Prompt building, token budgeting, dynamic sources, sessions, RAG, summaries, and message rendering.
loop/        Agent loop, tool parsing, tool execution helpers.
testutil/    Mocks used by tests.