GAI

📋 Requirements

• Go 1.26.x or newer
• API credentials for whichever provider you use

📦 Installation

go get github.com/lace-ai/gai

🧱 Prompt Builder

</summary>

context.New creates a Builder from a Definition. The definition supplies the renderer, system instructions, context sources, structured prompt input, token budget, output reserve, tokenizer, and optional debug sink.

builder := gaictx.New(gaictx.Definition{
  Renderer: &gaictx.XMLRenderer{},
  SystemInstructions: []gaictx.Part{
    gaictx.NewTextPart("Follow the system policy."),
  },
  ContextSources: []gaictx.ContextSource{
    history.NewHistory(sessionID, historyStore),
  },
  PromptInput: gaictx.PromptInput{
    User: gaictx.NewTextContent("Summarize the project status."),
  },
  TokenBudget:        128000,
  OutputTokenReserve: 4096,
  Tokenizer:          model.Tokenizer(),
})

The builder has two phases:

1. BuildContext calls each ContextSource in order. A source receives the remaining token budget and returns one Part.
2. BuildPrompt renders system instructions, built context sources, input context parts, current user content, and non-user messages from the loop conversation into one string.

Part is the unit of token counting and rendering. Built-in parts include TextPart, NamedPart, MessagePart, and SystemPart; NewJSONPart creates named structured JSON context. Custom parts implement Name, Tokens, and Render. XMLRenderer is the default, or provide another Renderer in the definition.

When TokenBudget is positive, the available context budget is TokenBudget - OutputTokenReserve - system instruction tokens. The active tokenizer is passed automatically to context sources that implement TokenizerSetter. A builder created by an agent.Agent also receives the agent tokenizer override, or the model tokenizer when no override is configured.

Use AppendSystemInstructions, AppendContextSource, and SetInput when the prompt needs to be changed after construction. SetDebugSink enables prompt-build diagnostics.

</details>

<details>

<summary>

🚀 Quick Start

🧭 Usage

The shortest path is to create a provider model, define an agent, and start a run. Import GAI's context package with an alias so it does not conflict with the standard library package.

provider := gemini.New(os.Getenv("GEMINI_API_KEY"), nil)
model, err := provider.Model("gemini-3-flash-preview")

assistant := agent.New(agent.Definition{
  Name:  "assistant",
  Model: model,
  Prompt: func(ctx context.Context, input agent.RunInput) (gaictx.PromptBuilder, error) {
	return gaictx.New(gaictx.Definition{
      SystemInstructions: []gaictx.Part{
        gaictx.NewTextPart("You are a concise, helpful assistant."),
      },
	}), nil
  },
})

workflow, err := assistant.NewRun(ctx, agent.RunInput{
  Prompt: gaictx.PromptInput{
    User: gaictx.NewTextContent("What is the capital of France?"),
  },
})

tokens, statuses, errs := workflow.Run(ctx)
go func() {
    for range statuses {
    }
}()
for token := range tokens {
    fmt.Print(token.Text)
}
for err := range errs {
    if err != nil {
        panic(err)
    }
}

Agents can include stream middleware in their definition. An input mapper can project the typed upstream workflow result into the ordinary RunInput accepted by the nested agent. This memory agent records its own output while passing the assistant response through unchanged.

memoryAgent := agent.New(agent.Definition{
  Name:  "memory",
  Model: model,
  Tools: []loop.Tool{saveMemoryTool},
  Prompt: func(ctx context.Context, input agent.RunInput) (gaictx.PromptBuilder, error) {
    return memoryPrompt(input), nil
  },
})

assistant := agent.New(agent.Definition{
  Name:   "assistant",
  Model:  model,
  Prompt: assistantPrompt,
  Middleware: []agent.Middleware{
    agent.NewAgentMiddleware(memoryAgent, agent.AgentMiddlewareConfig{
      Output:      agent.PreserveOutput,
      ErrorPolicy: agent.RecordError,
      MapInput: func(ctx context.Context, result agent.WorkflowResult) (agent.RunInput, error) {
		observation, err := buildMemoryObservation(ctx, result)
        if err != nil {
          return agent.RunInput{}, err
        }
		observationPart, err := gaictx.NewJSONPart("memory_observation", observation)
		if err != nil {
		  return agent.RunInput{}, err
		}
		return agent.RunInput{
		  ID: result.Input.ID,
		  Prompt: gaictx.PromptInput{
		    Context: []gaictx.Part{observationPart},
		  },
		  Meta: result.Input.Meta,
		}, nil
      },
    }),
  },
})

workflow, err := assistant.NewRun(ctx, agent.RunInput{
  Prompt: gaictx.PromptInput{
    User: gaictx.NewTextContent("What is the capital of France?"),
  },
  Meta: map[string]any{"session_id": sessionID},
})
tokens, statuses, errs := workflow.Run(ctx)

go func() {
  for range statuses {
  }
}()
for token := range tokens {
  fmt.Print(token.Text)
}
for err := range errs {
  if err != nil {
    panic(err)
  }
}

result := workflow.Result()
fmt.Printf("memory stage output: %s\n", result.Stages[0].Result.Text)

For a single non-agent request, call the model directly with model.Generate(ctx, ai.AIRequest{Prompt: "...", MaxTokens: 100}).

🧰 Tool Interface

</summary>

Tools must implement:

type Tool interface {
    Name() string
    Description() string
    Params() ai.ToolParameters
    Function(ctx context.Context, req *ai.ToolCall) *ToolResponse
}

Params returns a structured object schema. The runtime serializes it to JSON Schema for provider-native tool calling and for the prompt fallback:

func (t *SaveMemoryTool) Params() ai.ToolParameters {
    return ai.ToolParameters{
        Strict: true,
        Properties: []ai.ToolParameter{
            {
                Name:        "memory",
                Type:        ai.ToolParameterString,
                Description: "The memory text to save.",
                Required:    true,
            },
        },
    }
}

Tool calls are expected to arrive as JSON with this shape:

{
  "type": "function",
  "name": "tool_name",
  "arguments": {
    "some": "value"
  }
}

Tool call IDs are generated internally by the runtime and are not model-controlled. When a provider supports native tool calling, the loop sends tool definitions through AIRequest.Tools. Prompt-rendered JSON tool calls are retained as a fallback compatibility protocol.

</details>

<details>

<summary>

🧱 Package Layout

agent/          Reusable agent definitions and built-in components such as the summary agent.
ai/             Provider, model, tokenizer, request, and response abstractions, plus Gemini and Mistral.
context/        Prompt construction, parts, rendering, messages, and conversation interfaces.
context/history Persisted history sources and optional history summarization.
loop/           Model/tool execution loop and tool helpers.
testutil/       Mocks used by tests.

🤖 Agent Components

</summary>

The agent package turns reusable configuration into independent loop runs:

• Definition combines a name, model, tools, prompt factory, limits, optional tokenizer override, optional tool-response preprocessor, and ordered stream middleware.
• Agent tools are exposed automatically through a tool_definitions source placed before application context sources.
• Prompt builds a context.PromptBuilder for one RunInput.
• RunInput carries an ID, structured PromptInput, per-run output-token override, and metadata. PromptInput separates genuine user content from named machine context.
• Limits configures maximum loop iterations, retries, and default output tokens.
• Agent is created with agent.New; NewRun returns a Workflow, and Workflow.Run streams the loop through each configured middleware.
• AgentMiddleware adapts an ordinary agent with preserve, append, or replace output behavior. Workflow.Result exposes the primary result, final output, and named middleware stages.

The agent/summary package is a built-in component. summary.Definition returns a reusable agent definition, while summary.New returns a Summarizer that runs that agent and can be attached to a history source.

</details>

<details>

<summary>

🔗 Agent Workflows and Middleware

</summary>

Agent.NewRun creates a single-use Workflow. Calling Workflow.Run starts the primary loop and passes its stream through every entry in Definition.Middleware in declaration order. Callers must consume the token, status, and error channels. After they close, Workflow.Result() contains the immutable primary result, the final visible output, accumulated errors, and named middleware stages.

An ordinary agent can become middleware with NewAgentMiddleware. By default it receives the current visible text as named upstream_output context, plus the original run ID and metadata. Set AgentMiddlewareConfig.MapInput to deliberately project the typed upstream WorkflowResult into a different RunInput:

- PreserveOutput streams the upstream output unchanged and keeps the nested agent result only in WorkflowResult.Stages. - AppendOutput streams the upstream output, then emits the nested agent output after that stage completes successfully. - ReplaceOutput buffers the upstream output and emits the nested agent output only after the stage completes successfully.

Failed append and replace stages leave the upstream output unchanged. An agent used through NewAgentMiddleware cannot define middleware of its own; compose stages on the parent workflow instead.

Agent middleware runs only after a successful upstream result by default. Set AgentMiddlewareConfig.ShouldRun to implement policies such as failure auditing. Nested-agent errors are sent through the workflow error channel by default. Set ErrorPolicy: RecordError for best-effort stages whose failures should remain in StageResult.Result.Errors without failing the surrounding workflow.

For transformations that do not require another agent, use MiddlewareFunc:

passthrough := agent.MiddlewareFunc(func(
  ctx context.Context,
  run *agent.MiddlewareContext,
  upstream agent.Stream,
) agent.Stream {
  // A custom middleware owns all three streams. Returning upstream is a
  // zero-overhead pass-through; a transformer may return replacement channels.
  return upstream
})

assistant := agent.New(agent.Definition{
  Name:       "assistant",
  Model:      model,
  Prompt:     assistantPrompt,
  Middleware: []agent.Middleware{passthrough},
})

MiddlewareContext.Result() returns a concurrency-safe snapshot for custom middleware. Complete becomes true only after the final workflow streams close.

Set Definition.DebugSink to receive structured agent lifecycle events for run creation, workflow start/completion, primary completion, and middleware start/skip/success/failure. Agent execution also emits agent.run.create, agent.workflow.run, and agent.middleware.run OpenTelemetry spans. Event payloads contain counts and policy names by default; input and output text are included only when DebugSink.IncludeSensitiveData() returns true.

</details>

<details>

<summary>