![agents-exe logo](docs/logo-01.png)

# an LLM-agent tool

A handy LLM-agent tool, with a variety of calling and configuration modes so
that the LLM adapts to your workflow rather than the opposite.

## Architecture Overview

Agents-exe uses a layered architecture with a modern Entity-Component-System (ECS) based OS model at its core:

```
┌────────────────────────────────────────────────────────────────┐
│                    Interface Layer                              │
│  (CLI commands, TUI, MCP server, HTTP endpoints)                │
├────────────────────────────────────────────────────────────────┤
│                    OS Model Layer                               │
│  (Entity-Component-System, Resource Management,                 │
│   Conversation Tracking, Concurrent Access)                     │
├────────────────────────────────────────────────────────────────┤
│                    Agent Tree Layer                             │
│  (multi-agent hierarchy, reference validation, cycle detection) │
├────────────────────────────────────────────────────────────────┤
│                    Foundation Layer                             │
│  (sessions, tools, LLM integration, file loading)               │
└────────────────────────────────────────────────────────────────┘
```

### OS Model Benefits

- **Shared Toolboxes**: Multiple agents can share the same SQLite database, HTTP connection pool, or other resources
- **Better Resource Management**: Explicit lifecycle scopes with predictable cleanup
- **Concurrent Access**: STM-based synchronization for safe multi-agent operations
- **Durable Persistence**: Pluggable persistence layer (SQLite, PostgreSQL, file-based)
- **Complete Lineage Tracking**: Full call chains for debugging and accounting

See [docs/architecture.md](docs/architecture.md) for detailed architecture documentation and [docs/MIGRATION-OS.md](docs/MIGRATION-OS.md) for migration guidance.

## Quick Start

Getting started:

```console
agents-exe init
```

will guide you into writing an `agent.json` and its tools directory.

## Command Reference

### Global Options

These options apply to all commands:

| Option | Default | Description |
|--------|---------|-------------|
| `--api-keys FILE` | `~/.config/agents-exe/secret-keys` | Path to JSON file containing API keys |
| `--log-file LOGFILE` | `agents-logfile` | Raw log file for debugging |
| `--log-http URL` | (none) | HTTP endpoint for JSON log sink |
| `--log-json-file FILE` | (none) | Local JSON file log sink |
| `--session-json-file-prefix PREFIX` | (none) | Prefix for session JSON files |
| `--agent-file AGENTFILE` | (auto-discovered) | Root agent description file(s). Can be specified multiple times. Defaults to `agent.json` |

### Commands

#### `check`

Validates agent configuration files.

```console
agents-exe check
agents-exe check --agent-file custom-agent.json
```

#### `init`

Initializes a new agent project with a sample `agent.json` and tools directory structure.

```console
agents-exe init
agents-exe init --agent-file my-agent.json
```

#### `tui` - Terminal User Interface

Launches an interactive terminal UI for chatting with agents.

```console
agents-exe tui
agents-exe tui --agent-file my-agent.json
```

#### `run` - Execute a One-Shot Command

Runs a single prompt through an agent and outputs the result.

**Prompt Options:**

| Option | Short | Description |
|--------|-------|-------------|
| `--prompt TEXT` | `-p` | Add a text paragraph to the prompt |
| `--file FILE` | `-f` | Include contents of a file in the prompt |
| `--shell CMD` | | Include stdout of a shell command in the prompt |
| `--sep4 SEP` | `-s` | Add a short separator (4 repetitions) |
| `--sep40 SEP` | `-S` | Add a long separator (40 repetitions) |
| `--session-file FILE` | | Resume from or save to a session file |

**Examples:**

```console
# Simple prompt
agents-exe run --prompt "Hello, how are you?"

# Include file contents
agents-exe run --prompt "Summarize this:" --file README.md

# Include command output
agents-exe run -p "Explain this diff:" -s "---" --shell "git diff"

# Multiple prompts (concatenated with newlines)
agents-exe run -p "First part" -p "Second part"

# Resume a session
agents-exe run --session-file previous.json -p "Continue where we left off"
```

#### `echo-prompt`

Echoes the constructed prompt without sending it to the agent. Useful for debugging prompt construction.

```console
agents-exe echo-prompt -p "Hello" -f README.md
```

#### `describe`

Outputs a JSON self-description of the tool interface. Useful when agents-exe is used as a sub-tool.

```console
agents-exe describe
```

#### `mcp-server`

Runs agents-exe as an MCP (Model Context Protocol) server, exposing agents as tools via stdin/stdout.

```console
agents-exe mcp-server
agents-exe mcp-server --agent-file agent1.json --agent-file agent2.json
```

The MCP server exposes each configured agent as a callable tool to MCP clients like Claude Desktop.

#### `session-print`

Prints a session file in human-readable markdown format.

| Option | Description |
|--------|-------------|
| `--show-tool-call-results` | Show the results of tool calls |
| `--n-turns N` | Limit output to first N turns |
| `--repeat-system-prompt` | Repeat system prompt at each turn |
| `--repeat-tools` | Repeat available tools at each turn |
| `--antichronological` | Display newest turns first (default: oldest first) |

**Examples:**

```console
# Print entire session
agents-exe session-print my-session.json

# Print with tool results
agents-exe session-print --show-tool-call-results my-session.json

# Print last 10 turns only, newest first
agents-exe session-print --n-turns 10 --antichronological my-session.json
```

## Using as a Tool

The primary intended usage is to run agents-exe as a standalone application.

### Agents

An agent comprises a repeatable, parameterized LLM.

An agent requires:
- An LLM endpoint
- An API key for the endpoint
- A model
- A system prompt
- A set of command line tools
- A set of helper agents

The model and system-prompt are the typical LLM parameters you encounter.
Large, general models are more capable but more expensive. The system-prompt
gives instructions to align and guide the LLM output.

Command-line tools extend the capabilities of your LLM by performing side
effects from `agents-exe` (for instance, installing packages, probing the
network, reading files, etc).

Helper agents allow you to offload a part of the reasoning to more specialized
agents. This mechanism allows the main LLM to be responsible for coming up with a
solution and prompting other agents. The set of helper agents is also
referenced in a paragraph added to the system prompt so that you can reference
agents by name in your user prompts.

Both command line tools and helper agents are exposed to the model as extra
functions (or tools). If the LLM wants to run a function, `agents-exe` will do
so. When an LLM requires more than one function, `agents-exe` runs all the
functions concurrently.

### Adding New Executable-Program Tools

Tools are executable programs (often bash scripts) that adhere to the following specifications:
- When called with `describe` (no parameter), the program MUST return a description of the parameters allowed
- When called with `run` (with arguments/stdin), the program SHOULD always write something to stdout

The description format is a JSON-encoded object with the following structure:

```json
{
  "slug": "tool_name",
  "description": "What this tool does",
  "args": [
    {
      "name": "arg_name",
      "description": "Argument description",
      "type": "string",
      "backing_type": "string",
      "arity": "single",
      "mode": "dashdashspace"
    }
  ]
}
```

**Argument Modes:**

| Mode | Behavior |
|------|----------|
| `positional` | Added as consecutive arguments |
| `dashdashspace` | Added as `--arg-name value` (two arguments) |
| `dashdashequal` | Added as `--arg-name=value` (single argument) |
| `stdin` | Concatenated to stdin with blank line separators |

See `System.Agents.Tools.Bash.ScriptInfo` for the proper definition.

Agents-exe adheres to the same specifications, allowing you to use an
`agents-exe` invocation directly as a tool. This flexibility is useful when
running a hierarchy of agents with different access rights or across container
boundaries.

### Adding New Tools Using an MCP Server

Agents-exe can act as an MCP client of MCP servers.
For now, only the local-executable transport is supported.

Adding executables is done via the agent configuration:

```json
{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "my-agent",
    "mcpServers": [
      {
        "tag": "McpSimpleBinary",
        "contents": {
          "name": "my-mcp-server",
          "executable": "/path/to/executable",
          "args": ["arg0", "arg1"]
        }
      }
    ]
  }
}
```

For MCP servers which do not advertise tools-changed notifications, agents-exe
resorts to periodic polling to refresh the tools list.

### Adding New Agents

Agents defined in `agents-exe` can call other agents much like tools (and
indeed, sub-agents are exposed as "expert tools" to LLMs). Agents and
sub-agents can share the same API key but they do not have to.

Sub-agents can have further agents themselves. They are loaded from JSON files
in the tool-directory (relative to the agent's JSON file parent directory).

A typical config-tree for an agent:

```
/some-dir           # the agent root dir
  /agent.json       # the agent-definition file
  /tools            # the tool dir
    /aloha.json     # some tool
    /expert-1.json  # sub-agent definition
    /expert-1-tools # sub-agent's tool dir
      /...
    /ping.sh        # some bash tool
```

### API Keys

Agents need an API key to authenticate against the LLM endpoint.
By default `agents-exe` locates keys in `~/.config/agents-exe/secret-keys` but you can override with `--api-keys`.

The keys file format:

```json
{
  "keys": [
    {
      "id": "my-key-id",
      "value": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    }
  ]
}
```

The `id` must match the `apiKeyId` field in the agent's JSON description.

### Environment Variables

- `EDITOR` or `GIT_EDITOR` - Used for editing files (notably in the `init` command)

## Using as an MCP Server

The `agents-exe mcp-server` command runs a (stdin/stdout) MCP server exposing
agents as individual tools. It works with Claude Desktop and other MCP clients.

Tool names are deterministic across runs when using the same `--agent-file` arguments in the same order.

## Using as a Library

The code is designed to be usable as a Haskell library.

### Example: Using the OS Model

```haskell
import System.Agents.OS
import System.Agents.OS.Core
import System.Agents.OS.Persistence

main :: IO ()
main = do
    -- Initialize the OS with SQLite persistence
    os <- initializeOS defaultConfig
    backend <- createPersistenceBackend (SqliteBackendType "./agents.db")
    
    -- Create a shared SQLite toolbox
    tbConfig <- createToolboxConfig SqliteToolboxType "shared-db" 
        (object ["path" .= "./data.db"])
    tbId <- createToolbox os tbConfig
    
    -- Create agents that share the toolbox
    agent1 <- createAgent os (testAgentConfig 
        { agentToolboxBindings = [tbId] })
    agent2 <- createAgent os (testAgentConfig 
        { agentToolboxBindings = [tbId] })
    
    -- Run conversations
    runOSM os $ do
        conv1 <- startConversation agent1 defaultConvConfig
        result1 <- runConversationTurn conv1 "Hello, agent 1!"
        
        conv2 <- startConversation agent2 defaultConvConfig
        result2 <- runConversationTurn conv2 "Hello, agent 2!"
        
        -- Both agents can access the same database!
        pure (result1, result2)
```

See [docs/OS-API.md](docs/OS-API.md) for the complete API reference.

### Coding Style Guidelines

- Prefer qualified imports over prefixed/suffixed functions
- Keep prefixed/suffixed function names for helper variations
- Prefer passing multiple arguments
- Bundle arguments into Runtime or Props objects for long-running values or context
- No monad-tower, run in IO (library users can add their own)
- Use contravariant tracer to surface logs, be generous with traces

# agq — Automating GitHub Issue Queues

`agq` is a companion tool (see [`agq/README.md`](agq/README.md)) that turns
GitHub issues into an agent-driven task queue with DAG dependency enforcement.

## Typical Automation Loop

A cron job or CI step running every few minutes is enough to drive the full
cycle:

```console
# 1. Promote blocked issues whose dependencies are now satisfied
agq promote

# 2. Import ready issues into the local task queue
agq pull

# 3. Run pending tasks (each spins up agents-exe in a git worktree)
agq process

# 4. Merge agent PRs that target feature branches;
#    labels closed issues with agq/done-in-branch so promote() unblocks dependents
agq merge-prs
```

Steps 1–4 are intentionally idempotent — re-running them is always safe.

## Feature-branch Chains

When issues A → B → C are all destined for a shared feature branch, GitHub
will not auto-close A when the PR for A merges into that branch (only merges
into the default branch trigger auto-close). `agq merge-prs` bridges this gap:
after merging a feature-branch PR it parses its `Closes #N` lines and labels
those issues `agq/done-in-branch`. `agq promote` treats that label as
equivalent to a closed issue, so B is unblocked automatically.

# Build and Run

## Directly from Source

```console
git clone https://github.com/lucasdicioccio/agents-exe
cd agents-exe
cabal run -- agents-exe --help
cabal run -- agents-exe init
```

## Running Tests

```console
cabal test
```

For OS-specific tests:
```console
cabal test --test-option=--pattern="OS"
```

## Container Build

With [Podman](https://podman.io) (for [Docker](https://docker.com), replace with `docker`):

```console
git clone https://github.com/lucasdicioccio/agents-exe
cd agents-exe/bundling
podman build -f Containerfile.build -t agents-exe
podman run -it --entrypoint=bash agents-exe:latest
```

Then inside the container:

```console
agents-exe init
agents-exe tui
```

## JSON Logging

Agents-exe logs extensively. The default "Show Format" is for debugging.
For mechanical consumption, use JSON logging:

```console
agents-exe --log-json-file logs.json run -p "Hello"
agents-exe --log-http https://example.com/log-sink run -p "Hello"
```

## Advanced Configuration

Create an `agents-exe.cfg.json` in your project root (searched upward from current directory):

```json
{
  "agentsConfigDir": "/some/dir",
  "agentsDirectories": ["./agents"],
  "agentsFiles": [],
  "agentsLogs": {
    "logJsonHttpEndpoint": "https://example.org/log-sink",
    "logRawPath": "agents-exe.raw",
    "logJsonPath": "logs.json",
    "logSessionsJsonPrefix": "sessions"
  }
}
```

Absent this file, agents load from `~/.config/agents-exe/default`.

# Documentation

- [Architecture Overview](docs/architecture.md) - Detailed architecture documentation
- [OS API Reference](docs/OS-API.md) - Complete API documentation for the OS model
- [Migration Guide](docs/MIGRATION-OS.md) - Guide for migrating from old Runtime to new OS model
- [CLI Commands](docs/cli-commands.md) - Detailed CLI documentation
- [TUI Guide](docs/tui.md) - Terminal UI documentation
- [MCP Documentation](docs/mcp.md) - Model Context Protocol integration

# Roadmap

## LLM Providers

- [x] Mixed-API keys between agents and sub-agents
- [x] OpenAI
- [x] Mistral
- [x] Ollama
- [x] Moonshot
- [ ] vLLM

## MCP Support

- [ ] Server: expose tool registration directly
- [ ] Server: forward tool-reloads/notifications
- [ ] Server: expose completion/prompt/resources
- [ ] Networked transport

## Framework Features

- [ ] HTTP server with metrics
- [ ] Better async-linking
- [ ] CLI tooling to directly inspect/call tools
- [ ] Improve TUI mode

## OS Model

- [x] Core ECS types and world management
- [x] Resource management with lifecycle scopes
- [x] Concurrent access patterns (exclusive, read-write, pool)
- [x] Conversation and lineage tracking
- [x] Persistence layer (SQLite, file-based)
- [x] Runtime compatibility layer
- [ ] Full TUI/OneShot OS integration
- [ ] Web API foundation
- [ ] Distributed OS (multi-node support)

# Credits

Agents-exe is built on the shoulders of giants. We are grateful to the maintainers and contributors of the following open-source libraries:

## Core & Data
- **[base](https://hackage.haskell.org/package/base)** - The Haskell standard library
- **[bytestring](https://hackage.haskell.org/package/bytestring)** - Fast, compact binary strings
- **[text](https://hackage.haskell.org/package/text)** - Efficient Unicode text handling
- **[containers](https://hackage.haskell.org/package/containers)** - Maps, sets, and sequences
- **[vector](https://hackage.haskell.org/package/vector)** - Efficient arrays
- **[unordered-containers](https://hackage.haskell.org/package/unordered-containers)** - Hash-based containers
- **[hashable](https://hackage.haskell.org/package/hashable)** - Hashing for data types

## JSON & Serialization
- **[aeson](https://hackage.haskell.org/package/aeson)** - JSON parsing and generation
- **[aeson-pretty](https://hackage.haskell.org/package/aeson-pretty)** - Pretty-printing JSON
- **[yaml](https://hackage.haskell.org/package/yaml)** - YAML parsing
- **[scientific](https://hackage.haskell.org/package/scientific)** - Arbitrary precision numbers

## Concurrency & STM
- **[async](https://hackage.haskell.org/package/async)** - Asynchronous operations
- **[stm](https://hackage.haskell.org/package/stm)** - Software Transactional Memory
- **[stm-chans](https://hackage.haskell.org/package/stm-chans)** - STM channels
- **[stm-conduit](https://hackage.haskell.org/package/stm-conduit)** - STM with streaming
- **[unliftio](https://hackage.haskell.org/package/unliftio)** - Unlifted IO operations

## HTTP & Networking
- **[http-client](https://hackage.haskell.org/package/http-client)** - HTTP client engine
- **[http-client-tls](https://hackage.haskell.org/package/http-client-tls)** - TLS support for http-client
- **[http-types](https://hackage.haskell.org/package/http-types)** - HTTP types
- **[network-uri](https://hackage.haskell.org/package/network-uri)** - URI parsing

## Database & Storage
- **[sqlite-simple](https://hackage.haskell.org/package/sqlite-simple)** - SQLite bindings
- **[direct-sqlite](https://hackage.haskell.org/package/direct-sqlite)** - Low-level SQLite interface
- **[resource-pool](https://hackage.haskell.org/package/resource-pool)** - Connection pooling

## Terminal UI
- **[brick](https://hackage.haskell.org/package/brick)** - Terminal UI framework
- **[vty](https://hackage.haskell.org/package/vty)** - Terminal interaction
- **[text-zipper](https://hackage.haskell.org/package/text-zipper)** - Text editing for TUI

## Lua Integration
- **[hslua](https://hackage.haskell.org/package/hslua)** - Lua bindings for Haskell
- **[hslua-marshalling](https://hackage.haskell.org/package/hslua-marshalling)** - Lua marshalling utilities

## Streaming
- **[conduit](https://hackage.haskell.org/package/conduit)** - Streaming data library
- **[conduit-extra](https://hackage.haskell.org/package/conduit-extra)** - Additional conduit utilities

## CLI & Command Line
- **[optparse-applicative](https://hackage.haskell.org/package/optparse-applicative)** - Command-line option parsing

## Functional Programming
- **[lens](https://hackage.haskell.org/package/lens)** - Functional references
- **[mtl](https://hackage.haskell.org/package/mtl)** - Monad transformer library
- **[free](https://hackage.haskell.org/package/free)** - Free monads
- **[contravariant](https://hackage.haskell.org/package/contravariant)** - Contravariant functors

## Utilities
- **[filepath](https://hackage.haskell.org/package/filepath)** - File path handling
- **[directory](https://hackage.haskell.org/package/directory)** - Directory operations
- **[process](https://hackage.haskell.org/package/process)** - Process spawning
- **[temporary](https://hackage.haskell.org/package/temporary)** - Temporary files
- **[uuid](https://hackage.haskell.org/package/uuid)** - UUID generation
- **[time](https://hackage.haskell.org/package/time)** - Time handling
- **[base64-bytestring](https://hackage.haskell.org/package/base64-bytestring)** - Base64 encoding
- **[regex-tdfa](https://hackage.haskell.org/package/regex-tdfa)** - Regular expressions

## Testing & Benchmarking
- **[tasty](https://hackage.haskell.org/package/tasty)** - Testing framework
- **[tasty-hunit](https://hackage.haskell.org/package/tasty-hunit)** - HUnit integration for tasty
- **[criterion](https://hackage.haskell.org/package/criterion)** - Benchmarking framework
