日志分析工具

Install (core only — no external dependencies beyond PyYAML and typer)

pip install logatory

Build the vector index first (requires pip install 'logatory[embed]')

logatory llm index

Installation

Requirements: Python 3.11+

Docker container logs

No log aggregation stack (ELK, Loki, Graylog) required — if your services run in Docker, Logatory reads their logs straight from the daemon. Install the optional dependency and use the native docker command:

```bash pip install 'logatory[docker]'

Install and start Ollama: https://ollama.ai

ollama pull gemma3:4b

Docker

Environment variables for Docker

```bash

docker-compose.yml (or .env file)

LOGATORY_API_TOKEN=change-me-in-production LOGATORY_PII_SALT=a-long-random-string ```

Build and run manually

docker build -t logatory .

docker run -d \
  -p 8080:8080 \
  -v logatory-data:/data \
  -e LOGATORY_API_TOKEN=mytoken \
  -e LOGATORY_PII_SALT=mysalt \
  logatory

The container runs as a non-root user (logatory, UID 1001). The database and config are stored in /data.

Scanning log files inside Docker

Mount the host log directory and run a one-shot scan:

docker run --rm \
  -v /var/log:/logs:ro \
  -v logatory-data:/data \
  logatory \
  logatory scan /logs/syslog --track-errors

Quick Start

```bash

Quick start

docker compose up -d

The stack starts Logatory on port 8080 with a named volume for the SQLite database.

demo

Interactive demo and database seeding using synthetic data — no real log files, Ollama, or database required for demo run.

logatory demo [run|seed|clear]

demo run

Guided CLI walkthrough of all 7 feature sections (log parsing, PII, rules, error tracking, findings, anomaly detection, LLM):

logatory demo run           # pause after each section
logatory demo run --no-pause  # print everything at once

demo seed

Populate the SQLite database with synthetic findings and errors so the web dashboard has something to display immediately. Inserts 25 findings spread over 14 days (for the trend chart) and 5 error groups. All records are tagged internally and never mixed with real data.

logatory demo seed

demo clear

Remove every record written by demo seed. Real findings and errors are never touched.

logatory demo clear

---

Demo data for the web dashboard

Seed the database with synthetic findings and errors so the dashboard shows data immediately:

```bash

Remove all demo data (real data is untouched)

docker compose exec logatory logatory demo clear ```

Alternatively, upload a real log file via the browser at http://localhost:8080/upload for an instant, transient scan.

---

Optional feature sets

pip install 'logatory[web]'         # web dashboard + REST API (FastAPI, uvicorn, Jinja2)
pip install 'logatory[docker]'      # read logs from local Docker containers
pip install 'logatory[opensearch]'  # OpenSearch / Elasticsearch integration
pip install 'logatory[xlsx]'        # read .xlsx spreadsheet log exports
pip install 'logatory[claude]'      # Anthropic Claude API
pip install 'logatory[embed]'       # ChromaDB for RAG (llm ask command)

Install everything:

pip install 'logatory[web,docker,opensearch,xlsx,claude,embed]'

List the configured targets; --check probes each for reachability

logatory fleet list --check

Configuration

Run logatory init to generate a config.yaml (with a freshly generated PII salt), or copy config.yaml.example and adapt. When no --config is passed, Logatory looks for a config in $LOGATORY_CONFIG, ./config.yaml, then ~/.config/logatory/config.yaml.

logatory init                              # write ./config.yaml
logatory init --minimal                    # smaller starter config
logatory init -o ~/.config/logatory/config.yaml

Run logatory doctor to verify your setup — it checks that the config loads, a PII salt is set, the database directory is writable, the LLM provider is configured/reachable (and that cloud API keys are present), plugins load, and alert channels build. It exits non-zero on hard failures, so it works in CI too.

logatory doctor
logatory doctor --config ~/.config/logatory/config.yaml

```yaml

Custom PII patterns file (optional)

pii_rules_path: pii_rules.yaml

Prefer env var LOGATORY_PII_SALT over storing here

pii_salt: ""

Prefer env var LOGATORY_API_TOKEN

api_token: ""

Environment variables

---

Default config already points to http://localhost:11434

logatory llm info ```

config.yaml

llm: provider: claude model: claude-3-5-haiku-20241022

CLI Reference

All commands accept --config/-c <path> to specify a config file. Without it, Logatory auto-discovers one from (in order) $LOGATORY_CONFIG, ./config.yaml, then ~/.config/logatory/config.yaml; if none exist, built-in defaults are used.

---

REST API Bearer token — leave empty to disable auth (local dev)

OpenAI-compatible APIs

llm:
  provider: openai
  model: gpt-4o-mini
  endpoint: https://api.openai.com/v1

export OPENAI_API_KEY=sk-...

Web Dashboard & REST API

Start the server (requires pip install 'logatory[web]'):

logatory serve --port 8080

REST API v1

Base path: /api/v1/ Interactive docs: /api/docs

Authentication

Set api_token in config.yaml or via LOGATORY_API_TOKEN. Pass it as:

Authorization: Bearer <token>

Leave empty to disable auth (for local development or Docker with network-level access control).

Event ingestion example

curl -X POST http://localhost:8080/api/v1/events \
  -H "Authorization: Bearer mytoken" \
  -H "Content-Type: application/json" \
  -d '{"raw": "Failed password for root from 1.2.3.4 port 22", "source": "sshd"}'

Plugin directory — all *.py files here are auto-loaded at startup

plugins_dir: plugins/

Plugin System

Drop Python files into a directory and register custom rules, PII patterns, log-format parsers and source adapters. Enable in config.yaml:

plugins_dir: plugins/

A plugin file must expose a register(registry) function:

```python

plugins/my_plugin.py

def register(registry) -> None: # Custom detection rule registry.add_rule({ "id": "MY_DB_LEAK", "title": "Database credentials exposed in log", "description": "Fires when a connection string appears in a log message.", "level": "critical", "detection": { "match": [ {"field": "message", "op": "re", "value": r"postgresql://\S+:\S+@"}, ] }, })

# Custom PII pattern — redacts internal employee IDs registry.add_pii_pattern( name="employee_id", pattern=r"\bEMP-\d{4,8}\b", prefix="employee", )

# Load an entire directory of YAML rule files from pathlib import Path registry.add_rule_dir(Path(file).parent / "my_rules")

# Custom log-format parser — auto-detected like any built-in format # registry.add_parser(name="myfmt", detect=looks_like_myfmt, factory=MyParser)

# Custom source adapter — looked up by name like any built-in source # registry.add_adapter(name="kafka", adapter_cls=KafkaAdapter) ```

Plugin rules participate in logatory scan, logatory tail, and the web dashboard rule engine. Plugin PII patterns apply to every redaction pass; plugin parsers and adapters register into the global parser/adapter registries, so format auto-detection and source lookup pick them up everywhere. A plugin that raises an exception is logged as a warning and skipped — it never crashes the host process.

A complete, runnable example covering all four contribution types lives in plugins/example_plugin.py, and the full guide is in docs/PLUGINS.md.

---

LLM Integration

Free-form question; the agent gathers evidence with tools as needed

logatory agent ask "Why did the API start failing this afternoon?"