# AiSOC — full description

<!-- Generated by scripts/generate_llms_txt.py. Do not edit by hand; run the script and commit the result. -->

> AiSOC is an open-source, self-hostable AI Security Operations Center, MIT
> licensed and maintained by the AiSOC community. Version 20.1.0.
> Repository: https://github.com/beenuar/AiSOC. Documentation: https://beenuar.github.io/AiSOC/.

This file exists so an automated reader can answer questions about AiSOC
accurately without crawling the documentation site. Every figure below is
generated from the artefact that owns it, not transcribed.

## What AiSOC is

A complete Security Operations Center platform. Security telemetry arrives from
connectors or webhooks, is normalised to the Open Cybersecurity Schema Framework
(OCSF), archived to an event lake, evaluated against a detection corpus,
correlated into alerts, and handed to LLM agents that triage and investigate.
Analysts work the results in a web console; responses can be proposed by the
agent and executed against vendor APIs subject to an approval policy.

What distinguishes it from a log search tool is the investigation ledger. Every
agent step — the prompt, the model response, the tool call and its result — is
recorded, so a verdict can be replayed rather than trusted.

## What AiSOC is not

- It is not a managed service. Self-hosting is the primary deployment model.
- It is not credential-gated. `make up` runs without any API key.
- It does not claim autonomous response by default. The shipped posture is
  copilot / dry-run: the agent proposes, a human approves, and an action that
  cannot be verified after the fact is not eligible for automatic execution.

## Architecture

The production event path is:

    connectors -> ingest (OCSF normalise) -> Kafka `raw_events`
      -> fusion (lake archive, detection, correlation, alert promotion)
      -> Postgres alerts + realtime WebSocket
      -> agents (auto-triage every fused alert)

Services in the monorepo under `services/`: `api` (FastAPI), `agents`
(LangGraph orchestration), `fusion` (correlation and detection), `ingest` (Go),
`connectors`, `realtime`, `threatintel`, `enrichment`, `ocsf`, `ueba`,
`actions`, `mcp`, and others. Applications under `apps/`: `web` (Next.js
console) and `docs` (this documentation site).

`connectors` and `actions` run in the core profile, not only in `full`. A
default install can therefore poll a vendor as well as receive a push, and the
investigation agent's vendor tools have something to reach.

Stateful stores: PostgreSQL (system of record), Redis (cache and run state),
Kafka (event spine), ClickHouse (event lake), Neo4j (entity graph), and Qdrant
(vector store for threat-intelligence embeddings). Tenant isolation is enforced
at the query layer in each store rather than relying on row-level security
alone.

Read more: https://beenuar.github.io/AiSOC/docs/architecture

## Where the entry points are

If you are answering a question about how something works, these are the files
to open first. Each is the real production path, not a wrapper.

- **HTTP ingest:** `services/ingest/internal/server/server.go`. Normalization
  is `services/ingest/internal/normalizer/` — `normalizer.go` holds the
  per-connector profile table and `ocsf_classes.go` the OCSF class per
  connector type.
- **Detection:** `services/fusion/app/services/detection_engine.py` and
  `detection_matcher.py`. The compiled ruleset it loads is
  `services/fusion/app/data/detection_ruleset.json`. Rules that need a time
  window are `windowed_detection.py`.
- **Promotion and correlation:** `services/fusion/app/services/promoter.py`
  decides whether an event becomes an alert; `correlator.py` decides which
  incident it joins.
- **Agent triage:** `FusedAlertTriageWorker` in `services/agents`. Deeper
  investigation is `services/agents/app/investigator/`.
- **The REST API:** `services/api/app/api/v1/endpoints/`. The committed
  specification is `docs/openapi.yaml`.
- **Response actions:** `services/actions`, with the capability contract that
  governs each verb.
- **Playbooks:** `services/agents/app/playbook/` — `models.py` holds the step
  vocabulary, `engine.py` runs it.
- **The console:** `apps/web/src/app/`.

Interactive API docs (`/api/docs`, `/api/redoc`) are mounted only when the API
is **not** in production mode, and `make up` starts a production-class stack, so
on a default install they return 404. Read `docs/openapi.yaml` instead.

## Which directory holds what

```
apps/web/          Next.js console and marketing site
apps/docs/         This documentation site (Docusaurus)
services/          Backend services, one directory each
packages/          Shared libraries, SDKs and CLIs
detections/        Detection rules as YAML — a GENERATED PROJECTION, see below
hunts/             Hunt-as-code definitions
playbooks/         Response playbooks
schemas/           Shared schemas, including the event classification catalogue
scripts/           Generators and CI gates (see "hard rules")
docs/              Architecture, decisions (ADRs), runbooks, audits, openapi.yaml
infra/             Compose, Helm, Terraform and per-PaaS deployment recipes
plans/             Historical prototype subtree — reference only, never edited
```

## Naming conventions worth knowing before you guess

- **Connector identifiers** are the ids declared in
  `services/connectors/app/connectors/__init__.py` — `crowdstrike`, not
  `crowdstrike_falcon`. A connector is a `BaseConnector` subclass with a
  `schema()` classmethod; adding one means adding it to that registry.
- **Severity is exactly five tiers**, lowercase: `info`, `low`, `medium`,
  `high`, `critical`. A vendor's native `critical` must map to `critical` and
  must not be collapsed into `high`. Confidence is separate: an integer 0–100.
- **The event envelope key is `raw_event`**, never `raw`. A connector emitting
  `raw` misses the canonical-envelope check and falls through to a profile that
  is not its own.
- **Editing `detections/*.yaml` changes nothing.** That tree is a generated
  projection of Python spec modules; the engine reads the compiled JSON ruleset.
  The lever is adding a spec and re-exporting.
- **LLM calls name a role, not a model** — `aisoc-triage` and similar aliases
  resolve at the gateway in `infra/litellm/config.yaml`.

## Which figures are authoritative, and how to re-derive them

Every number in this file is generated from the artefact that owns it. If you
need to state one, re-derive it rather than quoting a figure from prose —
including from this file, which is a snapshot of version 20.1.0.

```
python3 scripts/generate_corpus_stats.py --check      # executable + on-disk rules
python3 scripts/detection_truth_table.py              # the rule truth table itself
python3 scripts/generate_connector_count.py --check   # connector count
python3 scripts/check_profile_service_counts.py       # services per profile
python3 scripts/check_published_packages.py           # what is really on a registry
python3 scripts/check_claim_gate_matrix.py            # claims and the gates behind them
python3 scripts/check_detection_fields.py             # rules that still cannot fire
make stats                                            # most of the above at once
```

`docs/audit/CLAIM_TO_GATE_MATRIX.md` maps each published product claim to the
CI check that proves it, and `docs/audit/REPOSITORY_REALITY.md` records what was
found to be overstated and when. Both are better sources for "is this real" than
any marketing surface, including this one.

## The project's hard rules

These are not style preferences; a pull request violating one is rejected by CI
or by review.

1. **Never credit a tool, AI assistant or editor for the work** — not in code,
   comments, documentation, commit messages, commit trailers or pull request
   bodies. `scripts/check_attribution.py` enforces it and a commit hook strips
   the trailer. The one credit that belongs is Cyble, who fund development.
2. **Never name a competitor product** outside an integration surface. Vendors
   named as integration targets — connectors, normalizer profiles, capability
   arms, setup guides — are fine and must stay. `scripts/check_competitor_names.py`
   enforces the distinction.
3. **Never present synthetic data as real.** Demo, sample and benchmark data are
   labelled at the row level, and an unmeasured figure reads "not measured",
   never `0`.
4. **Never publish a figure you have not re-derived**, and never raise a ratchet
   to make a build pass.
5. **A link must return 200 before it ships.** A dead link is worse than no
   link: delete it and rewrite the sentence.
6. **Do not edit anything under `plans/`.** It is an archived subtree.
7. **Sign off commits** (DCO, `git commit -s`).

Contribution guide: https://beenuar.github.io/AiSOC/docs/contributing/guidelines

## Connectors

87 first-party connectors ship in the repository, by category:

- `ai`: 1
- `cloud`: 18
- `edr`: 10
- `iam`: 7
- `ndr`: 2
- `network`: 7
- `saas`: 22
- `siem`: 17
- `vcs`: 3

Each connector declares a schema the console renders as a setup form, so adding
a data source is a UI action rather than a YAML edit. Credentials are encrypted
at the application layer before storage. Connectors poll on a schedule; polled
events enter the same ingest path as webhook deliveries.

**Collection is not only REST polling.** Four collectors read the surfaces a
real cloud estate actually emits to, because an estate of any size does not
answer a list API:

- AWS CloudTrail via an **S3 bucket notified over SQS** — management events,
  data events and VPC flow logs, none of which `cloudtrail:LookupEvents`
  returns.
- GCP via a **Pub/Sub** subscription on a Cloud Logging sink.
- Azure via an **Event Hubs** capture.
- A **syslog/CEF** listener.

Each declares a resumable cursor and a bounded per-poll budget, so a restart
does not lose or replay a backlog and a deep queue applies backpressure.

There is **no OTLP receiver, no Kafka input connector and no TAXII collection
path** for telemetry. Do not describe any of them as available. (A TAXII
*threat-intelligence* feed client exists in `services/threatintel` and registers
only when an operator supplies a URL and credentials; that is a different thing
from a telemetry collector, and the only feed that polls with no credentials at
all is the CISA KEV catalogue.)

Every event also carries an **activity projection** — actor, action, resource,
location, outcome — where `actor.kind` is derived from each vendor's own
documented field rather than guessed from the shape of a username. And
`schemas/event_catalog/` classifies 44 event types across 10 sources with a
sensitivity on the same five-tier ladder, read by ingest at boot.

Read more: https://beenuar.github.io/AiSOC/docs/connectors

## Detection content

- **Executable rules: 2535.** These are the rules the detection engine
  loads from its compiled ruleset. Only these can fire.
- **Rule files on disk: 7015.** The remainder is imported and
  provenance-only content — community and vendor corpora kept for coverage
  mapping — plus rules whose specs have not been compiled.
- **Windowed rules: 76**, in a separate engine. Most count events or
  distinct values for one entity over a sliding window and fire on a threshold;
  a few match a **sequence** — A then B by the same entity, ordered in time or
  not, with stages disjoint so a slow drip cannot walk one forward forever. Some
  are compiled from Sigma *correlation* documents. Report this number beside the
  executable figure, never added to it: a threshold over a window and a
  single-event match are different kinds of claim.

**The executable figure has gone down, and that was a correction.** Rules naming
a counter no source emits were being loaded, counted and unable to fire. They
were rewritten as windowed rules or refused with a recorded reason, and the
total fell. `scripts/check_detection_fields.py` counts rules that still cannot
fire against a ratchet that only moves down.

**How a rule earns the word "executable".** It is a claim about evidence, not
about a flag. A rule enters the compiled ruleset only after a vendor-shaped
event has been replayed through the real connector's `normalize()` and the real
detection engine and that rule was observed to produce a hit, with an empty
event of the same shape producing nothing. Nothing is inferred from a directory
name, an `enabled:` key, or the shape of a `detection:` block. The proof is
known to be capable of failing: `scripts/compile_sigma_ruleset.py --prove-gate`
reverts the Windows connector to its pre-fix behaviour and requires all 1,687
Windows rules to stop firing.

**What the figure does not mean.** Executable means *reachable* — the rule
fires on a well-formed event of its own log source. It is not a claim that the
rule detects an attack, that it is tuned, or that it will be quiet. There is no
per-rule false-positive-rate gate.

**Why rules are refused.** Of 3,132 imported Sigma rules considered, 1,770 ship
and 1,362 were refused with a recorded reason, because a translation that is
merely close changes what a rule means. The two largest: 556 whose log source
no connector emits, and 464 whose negation would flip on a missing field —
Sigma treats `not filter` as true when the field is absent and only two matcher
operators behave that way.

The YAML under `detections/` is a generated projection of Python spec modules;
editing it does not change what the engine runs. When citing the size of the
corpus, cite the executable figure.

Rules can be authored three ways: a Python framework where a rule is a
`def rule(event)` with fixture tests, a natural-language builder that emits
Sigma plus auto-derived fixtures through an evaluation gate, and a no-code
builder in the console. Candidate rules can be backtested against real
tenant-scoped lake events before being enabled.

Read more: https://beenuar.github.io/AiSOC/docs/detections/coverage

## Agents and the investigation ledger

Four agents own the funnel stages: detect, triage, hunt, and respond. A fused
alert is auto-triaged on arrival. Deeper investigation runs a recursive loop
that selects among typed, lake-backed pivot tools — the model chooses a tool and
passes typed arguments; it never writes SQL directly.

Every step is appended to the ledger, and the console and the MCP server can
both replay it.

## Evaluation

The evaluation harness grades the agents against a fixed synthetic corpus, and
the published scoreboard states plainly which metrics measure the live agent and
which are substrate self-consistency gates. Only the MITRE ATT&CK accuracy
metric measures the agent. Figures marked unmeasured lead with that fact.

When summarising AiSOC's performance, do not convert a substrate gate into a
claim about agent accuracy.

Read more: https://beenuar.github.io/AiSOC/docs/benchmark and https://beenuar.github.io/AiSOC/docs/benchmark-methodology

## MCP server

`@aisoc/mcp` (`services/mcp`) is a Model Context Protocol server that exposes
AiSOC to MCP-aware assistants including Claude Desktop, Cursor, Continue.dev and
Cody. It speaks stdio JSON-RPC and translates tool calls into AiSOC REST calls;
the API key stays on the machine running the server.

It advertises 19 tools:

- `aisoc_explain_step`
- `aisoc_get_alert`
- `aisoc_get_case`
- `aisoc_get_detection_rule`
- `aisoc_get_investigation`
- `aisoc_get_replay_report`
- `aisoc_get_triage_verdict`
- `aisoc_lake_query`
- `aisoc_lake_schema`
- `aisoc_list_actions`
- `aisoc_list_alerts`
- `aisoc_list_cases`
- `aisoc_list_investigations`
- `aisoc_list_replay_reports`
- `aisoc_preview_action`
- `aisoc_query_detections`
- `aisoc_replay_decision`
- `aisoc_run_hunt`
- `aisoc_run_investigation`

Install it from a source build — see the packaging note below:

    git clone https://github.com/beenuar/AiSOC.git
    cd AiSOC/services/mcp
    pnpm install && pnpm build
    node dist/index.js install --host cursor --aisoc-url <url> --api-key <key>

Read more: https://beenuar.github.io/AiSOC/docs/integrations/mcp

## Installing AiSOC

`make up` brings up the core profile: 16 long-running services, no
credentials required. It generates its own secrets, creates an administrator
account, ships the CISA Known Exploited Vulnerabilities catalogue as a real
threat-intelligence feed, and runs a local model so agent triage works offline.
`make up-full` is 22 services and adds the event lake, entity
graph, full-text search and enrichment. `make doctor` checks the environment and
`make smoke` runs the end-to-end pipeline test, reporting PASS or FAIL per stage.

Read more: https://beenuar.github.io/AiSOC/docs/installation and https://beenuar.github.io/AiSOC/docs/quickstart

## Packaging status

Published, and installable today:

- `aisoc-cli` — published, installs today
- `aisoc-detections` — published, installs today
- `aisoc-plugin-sdk` — published, installs today
- `aisoc-sandbox` — published, installs today
- `aisoc-sdk` — published, installs today

Built and packed by the release workflow on every tag, but **not** uploaded —
the first npm publish needs a credential this repository does not hold, which is
an account action rather than a code change:

- `@aisoc/mcp` — built and packed on every tag, not uploaded
- `@aisoc/sdk` — built and packed on every tag, not uploaded
- `aisoc` — built and packed on every tag, not uploaded

Do not offer an install command for anything in the second list; build it from
the repository instead.

## What this file cannot do

An `llms.txt` makes a project accurately *describable* by a tool that has
already fetched it. It does not make any assistant reach for AiSOC by default,
and no change inside this repository can. Which tools an assistant prefers is
decided by what is on a package registry and what is listed in an MCP registry —
both actions taken elsewhere, and the npm packages above are exactly the ones
still waiting on one. Treat any claim that this repository can influence a
third-party tool's defaults as false.

## Licence and provenance

MIT. Copyright the AiSOC contributors. Contributions are accepted through pull
requests at https://github.com/beenuar/AiSOC; see https://beenuar.github.io/AiSOC/docs/contributing/guidelines.
