Orbyx AI-SPM — Control Plane & Data Path

How a request flows through the platform — from browser to LLM response, and how every event is monitored.

User / Ingress
AI / LLM
Security / Guard
Threat / Enforcement
Infra / Cache
Data / Persistence
Observability
⚡ Kafka Event Bus — Async backbone connecting all services
cpm.audit.{tenant}
Raw request/response events emitted by API gateway
API → Processor → SPM Aggregator
cpm.posture.{tenant}
Enriched posture-scored events from Processor
Processor → SPM Aggregator → Postgres
cpm.cep.{tenant}
Complex event patterns — burst, drift, anomalies
Flink CEP → SPM API → Alert
cpm.guard.{tenant}
Content moderation results from guard-model
Guard Model → Policy Decider
cpm.enforcement.{tenant}
Block / freeze / rate-limit decisions
Freeze Controller → API Gateway
cpm.tool.{tenant}
Tool call requests and results from agent executor
Executor → Tool Parser → Agent
Data Path — Request / Response Flow
🌐 Browser (React UI) :3001
User submits a prompt. JWT token attached to every request.
HTTPS POST /api/v1/chat
🛡️ API Gateway :8080
① JWT auth — RS256 verify via public key
② Rate limit — Redis sliding-window counter (60 RPM)
③ Input guard — calls Guard Model (Llama Guard 3)
④ OPA policy — Rego policy evaluation (sync HTTP)
⑤ CEP check — burst & drift signals from Redis/Kafka
session lookup / memory read
🧠 Memory Service internal
Injects session (1 h), long-term (30 d), and system context into the prompt window via Redis.
enriched prompt → LLM
🤖 Claude (Anthropic API) external
Model generates a response. If a tool call is returned, the agent loop begins (Tool Parser → Executor → back to LLM).
response text → output guard
🔍 Output Guard internal
Second-pass semantic scan on Claude's response via Guard Model. Blocks PII leakage, policy violations.
approved response → browser
✅ Response delivered to user
Streamed back via SSE. Audit event published to Kafka simultaneously.
⚙️ Agent / Tool Loop (if tool call returned)
🔧 Tool Parser internal
Validates tool name and arguments against policy registry.
approved tool call
⚡ Executor internal
Runs the tool in sandbox, captures result. Emits cpm.tool event to Kafka.
tool result injected into next LLM turn
Control Plane — Monitoring & Enforcement
📋 Kafka Audit Events cpm.audit.t1
Every request, response, tool call, guard result, and enforcement action is published here.
consume
⚙️ Processor internal
Enriches raw audit events with posture score, model ID, and risk metadata. Republishes to cpm.posture.
consume + write
🗄️ SPM Aggregator internal
Writes all enriched events to Postgres (audit_export table). Feeds posture snapshots every 5 min.
read/write
📊 SPM API :8092
Model registry · posture scores · compliance reports. Triggers enforcement when rolling-avg risk score > 0.85.
🔬 Complex Event Processing
🌊 Flink CEP internal
Sliding-window stream analytics: burst detection (2 min), sustained volume (1 h), intent drift (Jaccard ≥ 0.65).
🎯 Proactive Threat Hunting (every 5 min)
🔎 Threat Hunting Agent internal
9 collectors run in parallel every 5 min:
Posture · Anomaly · Data leakage · Tool misuse
Prompt injection · Privilege escalation · CEP signals
Redis memory · Compliance gaps
Backend: LangChain + Groq (Llama 3.3 70B)
POST /api/v1/threat-findings
🎮 Agent Orchestrator :8094
Receives findings, scores sessions, manages cases, escalates critical findings to enforcement.
enforce block / freeze
🚫 Freeze Controller :8090
Freeze/unfreeze users or tenants. Publishes cpm.enforcement event so the API Gateway rejects subsequent requests.
📡 Observability
📈 Prometheus :9090
Scrapes /metrics from every service. Tracks request rate, latency, posture scores, enforcement events.
datasource
📊 Grafana :3000
Pre-provisioned dashboards: AI SPM posture · Engineering SLOs · Compliance audit trail.

Orbyx AI-SPM Platform · All services run on Docker Compose · Kafka + Redis + Postgres backbone