# Dataflow

The Dataflow core (`astraflow/dataflow/`) is the orchestration layer that manages async data flow between rollout generation and training.

## Key Components

- **`AstraFlow`** — Main orchestrator that composes data acquisition and serving. Manages rollout buffers, filtering, staleness tracking, and replay logic.
- **`DataAcquisition`** — Runs threaded producer loops that pull rollouts from RaaS, filter accepted samples, and publish to data serving.
- **`DataServing`** — Owns fresh/replay buffers, manages staleness and normalization, and exposes batch-fetch APIs.
- **`RaaS2InferenceEngine`** — HTTP client that submits rollout requests to RaaS and polls for completions.
- **`RaaSPool`** — Manages a pool of RaaS instances with heartbeating and dynamic registration.
- **`AstraFlowService`** — Flask HTTP service exposing REST endpoints. Trainers register via `POST /ready`, fetch batches via `GET /batch`, and trigger weight sync via `POST /notify_version`. RaaS instances join the pool via `POST /register_raas` and are removed via `POST /deregister_raas`.

## How It Fits

Dataflow sits between RaaS (upstream) and Trainer (downstream):

- Requests rollouts from RaaS via `RaaS2InferenceEngine`
- Buffers and serves training batches to Trainer via `AstraFlowService`
- Coordinates weight sync timing between Trainer and RaaS
