# Monitor Metrics Reference

This document provides an overview of the metric categories used in Trinity-RFT for tracking exploration, evaluation, and training progress.

## Metric Naming Convention

Most metrics follow a hierarchical naming convention: `{category}/{taskset_name}/{metric_name}/{statistic}`

- **Category**: Broad functional area (rollout, eval, time, actor, critic, etc.)
- **Taskset name**: Name of the taskset used, only applicable for eval metrics
- **Metric name**: Specific metric being measured
- **Statistic**: Aggregation method (mean, max, min, std, sum, etc.) if applicable


## Aggregation Type Suffix

Metric keys can optionally carry an **aggregation type suffix** to control how the metric is aggregated across experiences, runs, tasks, and steps. The suffix is appended to the key with a colon separator:

```
<metric_name>:<agg_type>
```

Supported aggregation types:

| Suffix | Behavior | Output key example |
|--------|----------|--------------------|
| *(none)* | Mean (default) | `rollout/accuracy/mean`, `/max`, `/min` |
| `:sum` | Sum | `rollout/total_tokens/sum` |
| `:max` | Maximum | `rollout/latency/max` |
| `:min` | Minimum | `rollout/latency/min` |
| `:last` | Last observed value | `rollout/model_version/last` |

Keys without a suffix default to **mean** aggregation, which preserves full backward compatibility with existing workflows and operators.

**Example usage in a workflow:**

```python
# Default mean aggregation (backward compatible)
exp.metrics["accuracy"] = 0.85
exp.metrics["reward"] = 1.0

# Explicit sum aggregation
exp.metrics["total_tokens:sum"] = len(tokens)
exp.metrics["time/env_interaction:sum"] = elapsed_time

# Last-value aggregation
exp.metrics["model_version:last"] = current_version
```

All aggregation logic is implemented in `trinity.utils.metrics`, which provides:
- `parse_metric_key(key)`: Parse a metric key into `(name, AggType)`.
- `aggregate_metrics(metric_dicts, prefix, ...)`: Batch-level aggregation (task → step) for training metrics.
- `aggregate_eval_metrics(metric_dicts, prefix, ...)`: Batch-level aggregation for evaluation metrics.
- `aggregate_run_level_metrics(metric_dicts)`: Experience-level → run-level aggregation.


## Metric Categories

In the following, metrics are categorized by their source component (where they are generated) and their metric prefix (the first part of the metric name).

### Explorer Metrics

Explorer metrics track performance during the rollout phase where the model generates responses, including rollout metrics (`rollout/`), eval metrics (`eval/`), and some time metrics (`time/`).


#### Rollout Metrics (`rollout/`)

Rollout metrics track performance during the rollout phase where the model generates responses.

- **Format**: `rollout/{metric_name}/{statistic}`
- **Examples**:
  - `rollout/accuracy/mean`: Average accuracy of generated responses
  - `rollout/format_score/mean`: Average format correctness score
  - `rollout/total_tokens/sum`: Total token count (sum aggregation)

**Metric Aggregation Process**:

Consider an exploration step with `batch_size` tasks, where each task has `repeat_times` runs. Rollout metrics (e.g., `rollout/`) are computed and aggregated at different levels:

- From *experiences* to *run level* (only effective for workflows with `can_repeat=False`): In `aggregate_run_level_metrics`, metrics are aggregated across all experiences generated by each run. The aggregation respects the `:agg` suffix — keys without a suffix are averaged, keys with `:sum` are summed, etc.

- From *run level* to *task level*: In `calculate_task_level_metrics`, metrics are aggregated across `repeat_times` runs of the same task. For example, `rollout/accuracy` is the average accuracy of all runs of the task.

- From *task level* to *step level*: In `aggregate_metrics`, metrics are aggregated across all tasks in the step. For example, `rollout/accuracy/mean`, `rollout/accuracy/max`, `rollout/accuracy/min` are the average, max, and min accuracy (`rollout/accuracy`) of all tasks in the step. For `:sum` metrics, only the `/sum` statistic is output.

The following diagram illustrates the aggregation process for rollout metrics:
```mermaid
graph TD
    subgraph Step["Batch_size=3 Tasks"]
        subgraph Task1["Task 1 (repeat_times=2)"]
            Run1_1["Run 1<br/>accuracy: 0.8"]
            Run1_2["Run 2<br/>accuracy: 0.9"]
            Run1_1 --> Task1_Metric["rollout/accuracy<br/>= 0.85"]
            Run1_2 --> Task1_Metric
        end
        subgraph Task2["Task 2 (repeat_times=2)"]
            Run2_1["Run 1<br/>accuracy: 0.6"]
            Run2_2["Run 2<br/>accuracy: 0.9"]
            Run2_1 --> Task2_Metric["rollout/accuracy<br/>= 0.75"]
            Run2_2 --> Task2_Metric
        end

        subgraph TaskN["Task 3 (repeat_times=2)"]
            Run3_1["Run 1<br/>accuracy: 0.95"]
            Run3_2["Run 2<br/>accuracy: 0.85"]
            Run3_1 --> Task3_Metric["rollout/accuracy<br/>= 0.9"]
            Run3_2 --> Task3_Metric
        end

        Task1_Metric --> Step_Metrics["Step Level Metrics<br/>rollout/accuracy/mean=0.83<br/>rollout/accuracy/max=0.9<br/>rollout/accuracy/min=0.75"]
        Task2_Metric --> Step_Metrics
        Task3_Metric --> Step_Metrics
    end
```

#### Eval Metrics (`eval/`) and Benchmark Metrics (`bench/`)

Evaluation metrics measure model performance on held-out evaluation tasks. These metrics are computed during periodic evaluation runs.

- **Format**: `eval/{task_name}/{metric_name}/{statistic}` or `bench/{task_name}/{metric_name}/{statistic}`
- **Examples**:
  - `eval/gsm8k-eval/accuracy/mean@4`: Mean accuracy across repeat_times=4 runs
  - `bench/gsm8k-eval/accuracy/best@4`: Best accuracy value across repeat_times=4 runs

- **Note**:
  - Eval and bench metrics are computed in the same way, the only difference is the prefix of the metric name.
  - By default, only the *mean* of the metric is returned. If you want to return detailed statistics, you can set `monitor.detailed_stats` to `True` in the config.

**Metric Aggregation Process**:

Consider an evaluation step with `len(eval_taskset)` tasks, where each task has `repeat_times` runs. Evaluation metrics (e.g., `eval/`, `bench/`) are computed and aggregated at different levels:

- From *experiences* to *run level* (only effective for workflows with `can_repeat=False`): In `aggregate_run_level_metrics`, metrics are aggregated across all experiences generated by each run.

- From *run level* to *task level*: In `calculate_task_level_metrics`, metrics are aggregated across `repeat_times` runs of the same task. For default (mean) metrics, bootstrap statistics are computed: `mean@N`, `std@N`, `best@N`, `worst@N`. For non-mean metrics (`:sum`, `:max`, etc.), only the corresponding single aggregation is output.

- From *task level* to *step level*: In `aggregate_eval_metrics`, metrics are aggregated across all tasks in the step. For example, `eval/dummy/accuracy/mean@2`, `eval/dummy/accuracy/std@2`, `eval/dummy/accuracy/best@2`, `eval/dummy/accuracy/worst@2` are the average, std, best, and worst accuracy (`eval/dummy/accuracy`) of all tasks in the step.


The following diagram illustrates the aggregation process on a dummy dataset with three tasks for evaluation metrics. By default, `mean@k`, `std@k`, `best@k`, `worst@k` of the metrics across all evaluation tasks are reported. You can set `monitor.detailed_stats` to `True` in the config to return detailed statistics.

```mermaid
graph TD
    subgraph Step["len(eval_taskset)=3 Tasks"]
        subgraph Task1["Task 1 (repeat_times=2)"]
            Run1_1["Run 1<br/>accuracy: 0.8"]
            Run1_2["Run 2<br/>accuracy: 0.9"]
            Run1_1 --> Task1_Metric["eval/dummy/accuracy/mean@2=0.85<br/>eval/dummy/accuracy/std@2=0.05"]
            Run1_2 --> Task1_Metric
        end

        subgraph Task2["Task 2 (repeat_times=2)"]
            Run2_1["Run 1<br/>accuracy: 0.6"]
            Run2_2["Run 2<br/>accuracy: 0.9"]
            Run2_1 --> Task2_Metric["eval/dummy/accuracy/mean@2=0.75<br/>eval/dummy/accuracy/std@2=0.15"]
            Run2_2 --> Task2_Metric
        end

        subgraph TaskN["Task 3 (repeat_times=2)"]
            Run3_1["Run 1<br/>accuracy: 0.95"]
            Run3_2["Run 2<br/>accuracy: 0.85"]
            Run3_1 --> Task3_Metric["eval/dummy/accuracy/mean@2=0.9<br/>eval/dummy/accuracy/std@2=0.05"]
            Run3_2 --> Task3_Metric
        end

        Task1_Metric --> Step_Metrics["Step Level Metrics<br/>eval/dummy/accuracy/mean@2=0.83<br/>eval/dummy/accuracy/std@2=0.083"]
        Task2_Metric --> Step_Metrics
        Task3_Metric --> Step_Metrics
    end
```

When you set `monitor.detailed_stats` to `True`, you will get detailed statistics including mean, std, min, max, as shown in the following diagram:

```mermaid
graph TD
    subgraph Step_Metrics_Summary["Detailed Statistics"]
        Stat1["eval/dummy/accuracy/mean@2/mean=0.83<br/>eval/dummy/accuracy/mean@2/std=0.062<br/>eval/dummy/accuracy/mean@2/max=0.9<br/>eval/dummy/accuracy/mean@2/min=0.75"]
        Stat2["eval/dummy/accuracy/std@2/mean=0.083<br/>eval/dummy/accuracy/std@2/std=0.047<br/>eval/dummy/accuracy/std@2/max=0.15<br/>eval/dummy/accuracy/std@2/min=0.05"]
        Step_Metrics --> Stat1
        Step_Metrics --> Stat2
    end
```


#### Time Metrics (`time/`)

Time metrics measure execution duration for various operations throughout the rollout process.

- **Format**: `time/{operation_name}`
- **Examples**:
  - `time/eval`: Time from the start of submitting evaluation tasks to the end of the evaluation phase; this duration includes both evaluation tasks and some rollout tasks.
  - `time/wait_explore_step`: Time to wait for one rollout step to complete.

**Note**:
  - Time measuring can be inaccurate due to the asynchronous nature of the rollout process, but it is still useful for monitoring the overall training progress.
  - Time metrics are reported in seconds unless otherwise specified.
  - Some training operations also report per-token timing metrics with the prefix `timing_per_token_ms/` (e.g., `timing_per_token_ms/update_actor`, `timing_per_token_ms/update_critic`, `timing_per_token_ms/adv`, `timing_per_token_ms/values`). These metrics normalize execution time by the number of tokens processed, providing efficiency measurements independent of batch size.


### Trainer Metrics

This category includes metrics that track the training dynamics of the policy (actor) model (`actor/`) and the value function (critic) model (`critic/`), as well as some performance metrics (`perf/`, `global_seqlen/`, `response_length/`, `prompt_length/`, `time/`). These metrics are adapted from [veRL](https://github.com/volcengine/verl). Interested users can refer to the [veRL documentation](https://verl.readthedocs.io/en/latest/index.html) for more details.


### Data Processing Metrics

This category includes metrics that track the processing of experiences through various pipeline operators (`experience_pipeline/`) and data sampling statistics (`sample/`). These metrics are aggregated at the step level, as the experience pipeline and data sampling are performed in each step.


#### Experience Pipeline Metrics (`experience_pipeline/` and `experience_pipeline/time/`)

Experience pipeline metrics track the processing of experiences through various pipeline operators. Each metric represents the count of the specific operator in one step.

- **Format**: `experience_pipeline/{metric_name}`
- **Examples**:
  - `experience_pipeline/experience_count`: Number of experiences processed
  - `experience_pipeline/group_advantages/reward_mean/mean`: Here `reward_mean` is the mean reward of each task, then we compute the mean of the mean rewards of all tasks in the step.

The following diagram illustrates the aggregation process for data processing metrics:
```mermaid
graph TD
    subgraph Step["4 Experiences in one step"]
        subgraph Task1["Experience 1"]
            Run1_1["Run 1<br/>reward_mean: 0.8"]
            Run1_2["Run 2<br/>reward_mean: 0.8"]
            Run2_1["Run 3<br/>reward_mean: 0.9"]
            Run2_2["Run 4<br/>reward_mean: 0.9"]
            Run1_1 --> Task1_Metric["rollout/accuracy<br/>= 0.85"]
            Run1_2 --> Task1_Metric
            Run2_1 --> Task1_Metric
            Run2_2 --> Task1_Metric
        end
    end
```

#### Sample Metrics (`sample/`)

Sample metrics track data sampling statistics during training.

- **Format**: `sample/{metric_name}`
- **Examples**:
  - `sample/model_version/mean`: Mean model version of sampled experiences
  - `sample/task_count`: Number of tasks in the sampled batch
