A `ValidationSet` contains a list of `ValidationCase`, which are in turn composed of ids and targets. The most common validation set is a pair of transcript id and `value` that the scanner should have returned.

Transcript ID | Expected Value
--- | ---
Fg3KBpgFr6RSsEWmHBUqeo | true
VFkCH7gXWpJYUYonvfHxrG | false

Note that values can be of any type returned by a scanner, and it is also possible to do greater than / less than checks or write custom predicates.

#### Development

How would you develop a validation set like this? Typically, you will review some of your existing transcripts using Scout View, decide which ones are good validation examples, copy their transcript id (which is the same as the sample UUID), then record the appropriate entry in a text file or spreadsheet.

Use the Copy button to copy the UUID for the transcript you are reviewing:

As you review transcript and find good examples, build up a list of transcript IDs and expected values. For example, here is a CSV file of that form:

    ctf-validation.csv

    id,target
    Fg3KBpgFr6RSsEWmHBUqeo,true
    VFkCH7gXWpJYUYonvfHxrG,false
    SiEXpECj7U9nNAvM3H7JqB,true

#### Scanning

You’ll typically create a distinct validation set for each scanner, and then pass the validation sets to `scan()` as a dict mapping scanner to set:

    scanning.py

    from inspect_scout import scan, transcripts_from, validation_set

    scan(
        scanners=[ctf_environment(), java_tool_usages()],
        transcripts=transcripts_from("./logs"),
        validation={
            "ctf_environment": validation_set("ctf-validation.csv")
        }
    )

You can also specify validation sets on the command line. If the above scan was defined in a `@scanjob` you could add a validation set from the CLI using the `-V` option as follows:

    scout scan scanning.py -V ctf_environment:ctf_environment.csv

This example uses the simplest possible id and target pair (transcript _id => boolean). Other variations are possible, see the IDs and Targets section below for details. You can also use other file formats for validation sets (e.g. YAML), see Validation Files for details.

#### Results

Validation results are reported in three ways:

- The scan status/summary UI provides a running tabulation of the percentage of matching validations.

- The data frame produced for each scanner includes columns for the validation:

  - `validation_target`: Ideal scanner result

  - `validation_result`: Result of comparing scanner `value` against `validation_target`

- Scout View includes a visual indication of the validation status for each transcript: