---
id: live_view_index
name: LiveView Skills Index
description: |
  Skills for opening and driving agent-controllable visualization
  components in the Pantheon UI sidebar — interactive viewers the agent
  can open, control, and read back. Viewers: Vitessce (spatial / single-
  cell omics), Viv (bioimage / microscopy), volume3d (3D image volumes —
  MIP/ISO), spatial3d (3D spatial transcriptomics), Mol*, IGV, Gosling,
  Cytoscape, MSA, RDKit, phylotree, plus agent-generated apps.
tags: [live-view, visualization, vitessce, viv, bioimage, spatial, single-cell, interactive]
---

# LiveView — Agent-Controllable UI Components

A **LiveView** is an interactive component the agent opens in the Pantheon
UI's right sidebar, then drives and observes through the `live_view` tools.
Unlike a static plot image, a LiveView is live: the agent changes its state
and the user sees it update; the user interacts with it and the agent reads
the result back.

Load the relevant skill file before building a visualization.

**Architecture** — only the LiveView SDK runtime is built in. Every viewer
is a **plugin**: a folder `skills/live_view/<name>/` holding `<name>.md`
(this guide), `adapter.js` (a `setup(lv, root)` module), and an optional
`demo.json`. `open_live_view` either resolves a named viewer plugin
(`view_type="vitessce"`) or loads an agent-generated component
(`view_type="custom"` + `module_url`). Adding a viewer = dropping a new
`<name>/` folder here; no app code changes.

## Available skills

### Vitessce — spatial / single-cell / imaging data

Open a Vitessce browser to explore spatial transcriptomics, single-cell, and
microscopy-imaging datasets: spatial scatterplots, gene-expression coloring,
heatmaps, cell-set selection, image layers.

**Skill file**: [vitessce/vitessce.md](./vitessce/vitessce.md)

**When to use**:
- Visualizing spatial transcriptomics (10x Visium, Xenium, MERFISH, …)
- Single-cell data with embeddings (UMAP/t-SNE) the user should explore
- Cell segmentations as **interactive** objects — click/hover a cell,
  colour by type or gene. (Just *viewing* boundaries on an image → use Viv;
  Vitessce does not render clean boundaries.)
- Spatial omics where cells/sets/embeddings matter, not just the image

### Viv — bioimage / microscopy viewer

Open a Viv viewer for high-resolution, multiplexed bioimaging — OME-TIFF
and OME-Zarr (OME-NGFF): multichannel fluorescence, microscopy, IF/IMC/
CODEX, whole-slide images. Channel colors, contrast, pan/zoom, overview.

**Skill file**: [viv/viv.md](./viv/viv.md)

**When to use**:
- The data *is an image* — OME-TIFF / OME-Zarr microscopy
- Multichannel fluorescence the user wants to recolour / adjust
- Cloud-hosted or local bioimages (served via `serve_local_data`)
- **Overlaying a cell segmentation / showing cell boundaries** on an image
  (boundaries as an extra channel — the preferred way to *view* a mask)

### volume3d — 3D volumetric image (MIP / iso-surface)

Open a GPU volume renderer for a **3D scalar image** — OPT / light-sheet /
micro-CT / MRI scans, confocal z-stacks, segmentation probability maps.
Ray-casts as a maximum-intensity projection (MIP) or an iso-surface (ISO);
rotate, switch mode, tune the threshold. The input is a 3D TIFF / NIfTI /
array — the skill converts it to an OME-NGFF Zarr pyramid.

**Skill file**: [volume3d/volume3d.md](./volume3d/volume3d.md)

**When to use**:
- A dense 3D *image volume* the user wants to see in 3D (a flat 2D image → Viv)
- OPT / light-sheet / micro-CT / MRI / confocal z-stack; a 3D mask or density field
- "render / rotate this 3D scan", "show the iso-surface", "MIP of this stack"

### spatial3d — 3D spatial transcriptomics (cell point cloud)

Open a 3D point-cloud viewer for **single-cell spatial data** — every cell a
point at its (x, y, z), coloured by a categorical obs column (cell type /
cluster) or by a gene's expression, in 3D / 2D / UMAP. Scales to ~1M+ cells.
The input is an AnnData (`.h5ad`) with 3D `obsm['spatial']` — the skill
converts it to a spatial Zarr.

**Skill file**: [spatial3d/spatial3d.md](./spatial3d/spatial3d.md)

**When to use**:
- **3D** spatial transcriptomics (Stereo-seq / MERFISH / Slide-seq / 3D
  reconstructions), or any AnnData with 3D `obsm['spatial']`
- The cells live in a 3D volume and the 3D arrangement matters (vs. Vitessce,
  which is 2D scatterplots) — colour cells by type or by gene
- "show the cells in 3D", "colour the embryo by <gene>", "where is <cell type>"

### Mol* — 3D molecular structures

Open a Mol* viewer for 3D macromolecular structures — proteins, nucleic
acids, complexes — from the RCSB PDB, the AlphaFold DB, or a local
`.pdb` / `.cif` file. Rotate, zoom, inspect; AlphaFold models colour by
pLDDT.

**Skill file**: [molstar/molstar.md](./molstar/molstar.md)

**When to use**:
- Showing a protein / nucleic-acid 3D structure (experimental or predicted)
- Visualising an AlphaFold prediction
- Any `.pdb` / `.cif` structure file (see also the `structural_biology` skill
  for obtaining / predicting structures)

### IGV — genome browser (tracks on a reference)

Open an IGV.js genome browser to view BAM/CRAM alignments, VCF variants,
BED/GFF annotations, bigWig coverage — on a reference genome (hg38, mm10,
custom FASTA, …). Pan, zoom, jump to a gene or locus.

**Skill file**: [igv/igv.md](./igv/igv.md)

**When to use**:
- "Look at this region in a genome browser" — RNA-seq pileups, ChIP/ATAC
  peaks, variant calling, splice junctions, CRISPR-screen hits
- Any BAM/CRAM/VCF/BED/GFF/bigWig on a reference genome
- A gene symbol or coordinate range to display

### Gosling — designed genomic figures (Vega-Lite-like grammar)

Open a Gosling.js view for **designed** genomic visualisations — circular
ideograms, multi-track / multi-sample layouts, comparative dual-genome
views, custom encodings. Driven by a declarative JSON **spec**.

**Skill file**: [gosling/gosling.md](./gosling/gosling.md)

**When to use**:
- Circular chromosome ideograms / circos-style plots
- Designed multi-track or sample-faceted genomic figures
- Comparative dual-genome / synteny visualisations
- Anything that's hard to build with matplotlib but easy with a grammar
- Use IGV instead if the task is to *look at a BAM / VCF / BED file at a
  locus*; the two viewers do not overlap.

### Cytoscape — biological networks & pathways

Open a Cytoscape.js view for interactive networks — protein-protein
interactions, signalling / metabolic / regulatory pathways, ontology
graphs. Nodes + edges as JSON, built-in layouts (cose, breadthfirst,
circle, dagre, ...), CSS-like stylesheet by selectors.

**Skill file**: [cytoscape/cytoscape.md](./cytoscape/cytoscape.md)

**When to use**:
- PPI networks (STRING, BioGRID, IntAct)
- Signalling / metabolic / regulatory pathways
- Gene-regulatory networks (TF → target)
- Any graph the user benefits from *interactively* (drag, hover, zoom)

### MSA — multiple sequence alignment viewer

Open a multiple-sequence-alignment view (EBI Nightingale's
`<nightingale-msa>`) for protein or DNA alignments. Standard colour
schemes (clustal, taylor, hydro, zappo, ...), configurable tile sizes.

**Skill file**: [msa/msa.md](./msa/msa.md)

**When to use**:
- Display a pre-computed alignment from MAFFT / MUSCLE / Clustal / MMseqs2
- Compare orthologs at a functional site / domain
- Pair with `phylotree` for tree + alignment side-by-side

### RDKit — 2D small-molecule depictions

Render 2D depictions of small molecules from SMILES / MOL block using
RDKit-JS (WebAssembly build). Complements `molstar` (3D macromolecules)
— RDKit is the canonical 2D view for drugs, metabolites, organics.

**Skill file**: [rdkit/rdkit.md](./rdkit/rdkit.md)

**When to use**:
- A SMILES (or list) to view as a 2D structure
- Pull drugs / metabolites from ChEMBL / PubChem / DrugBank → render
- Display the substrate / product of a reaction
- Highlight substructures or specific atoms on a molecule

### Phylotree — phylogenetic trees

Open a phylotree.js view for an interactive phylogenetic tree from a
Newick string. Linear or radial layout, branch-length-scaled; rerooting,
ladderise, clade collapse built in.

**Skill file**: [phylotree/phylotree.md](./phylotree/phylotree.md)

**When to use**:
- Show a phylogeny from IQ-TREE / RAxML / FastTree / MrBayes / BEAST
- Inspect / collapse / reroot a clade interactively
- Pair with `msa` for tree + alignment side-by-side

### Generate a custom LiveView app

Write your own interactive component with the LiveView SDK when no existing
viewer fits — a bespoke dashboard, custom plot, or tailored data view that
the agent can still open, drive, and observe.

**Skill file**: [live-view-app.md](./live-view-app.md)

**When to use**:
- The data / interaction doesn't match a ready-made viewer
- You need a tailored view of analysis output
- You want a custom interactive control surface for the user

## The `live_view` tools

| Tool | Purpose |
|------|---------|
| `open_live_view(view_type, title, state, module_url?)` | Open a viewer plugin (e.g. `view_type="vitessce"`) or a custom component (`view_type="custom"` + `module_url`); returns `view_id` |
| `serve_local_data(path)` | Expose a workspace file/dir over HTTP+CORS; returns a fetchable URL |
| `serve_endpoint(name, path, config?)` | Expose a lightweight Python HTTP endpoint over the same CORS data server; returns a fetchable `/api/<name>/` URL |
| `manage_endpoints(action, name?)` | Manage endpoints: `action="list"` lists all, `action="info"` checks one, `action="unregister"` removes one |
| `live_view_update(view_id, patch)` | Deep-merge a partial-state patch (drive it) |
| `live_view_set_state(view_id, state)` | Replace the whole state |
| `live_view_get_state(view_id)` | Read state, `status`, and `diagnostics` — incl. the user's own edits |
| `live_view_call(view_id, action, args)` | Invoke a component-defined action |
| `live_view_screenshot(view_id)` | Render the view to an image — `observe_images` it to see it |
| `list_live_views()` / `close_live_view(view_id)` | List / close |

Use `serve_local_data` when the browser should fetch a file you already wrote.
Use `serve_endpoint` when the browser should fetch computed data. It is not
specific to Gosling: any LiveView can use the returned URL if its config accepts
data URLs, and custom LiveView apps can call `fetch(url)` directly. Examples:
Gosling CSV/JSON/BED tracks, Cytoscape graph JSON, MSA alignment text, a custom
dashboard's computed table, or a future tileset-style API.

Endpoint modules must export `async def handle(request)`, `def build(): return
handler`, or `def build(config): return handler`, where the handler returns an
`aiohttp.web.Response` / `StreamResponse`. The frontend and endpoint coordinate
runtime parameters through ordinary HTTP: path tail (`/api/name/<tail>`), query
params, headers, or POST JSON. `config` is only for registration-time
JSON-serializable constants such as fixed paths or sample names; use request
parameters for interactive controls, and files plus `serve_local_data` for large
arrays or binary data. Keep handlers light: precompute heavy results before
serving, or run complex services in a separate process and proxy them in.

Use `manage_endpoints` to inspect or clean up registered endpoints:
- `manage_endpoints("list")` — list all active endpoints with their URLs
- `manage_endpoints("info", "track_name")` — check if a specific endpoint exists
- `manage_endpoints("unregister", "old_track")` — remove an endpoint that's no
  longer needed

Registering a new endpoint with the same name replaces the previous handler.
Endpoint handlers run in the data server's thread and should return quickly;
any exceptions are logged but clients receive only a generic error message.

Workflow: `open_live_view` → **verify** (`live_view_get_state` for
`diagnostics`, `live_view_screenshot` to see it — `status: ready` does NOT
mean it rendered correctly) → drive with `live_view_update` →
`live_view_get_state` before the next move. Never treat reading back your
own `live_view_update` value as verification.
