# Sidemantic

The universal metrics layer for consistent metrics across your data stack. Compatible with 15+ semantic model formats.

- **Supported Formats:** Sidemantic (YAML, Python or SQL), Cube, dbt MetricFlow, LookML, Hex, Rill, Superset, Omni, BSL, GoodData LDM, Snowflake Cortex, Malloy, OSI, AtScale SML, ThoughtSpot TML
- **Databases:** DuckDB, MotherDuck, PostgreSQL, BigQuery, Snowflake, ClickHouse, Databricks, Spark SQL (also via ADBC)

[Documentation](https://sidemantic.com) | [GitHub](https://github.com/sidequery/sidemantic) | [Docker Hub](https://hub.docker.com/repository/docker/sidequery/sidemantic) | [Discord](https://discord.com/invite/7MZ4UgSVvF) | [Demo](https://sidemantic.com/demo) (50+ MB data download, runs in your browser with Pyodide + DuckDB)

![Jupyter Widget Preview](preview.png)

The installer downloads the skill to `~/.agents/skills/sidemantic-modeler` and symlinks it into `~/.claude/skills/`.

## Quickstart

Install:
```bash
uv add sidemantic
```

Malloy support (uv):
```bash
uv add "sidemantic[malloy]"
```

HTTP API server (uv):
```bash
uv add "sidemantic[api]"
```

Notebook widget (uv):
```bash
uv add "sidemantic[widget]" jupyterlab
uv run jupyter lab
```

Marimo (uv):
```bash
uv add "sidemantic[widget]" marimo
uv run marimo edit
```

```python
import duckdb
from sidemantic.widget import MetricsExplorer

conn = duckdb.connect(":memory:")
conn.execute("create table t as select 1 as value, 'a' as category, date '2024-01-01' as d")
MetricsExplorer(conn.table("t"), time_dimension="d")
```

Define models in SQL, YAML, or Python:

<details>
<summary><b>SQL</b> (orders.sql)</summary>

```sql
MODEL (name orders, table orders, primary_key order_id);
DIMENSION (name status, type categorical);
DIMENSION (name order_date, type time, granularity day);
METRIC (name revenue, agg sum, sql amount);
METRIC (name order_count, agg count);
```
</details>

<details>
<summary><b>YAML</b> (orders.yml)</summary>

```yaml
models:
  - name: orders
    table: orders
    primary_key: order_id
    dimensions:
      - name: status
        type: categorical
      - name: order_date
        type: time
        granularity: day
    metrics:
      - name: revenue
        agg: sum
        sql: amount
      - name: order_count
        agg: count
```
</details>

<details>
<summary><b>Python</b> (programmatic)</summary>

```python
from sidemantic import Model, Dimension, Metric

orders = Model(
    name="orders",
    table="orders",
    primary_key="order_id",
    dimensions=[
        Dimension(name="status", type="categorical"),
        Dimension(name="order_date", type="time", granularity="day"),
    ],
    metrics=[
        Metric(name="revenue", agg="sum", sql="amount"),
        Metric(name="order_count", agg="count"),
    ]
)
```
</details>

Query via CLI:
```bash
sidemantic query "SELECT revenue, status FROM orders" --db data.duckdb
```

Or Python API:
```python
from sidemantic import SemanticLayer, load_from_directory

layer = SemanticLayer(connection="duckdb:///data.duckdb")
load_from_directory(layer, "models/")
result = layer.sql("SELECT revenue, status FROM orders")
```

## CLI

```bash
# Query
sidemantic query "SELECT revenue FROM orders" --db data.duckdb

# Interactive workbench (TUI with SQL editor + charts)
uvx --from "sidemantic[workbench]" sidemantic workbench models/ --db data.duckdb

# PostgreSQL server (connect Tableau, DBeaver, etc.)
uvx --from "sidemantic[serve]" sidemantic serve models/ --port 5433

# HTTP API server (JSON or Arrow)
uvx --from "sidemantic[api]" sidemantic api-serve models/ --port 4400 --auth-token secret

# Validate definitions
sidemantic validate models/

# Model info
sidemantic info models/

# Pre-aggregation recommendations
sidemantic preagg recommend --db data.duckdb

# Migrate SQL queries to semantic layer
sidemantic migrator --queries legacy/ --generate-models output/
```

## Demos

**Workbench** (TUI with SQL editor + charts):
```bash
uvx --from "sidemantic[workbench]" sidemantic workbench --demo
```

**PostgreSQL server** (connect Tableau, DBeaver, etc.):
```bash
uvx --from "sidemantic[serve]" sidemantic serve --demo --port 5433
```

**HTTP API server** (JSON or Arrow):
```bash
uvx --from "sidemantic[api]" sidemantic api-serve --demo --port 4400 --auth-token secret
```

**Colab notebooks:**

[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sidequery/sidemantic/blob/main/examples/notebooks/sidemantic_sql_duckdb_demo.ipynb) SQL + DuckDB

[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sidequery/sidemantic/blob/main/examples/notebooks/lookml_multi_entity_duckdb_demo.ipynb) LookML multi-entity

**SQL syntax:**
```bash
uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/sql/sql_syntax_example.py
```

**Comprehensive demo:**
```bash
uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/advanced/comprehensive_demo.py
```

**Symmetric aggregates:**
```bash
uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/features/symmetric_aggregates_example.py
```

**Superset with DuckDB:**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/superset_demo/run_demo.py
```

**Cube Playground:**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/cube_demo/run_demo.py
```

**Rill Developer:**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/rill_demo/run_demo.py
```

**OSI (complex adtech semantic model):**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/osi_demo/run_demo.py
```

**OSI widget notebook (percent-cell Python notebook):**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/osi_demo/osi_widget_notebook.py
```

See `examples/` for more.

## Core Features

- SQL query interface with automatic rewriting
- Automatic joins across models
- Multi-format adapters (Cube, MetricFlow, LookML, Hex, Rill, Superset, Omni, BSL, GoodData LDM, OSI, AtScale SML, ThoughtSpot TML, Graphene GSQL)
- SQLGlot-based SQL generation and transpilation
- Pydantic validation and type safety
- Pre-aggregations with explicit routing
- Predicate pushdown for faster queries
- Segments and metric-level filters
- Jinja2 templating for dynamic SQL
- PostgreSQL wire protocol server for BI tools
- HTTP API with JSON and Arrow IPC responses

## Multi-Format Support

Auto-detects: Sidemantic (SQL/YAML), Cube, MetricFlow (dbt), LookML, Hex, Rill, Superset, Omni, BSL, GoodData LDM, OSI, AtScale SML, ThoughtSpot TML, Graphene GSQL

```bash
sidemantic query "SELECT revenue FROM orders" --models ./my_models
```

```python
from sidemantic import SemanticLayer, load_from_directory

layer = SemanticLayer(connection="duckdb:///data.duckdb")
load_from_directory(layer, "my_models/")  # Auto-detects formats
```

## Databases

| Database | Status | Installation |
|----------|:------:|--------------|
| DuckDB | ✅ | built-in |
| MotherDuck | ✅ | built-in |
| PostgreSQL | ✅ | `uv add sidemantic[postgres]` |
| BigQuery | ✅ | `uv add sidemantic[bigquery]` |
| Snowflake | ✅ | `uv add sidemantic[snowflake]` |
| ClickHouse | ✅ | `uv add sidemantic[clickhouse]` |
| Databricks | ✅ | `uv add sidemantic[databricks]` |
| Spark SQL | ✅ | `uv add sidemantic[spark]` |

## Docker

The published image is [`sidequery/sidemantic`](https://hub.docker.com/r/sidequery/sidemantic) on Docker Hub. Mount your models directory as a volume at `/app/models`:

```bash
docker run -p 5433:5433 -v ./models:/app/models sidequery/sidemantic
```

Demo mode (built-in sample data, no volume needed):

```bash
docker run -p 5433:5433 sidequery/sidemantic --demo
```

See [`examples/docker/`](examples/docker/) for MCP mode, env vars, building from source, and integration test services.

For Cloudflare Worker + Container deployment, see [`examples/cloudflare_containers/`](examples/cloudflare_containers/).

## HTTP API

Start the API server:

```bash
uvx --from "sidemantic[api]" sidemantic api-serve models/ --db data.duckdb --port 4400 --auth-token secret
```

Compile a structured semantic query:

```bash
curl -s http://localhost:4400/compile \
  -H "Authorization: Bearer secret" \
  -H "Content-Type: application/json" \
  -d '{"dimensions":["orders.status"],"metrics":["orders.total_amount"]}'
```

Run a structured query as JSON:

```bash
curl -s http://localhost:4400/query \
  -H "Authorization: Bearer secret" \
  -H "Content-Type: application/json" \
  -d '{"dimensions":["orders.status"],"metrics":["orders.total_amount","orders.order_count"]}'
```

Run a structured query as Arrow IPC:

```bash
curl -s http://localhost:4400/query \
  -H "Authorization: Bearer secret" \
  -H "Accept: application/vnd.apache.arrow.stream" \
  -H "Content-Type: application/json" \
  -d '{"metrics":["orders.order_count"]}' \
  > result.arrow
```

Execute rewritten SQL over HTTP:

```bash
curl -s http://localhost:4400/sql \
  -H "Authorization: Bearer secret" \
  -H "Content-Type: application/json" \
  -d '{"query":"SELECT status, total_amount FROM orders ORDER BY status"}'
```

## Agent Skill

Sidemantic ships an [agent skill](skills/sidemantic-modeler/) that teaches Claude Code, Codex, and other `SKILL.md`-compatible agents to build, validate, and query semantic models.

**One-liner install (no clone required):**

```bash
curl -fsSL https://raw.githubusercontent.com/sidequery/sidemantic/main/skills/install.sh | bash
```

**npx / bunx:**

```bash
npx skills add https://github.com/sidequery/sidemantic --skill sidemantic-modeler
# or
bunx skills add https://github.com/sidequery/sidemantic --skill sidemantic-modeler
```

## How mature is Sidemantic?

Sidemantic is an ambitious but young semantic layer project. You could encounter rough patches, especially with the more exotic features like converting between semantic model formats or serving semantic layers via the included Postgres protocol server.

## Testing

```bash
uv run pytest -v
```

This prints line coverage for `sidemantic` with missing lines in the terminal.
