# Mapping: source -> Learn URL

The single most important fact to internalize about Learn:
**source filesystem path is irrelevant for routing**. A page's
URL on `learn.netdata.cloud` is computed from frontmatter that
`ingest.py` INJECTS from `<repo>/docs/.map/map.yaml`. Without
this mental model, every other rule seems arbitrary.

## The map.yaml source of truth

`<repo>/docs/.map/map.yaml` is the canonical declaration of
which files are published on Learn and where they appear. It is
a hierarchical tree of nodes; each leaf node represents one
published page.

Schema: `<repo>/docs/.map/map.schema.json`. Authoring guide:
`<repo>/docs/.map/README.md`.

### Per-node fields

Each `meta` block declares one publication node:

| Field | Type | Required | Notes |
|---|---|---|---|
| `meta.label` | string | yes | Display name. Becomes `sidebar_label` AND drives the destination filename. |
| `meta.edit_url` | string | yes (for doc-emitting nodes) | Must match `^https://github\.com/netdata/<repo>/edit/<branch>/.+\.(md\|mdx)$`. The unique key ingest uses to look up source files. Optional ONLY for category-only nodes that contain integration placeholders. |
| `meta.path` | string | no | Single-segment override of the URL path component if it should differ from the label. |
| `meta.description` | string | no | Frontmatter description; powers the page's meta description. |
| `meta.keywords` | array<string> | no | Frontmatter keywords; powers in-app search and Learn page metadata. |

The schema sets `additionalProperties: false`. Unknown keys
fail validation and abort the run with exit code 2
(`ingest/ingest.py:2815-2819`).

### Branches

For all repos in the source list (see `pipeline.md`), the
`edit_url` branch is `master` -- with one exception: the
`netdata/.github` repo uses `main` branch instead of `master`
(`ingest.py:1232-1235`).

### Integration placeholders

`integration_placeholder: true` rows in `map.yaml` are
substituted at ingest time by integration pages
auto-discovered from the source repos. The `populate_integrations`
step (`ingest.py:752-1003`) finds every `.md` carrying the
`DO NOT EDIT THIS FILE DIRECTLY` marker
(`INTEGRATION_MARKER`, `ingest.py:113`), parses its hidden
`<!--startmeta...endmeta-->` block, buckets by category
(collectors / exporters / secretstore / functions /
authentication / cloud-notifications / agent-notifications /
logs), sorts by `(learn_rel_path, sidebar_label)`, and
**inserts the rows in place of** the matching placeholder.
Result is written to
`${NETDATA_REPOS_DIR}/learn/ingest/generated_map.yaml`.

So integration pages do NOT need explicit `map.yaml` rows;
they are pulled in via the placeholder mechanism.

## Frontmatter that ingest INJECTS

When ingest matches a source file's `custom_edit_url` to a
`map.yaml` row, it writes a hidden HTML-comment metadata block
at the top of the source file. After `sanitize_page` runs, the
comment delimiters become real YAML frontmatter (`<!--` ->
`---`, `-->` -> `---`).

Per `ingest.py:1221-1299` plus `ingest.py:1832-1897`:

| Field | Source | Notes |
|---|---|---|
| `custom_edit_url` | `meta.edit_url` from `map.yaml` | Used by the "Edit this page" link. For autogenerated grid pages, this is `null`. |
| `sidebar_label` | `meta.label` | Drives sidebar AND destination filename. |
| `learn_status` | `"Published"` or `"AUTOGENERATED"` | Anything other than `Published` is excluded from `to_publish`. |
| `learn_rel_path` | reconstructed from `map.yaml` hierarchy | Drives the destination directory tree. `root` means write to `docs/` directly. |
| `keywords` | `meta.keywords` | Normalized to inline YAML array. CSV strings, brackets, and Python lists all accepted. |
| `description` | `meta.description` | Skipped when empty/None to minimize diffs. |
| `sidebar_position` | computed from map traversal order | Always written as quoted string `sidebar_position: "10"`. |
| `slug` | computed (`/<learn_rel_path>/<label>` lowercased) | Author-supplied `slug:` in source is preserved as override (`ingest.py:1890-1895`). |
| `learn_link` | `https://learn.netdata.cloud/docs<slug>` | Updated each ingest. Validated daily by `check_learn_links.py`. |

Special case: paths containing `Collecting Metrics` get
`toc_max_heading_level: 6` and `toc_collapsible: true` injected
(`ingest.py:1251-1253`).

## Source-path-to-URL computation

For files marked `learn_status: Published`, the destination is
computed by `create_mdx_path_from_metadata`
(`ingest.py:1140-1204`):

```
docs/<learn_rel_path>/<sanitized_sidebar_label>.mdx
```

### Sanitization rules

`sidebar_label` is sanitized for the filename (`ingest.py:1159-1175`):
1. Strip `'`, `:`, `/`, `(`, `)`, `,`, backtick.
2. Collapse repeated whitespace to single space.

For the URL **slug** (different from filename), additionally:
- Lowercase everything.
- Spaces -> `-`.
- `//` -> `/`.

Filename keeps original case; slug is lowercase. So
`docs/Welcome to Netdata/Welcome to Netdata.mdx` exists with
spaces and capitals.

### Special-cases

1. **Functions integrations** -- if the source `custom_edit_url`
   is under `/integrations/functions/`, the filename uses the
   function slug from the URL stem (with `-` -> ` `) instead of
   `sidebar_label` (`ingest.py:1153-1157`). Avoids collisions
   when many integrations share a "Top Queries" label.

2. **Category overview pages** -- if the last two segments of
   the slug are equal (e.g. `/collecting-metrics/collecting-metrics`),
   `create_mdx_path_from_metadata` returns the slug with the
   duplicate trimmed (`ingest.py:1178-1191`). On disk the file
   is still written as `docs/<X>/<X>.mdx`; Docusaurus then
   routes it to the parent URL `/<X>` because it has the same
   slug.

3. **Frontmatter `slug:` override** -- if the source file
   already has `slug:` declared in frontmatter
   (custom-authored, like `docs/ask-nedi.mdx`), that value wins
   (`ingest.py:1890-1895`). For these files, the
   author-supplied slug must be set BEFORE the
   `<!-- ... -->` metadata block so it survives the rewrite.

4. **`id:` in mapDict** -- if the metadata in `mapDict` carries
   an `id`, the last segment of the URL is replaced by the id
   (`ingest.py:2120-2126`).

### Edge cases

- `README.md`, `index.md`, `_index.md`, `index.mdx`, hidden
  directories: NONE of these are special-cased in `ingest.py`.
  The Python pipeline never publishes a file unless it appears
  in `map.yaml` (or comes from an integration placeholder), so
  filename has no routing effect.
- Files at repo root vs deep paths: same rule -- destination is
  driven by `map.yaml.meta.label` + the node's position in the
  hierarchy, not by source path depth.
- Special filenames: ignored. Only `map.yaml` matters.

## Examples

### Add a new top-level page

1. Create `<repo>/docs/getting-started-netdata/quick-tour.md`.
   The exact path doesn't matter for routing -- use a
   reasonable location.
2. Add to `<repo>/docs/.map/map.yaml`:
   ```yaml
   - meta:
       label: Quick Tour
       edit_url: https://github.com/netdata/netdata/edit/master/docs/getting-started-netdata/quick-tour.md
       description: A 5-minute tour of Netdata.
       keywords: [tour, getting-started, intro]
   ```
   under the appropriate parent.
3. Resulting Learn URL: `https://learn.netdata.cloud/docs/<parent-slug>/quick-tour`.
4. Resulting filename in learn repo: `docs/<parent>/Quick Tour.mdx` (with spaces and capitals).

### Move a page (different sidebar location, same source file)

1. In `map.yaml`, move the node to its new parent.
2. Keep `meta.edit_url` unchanged -- still points to the same
   source file.
3. After ingest, the new `learn_rel_path` is computed from the
   new tree position; the diff between previous and current
   target produces an automatic Netlify redirect from the old
   URL to the new (see `redirects.md`).

### Rename a page

Same as a move -- the URL slug is derived from `sidebar_label`,
so changing `meta.label` changes the URL. Old URL is
auto-redirected.

### Delete a page

The map.yaml side is just removing the node and the source
file. There is one manual step on the learn-repo side
(redirect surgery in
`LegacyLearnCorrelateLinksWithGHURLs.json`); see
`redirects.md` and `recipes/delete-doc-page.md`.

## Path-collision rule

If two `to_publish` entries collide case-insensitively
(`<learn_rel_path>, <sidebar_label>` matches), ingest emits a
warning at `ingest.py:2891-2908`. On case-insensitive
filesystems (macOS / Windows), one file silently overwrites
the other. On Linux they coexist as separate files.

## What lookups happen

The lookup key is the **canonical edit URL**:

```
https://github.com/netdata/<repo>/edit/<branch>/<repo-rel-path>
```

(`ingest.py:1232-1235`). Branch is `master` for everything
except `.github`, which uses `main`. This is the join key
between ingest's filesystem walk and `map.yaml`.

If a source file's edit URL doesn't match any `map.yaml` row,
the file is skipped. No warning -- silent skip.

## Inter-page linking

Markdown links between Learn pages must use the **GitHub-relative
path with `.md` extension** — NOT the learn.netdata.cloud URL,
and NOT a bare slug without extension. The ingest pipeline
rewrites these links to the correct Learn URLs during processing.

**Correct:**

```markdown
See [Configuration](/docs/network-flows/configuration.md) for details.
```

**Wrong (will NOT resolve on Learn or GitHub):**

```markdown
See [Configuration](/network-flows/configuration) for details.
```

The link path is the repo-relative path to the source `.md` file,
prefixed with `/docs/`. The ingest pipeline matches it against the
`map.yaml` tree and rewrites it to the final Learn URL.

This also means links work natively on GitHub — readers browsing
the source repo can click through to the linked file.
