# Artifacts and banners

Every file the integrations pipeline reads or writes, with
banner conventions and edit rules.

## Tracked vs runtime artifacts

| Path | Producer | Tracked in git? | Auto-regenerated by |
|---|---|---|---|
| `integrations/integrations.js` | `gen_integrations.py:1311-1325` | NO -- gitignored (`.gitignore:159`) | Every CI run; `rm`-ed by the workflow before opening the PR |
| `integrations/integrations.json` | `gen_integrations.py:1311-1325` (clean variant) | NO -- gitignored (`.gitignore:160`) | as above |
| `<plugin-dir>/integrations/<slug>.md` (per-integration) | `gen_docs_integrations.py` | **YES (committed)** | `generate-integrations.yml` auto-PR; verified in `check-markdown.yml` |
| `<plugin-dir>/README.md` (symlink) | `gen_docs_integrations.py:make_symlinks` (`:527-544`) | **YES** (the symlink itself) | as above |
| `src/health/notifications/<dir>/README.md` (DIRECT, not a symlink) | `gen_docs_integrations.py:agent_notification branch` (`:488-496`) | **YES** | as above |
| `src/collectors/COLLECTORS.md` | `gen_doc_collector_page.py` | **YES** | as above |
| `src/collectors/SECRETS.md` | `gen_doc_secrets_page.py` | **YES** | as above |
| `src/collectors/SERVICE-DISCOVERY.md` | `gen_doc_service_discovery_page.py` | **YES** | **NOT** in `generate-integrations.yml` -- manual or cmake `render-docs` only -- see `gotchas.md` |
| `src/go/plugin/ibm.d/modules/<m>/metadata.yaml` | ibm.d `docgen` | **YES** | manual `go generate ./...` -- never CI |
| `src/go/plugin/ibm.d/modules/<m>/README.md` | ibm.d `docgen` | **YES** | as above |
| `src/go/plugin/ibm.d/modules/<m>/config_schema.json` | ibm.d `docgen` | **YES** | as above |
| `src/go/plugin/ibm.d/modules/<m>/contexts/zz_generated_contexts.go` | ibm.d `metricgen` | **YES** | as above |
| Hand-written `metadata.yaml` (non-ibm.d), `config_schema.json`, stock `.conf`, `health.d/<...>.conf`, hand-written `README.md` | collector author | **YES** | none -- author edits + commits manually |

## Banner conventions per file kind

### `integrations/integrations.js`

Top of file (`integrations/templates/integrations.js:1-2`):

```js
// DO NOT EDIT THIS FILE DIRECTLY
// It gets generated by integrations/gen_integrations.py in the Netdata repo
```

Body shape:

```js
export const categories = [/* JSON array of category trees */];
export const integrations = [/* JSON array of integration objects */];
```

The body uses the rendered-with-folding/`{% details %}` markers
(the `clean=False` variant). Consumed by the cloud-frontend
dashboard which has its own renderer that interprets these
markers.

### `integrations/integrations.json`

Pure JSON, no banner. Top-level shape:

```json
{ "categories": [...], "integrations": [...] }
```

Uses the `clean=True` variant (markers stripped). Consumed by
`scripts/checkIntegrations.js` in the dashboard repo (validation
fetch from `https://raw.githubusercontent.com/netdata/netdata/master/integrations/integrations.json`)
and by anything that needs GitHub-renderable markdown.

### Per-integration `.md` (collector, exporter, secretstore, etc.)

Every per-integration `.md` opens with a `<!--startmeta`
HTML-comment block. This block is consumed by the Learn ingest
pipeline (`<repo>/docs/.map/map.yaml` row matching is by
`custom_edit_url`).

Schema:

```markdown
<!--startmeta
custom_edit_url: "https://github.com/netdata/netdata/edit/master/<plugin-dir>/integrations/<slug>.md"
meta_yaml: "https://github.com/netdata/netdata/edit/master/<plugin-dir>/metadata.yaml"
sidebar_label: "<display name>"
learn_status: "Published"
learn_rel_path: "<Learn category path>"
keywords: ['k1', 'k2', ...]                                    # only when metadata has keywords
most_popular: true                                             # only when metadata sets it
message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE"
endmeta-->
```

The exact wording of `message:` varies by type. From
`gen_docs_integrations.py`:

| Integration type | `message:` text |
|---|---|
| `collector` | `DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE` |
| `exporter` | `... GENERATED BY THE EXPORTER'S metadata.yaml FILE` |
| `agent_notification`, `cloud_notification` | `... GENERATED BY THE NOTIFICATION'S metadata.yaml FILE` |
| `logs` | `... GENERATED BY THE LOGS' metadata.yaml FILE` |
| `authentication` | `... GENERATED BY THE AUTHENTICATION'S metadata.yaml FILE` |
| `secretstore` | `... GENERATED BY THE SECRETSTORE'S metadata.yaml FILE` |
| `service_discovery` | `... GENERATED BY THE SERVICE DISCOVERY DISCOVERER'S metadata.yaml FILE` |

After the banner, the script injects a community-or-Netdata
badge image before the first `##` (`:422-437`). Badge color
depends on `meta.community` boolean.

### `learn_rel_path` mapping

The first category in `meta.monitored_instance.categories` is
remapped to a Learn navigation path (`gen_docs_integrations.py`
per-type branches):

| Integration type | Mapping |
|---|---|
| `collector` | `data-collection.<x>` -> `Collecting Metrics/Collectors/<x>` |
| `exporter` | hardcoded `Exporting Metrics/Connectors` |
| `agent_notification` | `<x>` with `notifications` -> `Alerts & Notifications/Notifications` |
| `cloud_notification` | same as above |
| `logs` | `<x>` with `logs` -> `Logs` |
| `authentication` | `<x>` with `authentication` -> `Netdata Cloud/Authentication & Authorization/Cloud Authentication & Authorization Integrations` |
| `secretstore` | hardcoded `Collecting Metrics/Secrets Management/Secret Stores` |
| `service_discovery` | hardcoded `Collecting Metrics/Service Discovery` |

### `<plugin-dir>/README.md` (symlink case)

When the symlink rule fires (single-integration directory), the
README is a symlink to `integrations/<slug>.md`. The symlink
target's banner is the per-integration banner above, with two
adjustments:
- `custom_edit_url` points at the README path, not the
  integration file path -- so editing through Learn opens the
  README.
- The body has `{element}/{symlinks[element]}` replaced with
  `{element}/README.md` (`:542-544`) so internal anchor refs
  don't break.

### `src/health/notifications/<dir>/README.md` (direct case)

Same banner as above. Written DIRECTLY, not as a symlink. Keep
in mind for the five-file consistency rule -- the README.md is
the generated artifact.

### `src/collectors/COLLECTORS.md`, `SECRETS.md`, `SERVICE-DISCOVERY.md`

**No generated-file banner.** `COLLECTORS.md` currently opens
with `<!-- markdownlint-disable-file -->` and then its marketing
header (`# Monitor anything with Netdata`). `SECRETS.md` and
`SERVICE-DISCOVERY.md` open with their marketing headers
(`# Secrets Management`, `# Service Discovery`). This is a
known oversight -- there is no DO-NOT-EDIT marker on these
files even though they are generated. Editing them directly
will be silently overwritten on the next CI run (for
COLLECTORS.md and SECRETS.md). For SERVICE-DISCOVERY.md the
absence of CI wiring means manual edits stick until someone
runs the script.

`COLLECTORS.md` is the Learn page titled "Monitor anything with
Netdata". It is generated by `integrations/gen_doc_collector_page.py`
from `integrations/integrations.js`. The generator groups normal
collector entries by children of `data-collection` and also treats
the top-level `flows` category as a first-class section named
`Network Flows`; without that special case, NetFlow / IPFIX / sFlow
and flow enrichment integrations either appear in the wrong generic
section or are easy to miss.

To update the page, do not edit the generated table rows in
`src/collectors/COLLECTORS.md`. Edit the source
`metadata.yaml` / `integrations/categories.yaml` or the
generator, then run:

```bash
python3 integrations/gen_integrations.py
python3 integrations/gen_doc_collector_page.py
```

### ibm.d-generated files

| File | Banner |
|---|---|
| `metadata.yaml` | First line: `# Generated metadata.yaml for <module> module` (template at `src/go/plugin/ibm.d/docgen/main.go:562`) |
| `zz_generated_contexts.go` | Filename prefix `zz_generated_`. Package doc at `<module>/contexts/doc.go:2-3` declares `DO NOT EDIT zz_generated_contexts.go - it is auto-generated from contexts.yaml`. |
| `README.md` | Banner depends on `docgen`'s readme template. Includes module info, metric tables, config tables. |
| `config_schema.json` | Pure JSON, no banner. |

See `ibm-d.md` for the full chain.

## Maintainer rules

1. **Per-integration `.md` files: NEVER edit by hand.** Edit
   the source `metadata.yaml`, regenerate, commit.
2. **Symlinked `README.md` files: NEVER edit by hand.** Same
   reason -- they point at the generated `.md`.
3. **`src/health/notifications/<dir>/README.md`: NEVER edit by
   hand.** Direct generated file, looks identical to a normal
   README -- the `<!--startmeta` banner is the giveaway.
4. **`src/collectors/COLLECTORS.md` / `SECRETS.md`: NEVER edit
   the dynamic tables.** Edit the source `metadata.yaml`. The
   static prose ("# Monitor anything with Netdata", the
   secret-store onboarding text) is baked into the generator
   scripts -- to change static prose, edit
   `gen_doc_collector_page.py` / `gen_doc_secrets_page.py` and
   commit.
5. **`src/collectors/SERVICE-DISCOVERY.md`: same rule.** Caveat:
   the script is not in CI, so manual regen is required after
   editing source metadata until that gap is fixed.
6. **ibm.d generated files: NEVER edit.** Edit `contexts.yaml`,
   `config.go`, or `module.yaml` instead and re-run
   `go generate`.
