# ADR-0014: Provider-Endpoint Flag Discipline for v5 GCP Provider

## Status
Accepted

## Context

terraform-provider-google v5.x routes API calls through per-service
endpoint flags (`compute_custom_endpoint`, `sql_custom_endpoint`,
`kms_custom_endpoint`, etc.). infrafactory's generator template
(`internal/cli/generate_command.go::buildGoogleProviderBlock`) injects
those flags pointing at fakegcp so apply doesn't escape to the real
cloud. The pattern is straightforward — except when it isn't.

During the 2026-06-01 sweep we hit three repeated failure shapes that
all looked superficially different but had the same root cause —
mismatches between the URL a provider client constructs and the URL
fakegcp serves:

1. **Trailing-path doubling** (T2, T-E, T11 partial). Some lib clients
   prepend `v1/projects/...` to BasePath themselves. With
   `kms_custom_endpoint = "%s/v1/"` the URL became `/v1/v1/projects/...`
   which fakegcp 501'd. The v5 provider's strip-version regex
   (`(?P<base>http[s]://.*)(?P<version>/[^/]+?/$)`) requires `https://`
   literally, so it never fires on our `http://` fakegcp endpoints —
   the strip is a no-op and the trailing path doubles.

2. **Distinct v1 vs v3 endpoint flags** (T-D-2). The provider has
   *separate* endpoint overrides for cloudresourcemanager v1 and v3
   (`cloud_resource_manager_custom_endpoint` vs
   `resource_manager_v3_custom_endpoint`). Newer code paths
   (`google_service_networking_connection`'s getProject preflight,
   among others) use v3 only. Setting only v1 leaves v3 calls
   escaping to the real cloud, surfacing as a misleading
   `Error 401 ACCESS_TOKEN_TYPE_UNSUPPORTED` that looks like an
   auth problem but is actually a missing endpoint flag.

3. **Dual URL prefixes against the same endpoint** (T-E-2). KMS in
   particular uses TWO URL shapes: the lib client prepends
   `v1/projects/...` (so READ paths land at `/v1/projects/...`), but
   template-based code uses raw `{{KMSBasePath}}projects/...` with no
   v1 prefix (so CREATE paths land at `/projects/...`). fakegcp must
   register handlers under BOTH prefixes.

The misdiagnosis cost across these three shapes was significant: each
one's symptom looked unrelated to the others, and the v5 SDK's error
messages often misattribute the cause (auth-style errors for what is
actually a missing route).

## Decision

Three rules for endpoint-flag work going forward:

1. **Host-only by default.** New `*_custom_endpoint` entries in
   `buildGoogleProviderBlock` should default to `%s/` (host-only) and
   document with inline comments WHY when a trailing version-path is
   needed. The default-host-only-form sidesteps the strip-regex
   `https://`-requirement bug entirely.

2. **Use the v5 binary as ground truth.** When a new GCP scenario
   surfaces an "escape to real cloud" symptom, run
   `strings <provider-binary> | grep -E "GOOGLE_.*_CUSTOM_ENDPOINT"`
   to enumerate every endpoint flag the provider actually reads. If
   there are version-suffixed variants (`_V3_`, `_V2_`, etc.), set
   them all. The session that diagnosed T-D-2 discovered
   `GOOGLE_RESOURCE_MANAGER_V3_CUSTOM_ENDPOINT` this way.

3. **Dual-prefix mock routes when the symptom is split apply/read.**
   When CREATE 501s but READ 200s (or vice versa) against the same
   resource, the lib is using two URL shapes. Register the same
   handler set under both prefixes via a shared closure (see
   `fakegcp/handlers/handlers.go::registerKMSRoutes` for the canonical
   pattern landed in fakegcp `a3b1ea8`).

## Consequences

**Benefits**:
- Future endpoint-flag misses will be diagnosable in minutes
  (binary-strings hunt) rather than the multi-hour debugging sessions
  T-D-2 took.
- The pattern of "register the same handler set under multiple URL
  prefixes" generalises to other services likely to exhibit the same
  split (Compute, Container, etc., if they ever surface similar
  symptoms).
- Inline comments in `buildGoogleProviderBlock` now record the
  *why* of each non-host-only entry, so future contributors don't
  "tidy up" trailing-path values back to a form that re-triggers the
  doubling bug.

**Tradeoffs**:
- Cross-repo discipline: every endpoint-flag addition in
  `infrafactory/internal/cli/generate_command.go` may need a matching
  route in `fakegcp/handlers/handlers.go`. There's no automated
  enforcement linking the two.
- The "host-only by default" rule occasionally needs to be relaxed
  per-service. Each such relaxation should carry an inline comment
  citing the specific provider lib behaviour that requires it
  (see `cloud_resource_manager_custom_endpoint` and
  `service_usage_custom_endpoint` for the exception pattern).

**Follow-up work**:
- The remaining `*_custom_endpoint` entries that still ship with
  trailing version paths (`cloud_resource_manager_custom_endpoint`,
  `pubsub_custom_endpoint`, `storage_custom_endpoint`,
  `secret_manager_custom_endpoint`, `redis_custom_endpoint`,
  `cloud_run_v2_custom_endpoint`) should be audited per rule 2 — for
  each, confirm whether a matching `_V3_` flag exists and whether
  the trailing path is load-bearing or a no-op. Tracked as T11
  in `docs/NEXT_SESSION.md` (partial).

## 2026-06-02 amendment — auth pipeline discipline (rule 4)

The 2026-06-02 deterministic sweep (post-classifier-fix) surfaced a
new class of "looks like endpoint escape but isn't" failure on
`google_project_service` (and `google_service_networking_connection`).
Symptoms identical to a v1/v3 misroute (`401 ACCESS_TOKEN_TYPE_UNSUPPORTED`
allegedly from `cloudresourcemanager.googleapis.com`), but every
`*_custom_endpoint` was correct and fakegcp's handler verifiably
returned 200 to curl with the same bearer token.

Root cause: when `user_project_override` defaults to true OR the
parent process has any `GOOGLE_APPLICATION_CREDENTIALS`,
`GOOGLE_CREDENTIALS`, `GOOGLE_OAUTH_ACCESS_TOKEN`,
`GOOGLE_CLOUD_KEYFILE_JSON`, `CLOUDSDK_*`, or `GCLOUD_*` env var set,
the v5 SDK's auth pipeline bypasses the `access_token` short-circuit
and probes the metadata server / token-exchange BEFORE the request
reaches the configured BasePath. The 401 surfaces with
`cloudresourcemanager.googleapis.com` in its `service` field because
that's the API the SDK was *about* to call — the actual escape happens
one layer up, in auth.

**Rule 4: Auth pipeline discipline.**
The Google provider block must include:
1. `access_token = "fake-token"` (already required by rule 1)
2. `user_project_override = false` (disables x-goog-user-project quota
   header → no user-account OAuth requirement)
3. NOT a `credentials` attribute. The v5 provider rejects HCL that
   sets both with a fatal `Invalid Attribute Combination` error.
   `access_token` alone suffices for the bearer; ADC probing is
   prevented by stripping the env-var family in the subprocess.

AND the harness must strip the env-var families above from the tofu
subprocess environment. Stripping is in
`internal/cli/exec_runner.go::stripGCPAuthEnv`; the provider-block
fields are templated in
`internal/cli/generate_command.go::buildGoogleProviderBlock`.

Revision history:
- 2026-06-02 first draft of rule 4 included a stub `credentials`
  attribute alongside `access_token`. The 9-scenario re-validation
  exposed the mutual-exclusion bug — every GCP iteration failed at
  validate with `Invalid Attribute Combination`. Removed the
  `credentials` line; `access_token` + env-strip is sufficient.

**Rule 5: Batching disabled + project-level IAM resources retired.**

The v5 provider's `BatchingConfig` wrapper aggregates writes for
`google_*_iam_*` resources and `google_project_service`, constructing
its OWN `cloudresourcemanager` client that does not reliably honor
`cloud_resource_manager_custom_endpoint`. The 2026-06-02
`gcp-full-stack` sweep saw `google_project_iam_member.*` escape to
real `cloudresourcemanager.googleapis.com` with
`ACCESS_TOKEN_TYPE_UNSUPPORTED` despite every other IAM /
ResourceManager call landing cleanly on fakegcp.

The Google provider block must include:

```
batching {
  enable_batching = false
}
```

This forces each IAM mutation through the normal client path that
respects the endpoint override.

Even with batching disabled, **the v5 IAM client path for
project-level resources (`google_project_iam_member` / `_binding` /
`_policy`) bypasses the endpoint override at a deeper layer than
the batching wrapper**. The sweep confirmed iter 1 still escaped
post-batching-fix.

**Substitute**: use the SA-level IAM family
(`google_service_account_iam_member` / `_binding`) — these hit
`iam.googleapis.com` which fakegcp serves via SA-IAM round-trip
handlers (fakegcp PR #10). The GCP prompts (`prompts/gcp/phase1`,
`phase2`, `phase3`) instruct the LLM to omit project-level IAM
resources entirely, mirroring the rule-9 retirement of
`google_project_service` + `google_service_networking_connection`.

The provider-block change is templated in
`internal/cli/generate_command.go::buildGoogleProviderBlock`. The
prompt changes are in the three `prompts/gcp/phaseN*.md` files.
