06 — Secrets and identity

This repo is values + cached charts, all of it world-readable from Git. No secret value should ever exist in a file here. The platform pattern is to express secrets as references and let the in-cluster machinery materialize them.

The components

External Secrets Operator (ESO)

Each cluster carries an external-secrets/ override directory. ESO runs in the cluster, reads ExternalSecret CRs, fetches the secret value from a backend (GCP Secret Manager or Vault), and materializes a Kubernetes Secret for workloads to mount.

The ExternalSecret CR refers to a backend by name only (e.g., gcpsm/prod/<service>/api-token). The CR is checked into Git; the value is not.
The backend is configured per-cluster in external-secrets/custom-values.yaml.

GCP Secret Manager

The default backend for most prod clusters. Secrets are project-scoped under the cluster's GCP project. Workload Identity binds the ESO service account to a Google service account that has secretmanager.secretAccessor on the relevant secrets.

Vault

Used where additional capabilities are required — dynamic credentials, transit encryption, PKI. Vault runs in-cluster on Raft HA. Edits to Vault overrides (helm-overrides/<cluster>/vault/custom-values.yaml) — especially seal config, HA storage, autounseal — require platform-team review. Vault HA write-path failure is a paging incident. See ../docs/global/escalation-matrix.md row 8.

Workload Identity

GKE Workload Identity binds Kubernetes service accounts to Google service accounts via the iam.gke.io/gcp-service-account annotation. This is how pods authenticate to GCP APIs (Secret Manager, Cloud Storage, Pub/Sub) without long-lived keys.

The annotation is set in the chart's values (serviceAccount.annotations) — agent-editable. The IAM binding itself is set up out-of-band via Terraform in the platform IaC repo, not here.

What never goes in this repo

Plain-text passwords, API tokens, certificates, private keys.
Base64-encoded secrets in Secret manifests.
TLS keys / certs (use cert-manager issuers + ESO references instead).
GCP service-account JSON keys (Workload Identity replaces them).
OAuth client secrets (Secret Manager → ESO).
Webhook URLs that contain a credential token in the path.

If the value would be useful to an attacker who clones this repo, it does not belong here.

TruffleHog — last line of defense

The pre-commit hook scans staged content for high-entropy strings and known secret formats. NEVER bypass.

git commit --no-verify is blocked by Sanctity rule.
If the hook flags a real secret, rotate the credential first (the moment it touched a Git working tree it is already half-burned), then move it to ESO.
If the hook flags a false positive, fix the regex in pre-commit-scripts/ rather than excluding the file.

Detail: ./08-pre-commit-and-hooks.md.

The pre-commit hook telemetry exception

The post-commit Cursor metric collector POSTs to observe.meeshogcp.in. This is platform-managed automation, not agent-initiated, and is the only outbound call to a *.meeshogcp.in host the agent will ever observe in this repo. The agent must still refuse any new call to such hosts. See ../docs/global/SANCTITY_RULES.md.

Common patterns

Pattern	Looks like
Pod reads a Secret Manager value	`ExternalSecret` CR → ESO materializes `Secret` → pod mounts via `envFrom.secretRef` or `volumes.secret`
Pod calls a GCP API	KSA annotated with `iam.gke.io/gcp-service-account: <gsa>@<project>.iam.gserviceaccount.com` (Workload Identity)
TLS for an Ingress	`cert-manager` `Certificate` CR + Vault PKI or Let's Encrypt issuer
Vault dynamic DB credential	Vault DB secrets engine + ESO `VaultDynamicSecret` (or app-side Vault Agent sidecar)

4.4 KiB Raw Blame History