> Per AI Blitz Plan §claude. Layer: 1. Repo: devops-infra-helm-charts. # 05 — Deploy lifecycle End-to-end story of how a values change reaches a live cluster, where the safety gates are, and what happens when they fail. ## The path ``` edit override (branch) → commit (TruffleHog runs) → push → PR → reviewer approves → merge to main → Argo CD on cluster reconciles (auto-sync OR human Sync click) → manifests applied → workload changes ``` ## Stage 1 — Branch and edit - Branch off `main`. Never push to `main` directly. - Edit one `helm-overrides///custom-values.yaml` (or its raw sidecars). - No drive-by edits, no cross-cluster normalization in the same PR. See [`../docs/global/SANCTITY_RULES.md`](../docs/global/SANCTITY_RULES.md). ## Stage 2 — Local validation - `helm template` against the override — confirms render succeeds. - For wrappers: `helm dependency update` first. - For raw sidecars: `kubectl apply --dry-run=client`. If render fails locally, it will fail in Argo CD's `OutOfSync → SyncFailed`. Fix before pushing. ## Stage 3 — Commit `git commit` triggers pre-commit hooks: - **TruffleHog** — blocking. Real secrets bounce. Never `--no-verify`. - **CAC, Yaak** — gated on paths this repo doesn't have, no-op. Post-commit: - **Cursor metric collector** — background, non-blocking. Pings `observe.meeshogcp.in` with commit telemetry. Failure here does not block. Detail: [`./08-pre-commit-and-hooks.md`](./08-pre-commit-and-hooks.md). ## Stage 4 — PR + review (safety gate 1) The reviewer's job: 1. Confirm the change touches only the cluster × app named in the PR. 2. Confirm any per-cluster scheduling fields were rewritten, not copy-pasted. 3. Confirm `fullnameOverride` is unchanged. 4. Confirm no secret materializes in the file. 5. Confirm chart `Chart.yaml` dep bumps came with `Chart.lock` refresh and a changelog reference. 6. Confirm versioned-sibling deletes have no sister-repo references. If a chart fork is suspected, escalate per [`../docs/global/escalation-matrix.md`](../docs/global/escalation-matrix.md) row 1. ## Stage 5 — Merge to `main` Merging is the deploy event. Argo CD on every cluster whose Application points at the changed path will move to `OutOfSync`. ## Stage 6 — Argo CD reconcile (safety gate 2) Two reconciliation modes, set per Application in the sister repo: - **Manual sync** (prod default for infra) — Argo waits for a human Sync click. Engineer reviews the diff in the Argo UI before applying. - **Auto-sync** — Argo applies on its own. Reserved for low-risk leaves (`kube-state-metrics`, monitoring agents). Background on the manual-sync default: [`../wiki/analyses/ADR-A5-manual-sync-default-for-infra.md`](../wiki/analyses/ADR-A5-manual-sync-default-for-infra.md). The Argo Application also defines: - `syncPolicy.automated.prune` — whether Argo deletes objects no longer in Git. - `syncPolicy.automated.selfHeal` — whether Argo reverts manual cluster edits. - `syncOptions` — `CreateNamespace`, `ServerSideApply`, `RespectIgnoreDifferences`, retry/backoff. - Sync waves via annotations (in chart templates or sidecars). These all live in the **sister repo**, not here. See [`../docs/global/coding-guidelines/argocd.md`](../docs/global/coding-guidelines/argocd.md). ## Failure modes | Failure | Where it surfaces | Read | |---------|------------------|------| | Render error in `helm template` | Argo Application status `ComparisonError` | Re-render locally; fix values | | `OutOfSync → SyncFailed` after Sync click | Argo UI events | [`../docs/platform/runbooks/argocd-sync-failure.md`](../docs/platform/runbooks/argocd-sync-failure.md) | | Pods land but stay `Pending` | `kubectl get pods` on target cluster | [`../docs/platform/runbooks/pod-pending-scheduling.md`](../docs/platform/runbooks/pod-pending-scheduling.md) | | Ingress 5xx after Contour change | `contour-external` envoy logs / synthetic probes | [`../docs/platform/runbooks/ingress-down.md`](../docs/platform/runbooks/ingress-down.md) | | Drift reappears after `kubectl edit` | `selfHeal: true` doing its job | Edit Git, not the cluster | ## Sister-repo coupling Almost every non-trivial change is a **paired PR**: - New app on cluster: PR here (override) + PR in sister repo (Application). - New cluster: PR here (cluster directory) + PR in sister repo (`ApplicationSet` cluster generator). - Blue-green sibling cutover: PR here (sibling values) + PR in sister repo (Application `targetRevision` / chart path). Procedures: [`../docs/platform/procedures/onboard-app-to-cluster.md`](../docs/platform/procedures/onboard-app-to-cluster.md), [`../docs/platform/procedures/onboard-new-cluster.md`](../docs/platform/procedures/onboard-new-cluster.md), [`../docs/platform/procedures/blue-green-chart-migration.md`](../docs/platform/procedures/blue-green-chart-migration.md), [`../docs/platform/procedures/deboard-app.md`](../docs/platform/procedures/deboard-app.md). ## See also - [`./00-overview.md`](./00-overview.md) - [`./04-override-hierarchy.md`](./04-override-hierarchy.md) - [`../docs/architecture.md`](../docs/architecture.md) - [`../docs/global/agent-operations-guide.md`](../docs/global/agent-operations-guide.md)