added repo
This commit is contained in:
Your Name
@@ -0,0 +1,103 @@
|
||||
> Per AI Blitz Plan §claude. Layer: 1. Repo: devops-infra-helm-charts.
|
||||
|
||||
# 05 — Deploy lifecycle
|
||||
|
||||
End-to-end story of how a values change reaches a live cluster, where the safety gates are, and what happens when they fail.
|
||||
|
||||
## The path
|
||||
|
||||
```
|
||||
edit override (branch) → commit (TruffleHog runs) → push → PR
|
||||
→ reviewer approves → merge to main
|
||||
→ Argo CD on cluster reconciles (auto-sync OR human Sync click)
|
||||
→ manifests applied → workload changes
|
||||
```
|
||||
|
||||
## Stage 1 — Branch and edit
|
||||
|
||||
- Branch off `main`. Never push to `main` directly.
|
||||
- Edit one `helm-overrides/<cluster>/<app>/custom-values.yaml` (or its raw sidecars).
|
||||
- No drive-by edits, no cross-cluster normalization in the same PR. See [`../docs/global/SANCTITY_RULES.md`](../docs/global/SANCTITY_RULES.md).
|
||||
|
||||
## Stage 2 — Local validation
|
||||
|
||||
- `helm template` against the override — confirms render succeeds.
|
||||
- For wrappers: `helm dependency update` first.
|
||||
- For raw sidecars: `kubectl apply --dry-run=client`.
|
||||
|
||||
If render fails locally, it will fail in Argo CD's `OutOfSync → SyncFailed`. Fix before pushing.
|
||||
|
||||
## Stage 3 — Commit
|
||||
|
||||
`git commit` triggers pre-commit hooks:
|
||||
|
||||
- **TruffleHog** — blocking. Real secrets bounce. Never `--no-verify`.
|
||||
- **CAC, Yaak** — gated on paths this repo doesn't have, no-op.
|
||||
|
||||
Post-commit:
|
||||
|
||||
- **Cursor metric collector** — background, non-blocking. Pings `observe.meeshogcp.in` with commit telemetry. Failure here does not block.
|
||||
|
||||
Detail: [`./08-pre-commit-and-hooks.md`](./08-pre-commit-and-hooks.md).
|
||||
|
||||
## Stage 4 — PR + review (safety gate 1)
|
||||
|
||||
The reviewer's job:
|
||||
|
||||
1. Confirm the change touches only the cluster × app named in the PR.
|
||||
2. Confirm any per-cluster scheduling fields were rewritten, not copy-pasted.
|
||||
3. Confirm `fullnameOverride` is unchanged.
|
||||
4. Confirm no secret materializes in the file.
|
||||
5. Confirm chart `Chart.yaml` dep bumps came with `Chart.lock` refresh and a changelog reference.
|
||||
6. Confirm versioned-sibling deletes have no sister-repo references.
|
||||
|
||||
If a chart fork is suspected, escalate per [`../docs/global/escalation-matrix.md`](../docs/global/escalation-matrix.md) row 1.
|
||||
|
||||
## Stage 5 — Merge to `main`
|
||||
|
||||
Merging is the deploy event. Argo CD on every cluster whose Application points at the changed path will move to `OutOfSync`.
|
||||
|
||||
## Stage 6 — Argo CD reconcile (safety gate 2)
|
||||
|
||||
Two reconciliation modes, set per Application in the sister repo:
|
||||
|
||||
- **Manual sync** (prod default for infra) — Argo waits for a human Sync click. Engineer reviews the diff in the Argo UI before applying.
|
||||
- **Auto-sync** — Argo applies on its own. Reserved for low-risk leaves (`kube-state-metrics`, monitoring agents).
|
||||
|
||||
Background on the manual-sync default: [`../wiki/analyses/ADR-A5-manual-sync-default-for-infra.md`](../wiki/analyses/ADR-A5-manual-sync-default-for-infra.md).
|
||||
|
||||
The Argo Application also defines:
|
||||
|
||||
- `syncPolicy.automated.prune` — whether Argo deletes objects no longer in Git.
|
||||
- `syncPolicy.automated.selfHeal` — whether Argo reverts manual cluster edits.
|
||||
- `syncOptions` — `CreateNamespace`, `ServerSideApply`, `RespectIgnoreDifferences`, retry/backoff.
|
||||
- Sync waves via annotations (in chart templates or sidecars).
|
||||
|
||||
These all live in the **sister repo**, not here. See [`../docs/global/coding-guidelines/argocd.md`](../docs/global/coding-guidelines/argocd.md).
|
||||
|
||||
## Failure modes
|
||||
|
||||
| Failure | Where it surfaces | Read |
|
||||
|---------|------------------|------|
|
||||
| Render error in `helm template` | Argo Application status `ComparisonError` | Re-render locally; fix values |
|
||||
| `OutOfSync → SyncFailed` after Sync click | Argo UI events | [`../docs/platform/runbooks/argocd-sync-failure.md`](../docs/platform/runbooks/argocd-sync-failure.md) |
|
||||
| Pods land but stay `Pending` | `kubectl get pods` on target cluster | [`../docs/platform/runbooks/pod-pending-scheduling.md`](../docs/platform/runbooks/pod-pending-scheduling.md) |
|
||||
| Ingress 5xx after Contour change | `contour-external` envoy logs / synthetic probes | [`../docs/platform/runbooks/ingress-down.md`](../docs/platform/runbooks/ingress-down.md) |
|
||||
| Drift reappears after `kubectl edit` | `selfHeal: true` doing its job | Edit Git, not the cluster |
|
||||
|
||||
## Sister-repo coupling
|
||||
|
||||
Almost every non-trivial change is a **paired PR**:
|
||||
|
||||
- New app on cluster: PR here (override) + PR in sister repo (Application).
|
||||
- New cluster: PR here (cluster directory) + PR in sister repo (`ApplicationSet` cluster generator).
|
||||
- Blue-green sibling cutover: PR here (sibling values) + PR in sister repo (Application `targetRevision` / chart path).
|
||||
|
||||
Procedures: [`../docs/platform/procedures/onboard-app-to-cluster.md`](../docs/platform/procedures/onboard-app-to-cluster.md), [`../docs/platform/procedures/onboard-new-cluster.md`](../docs/platform/procedures/onboard-new-cluster.md), [`../docs/platform/procedures/blue-green-chart-migration.md`](../docs/platform/procedures/blue-green-chart-migration.md), [`../docs/platform/procedures/deboard-app.md`](../docs/platform/procedures/deboard-app.md).
|
||||
|
||||
## See also
|
||||
|
||||
- [`./00-overview.md`](./00-overview.md)
|
||||
- [`./04-override-hierarchy.md`](./04-override-hierarchy.md)
|
||||
- [`../docs/architecture.md`](../docs/architecture.md)
|
||||
- [`../docs/global/agent-operations-guide.md`](../docs/global/agent-operations-guide.md)
|
||||
Reference in New Issue
Block a user