added repo

2026-08-26 03:39:42 +05:30
parent 45c25a95af
commit b8575bb8b9
6889 changed files with 1217125 additions and 0 deletions
@@ -0,0 +1,83 @@
+# ADR-A2 — Versioned chart siblings for blue-green migrations
+
+> **Status:** Accepted (in production — multiple sibling pairs exist today).
+> **Repo:** `devops-infra-helm-charts`.
+> **Related:** [blue-green-chart-migration.md](../../docs/platform/procedures/blue-green-chart-migration.md), [SANCTITY_RULES R8](../../docs/global/SANCTITY_RULES.md).
+
+---
+
+## Context
+
+When a chart needs an upgrade with breaking template changes — immutable selector mismatches, removed values keys, major-version semantics — you cannot just bump `dependencies[].version` and call it done. The bump renders a different shape against the same overrides; on every cluster, the next sync would push a Helm upgrade that may fail mid-flight (immutable field) or succeed in ways that surprise the operator.
+
+The team has chosen a **versioned-sibling** pattern: keep the old chart directory live as `<chart>` and introduce the new version as `<chart>-<variant>` where the variant is one of:
+
+| Variant | Convention |
+|---------|------------|
+| `<chart>-green` | Blue-green pair (the new is "green") |
+| `<chart>-vX.Y.Z` | Pinned target version |
+| `<chart>-latest` | Work-in-progress, soon to subsume the old |
+| `<chart>-old` | Reverse pattern — `<chart>` is the new; `-old` is kept for rollback |
+
+Live examples in the repo today:
+
+- `argo-cd` ↔ `argo-cd-green`
+- `contour` ↔ `contour-v1.33.3`
+- `keda` ↔ `keda-2.17.1`
+- `opentelemetry-collector` ↔ `opentelemetry-collector-latest`
+- `victoria-metrics-cluster` ↔ `victoria-metrics-cluster-latest`
+- `victoria-metrics-agent` ↔ `victoria-metrics-agent-latest`
+- `sonarqube` ↔ `sonarqube-old` (reverse — sonarqube is new)
+
+## Decision
+
+For chart upgrades that involve breaking changes, create a sibling directory `<chart>-<variant>` and migrate cluster-by-cluster by repointing the `Application.spec.source.path` in the sister repo. Both directories remain live for the duration of the migration.
+
+Do not "consolidate" siblings as a maintenance PR — the split is intentional.
+
+## Rationale
+
+1. **Per-cluster cutover with rollback.** Each cluster's Argo `Application` flips one path; if the flip fails, that cluster's revert is a one-line PR in the sister repo. Other clusters are untouched.
+
+2. **No values-shape coupling.** When the new chart uses different values keys, the new chart's overrides can be authored at leisure and tested before any cluster cuts over. The old chart keeps rendering the old shape against the old overrides.
+
+3. **Immutable-field changes get a clean exit.** A bump in place that changes `Deployment.spec.selector` fails to apply (immutable). The sibling pattern lets you delete-and-recreate the workload as a one-time per-cluster event during cutover, rather than a fleet-wide failure mode.
+
+4. **Supports staged rollouts.** Some clusters cut over in week 1, others in week 4. The sister repo can hold both states simultaneously without forcing a full-fleet flip.
+
+5. **Tooling unchanged.** Argo CD, `helm template`, pre-commit hooks all see two parallel directories; no special handling.
+
+## Consequences
+
+### Accepted
+
+- **Repo bigger** during migration windows. A migration in flight has both `<chart>` and `<chart>-<variant>` live.
+- **Two charts to maintain** during the window. A CVE patch landing on upstream during the migration may need to be applied to both.
+- **Sister repo carries the routing decision.** This repo doesn't know which clusters have cut over; that info lives in `devops-infra-argo-config`.
+- **Naming inconsistency.** `-green`, `-vX.Y.Z`, `-latest`, `-old` aren't unified — different migrations chose different conventions. New migrations should pick the most descriptive (`-vX.Y.Z` if the target version is known; `-green` if the migration is a blue-green flip).
+
+### Mitigated
+
+- **Procedure** ([blue-green-chart-migration](../../docs/platform/procedures/blue-green-chart-migration.md)) names the steps explicitly: introduce sibling, render-and-diff, per-cluster cutover, retire old.
+- **Sanctity rule** ([R8](../../docs/global/SANCTITY_RULES.md)) prevents accidental deletion before all clusters have cut over.
+
+### Open
+
+- **Naming convention.** Should the team standardise on `<chart>-vX.Y.Z` for all future migrations? Today the choice is ad-hoc.
+- **CI/automation** to flag long-running migrations (siblings live > N weeks). Today migrations stall sometimes; nothing alerts.
+- **Per-cluster cutover tracking.** Today, knowing "which clusters still point at the old chart" requires `grep` against the sister repo. A small dashboard would help.
+
+## Alternatives considered
+
+| Alternative | Why not |
+|-------------|---------|
+| **In-place bump.** Just change `dependencies[].version` and merge. | Works for compatible bumps; for breaking bumps, fails on the first immutable-field mismatch and may leave the cluster broken. |
+| **Branch-as-environment** (e.g. an `int` branch with the new chart). | Doesn't help — Argo CD reads `main`. The sibling pattern is more flexible; per-cluster paths beat branches for this. |
+| **Helm `--atomic` upgrades.** Argo can pass `--atomic` to roll back failed upgrades. | Doesn't address breaking values-shape changes that succeed-but-render-wrong. |
+| **One big PR that bumps the chart and updates every override.** | Untestable; impossible to roll back one cluster. |
+
+## References
+
+- Procedure: [blue-green-chart-migration](../../docs/platform/procedures/blue-green-chart-migration.md).
+- Sanctity rule: [R8](../../docs/global/SANCTITY_RULES.md).
+- Live siblings (today): [docs/architecture.md §Cross-cutting concerns](../../docs/architecture.md).