added repo

2026-08-26 03:39:42 +05:30
parent 45c25a95af
commit b8575bb8b9
6889 changed files with 1217125 additions and 0 deletions
@@ -0,0 +1,66 @@
+# ADR-A5 — Manual sync is the default for infra Applications
+
+> **Status:** Accepted (status quo — most infra Applications lack `automated`).
+> **Repo:** `devops-infra-helm-charts` (consumer of the decision; the `Application` shapes that enact it live in the sister repo).
+> **Related:** [SANCTITY_RULES R3](../../docs/global/SANCTITY_RULES.md) (analogue from the application-side repo), [argocd-sync-failure.md](../../docs/platform/runbooks/argocd-sync-failure.md), [deboard-app.md](../../docs/platform/procedures/deboard-app.md).
+
+---
+
+## Context
+
+Argo CD `Application` resources can have a `spec.syncPolicy.automated` block that auto-applies any diff between git and the cluster on every reconcile cycle. With it: a merge to `main` deploys immediately. Without it: a merge updates the Application's *desired state*, but the cluster doesn't change until a human (or external trigger) clicks **Sync** in the Argo CD UI (or runs `argocd app sync`).
+
+Most Applications routing to this repo (`devops-infra-helm-charts`) **do not have `automated`** set. A small minority of infra Applications — typically things that should self-heal aggressively (canary-bot, statsd-exporter, vmextractor) — do.
+
+## Decision
+
+For infra Applications routed by `github.com/Meesho/devops-infra-argo-config`, the default is **manual sync** — `spec.syncPolicy` contains only `syncOptions: [CreateNamespace=true]`, with no `automated` block. Adding `automated.{prune,selfHeal}: true` to a service-tier Application is a deliberate, headline-of-the-PR change.
+
+## Rationale
+
+1. **Production blast radius.** A merge here can cascade across many clusters. If a values change is wrong, the manual-sync default means an operator has a chance to spot it (in Argo CD's Diff view) before clicking through. Auto-sync would push the broken change to every cluster simultaneously on the next reconcile.
+
+2. **Per-cluster cutover.** A typical chart bump or values change rolls out cluster-by-cluster. The operator clicks Sync on cluster A, watches, then proceeds to cluster B. Auto-sync forces a fleet-wide flip with no soak window.
+
+3. **Out-of-band drift detection.** Manual sync makes drift visible — when someone `kubectl edit`-s a release on a cluster, Argo CD shows `OutOfSync` and surfaces the diff. With auto-sync, the drift is silently overwritten on the next reconcile, hiding the fact that someone made an out-of-band change.
+
+4. **Sync click is the agent's hard stop.** [AGENT_BOUNDARIES.md](../../docs/global/AGENT_BOUNDARIES.md) classifies "click Sync" as Layer 2 advisory — the agent recommends the command but never executes. The default of manual sync makes this enforceable: the agent literally cannot deploy without a human in the loop.
+
+5. **Safe by default; opt in for the loop closures.** Apps that genuinely should self-heal (canary-bot — purpose is to test traffic; statsd-exporter — purpose is fleet-wide telemetry) can be opted in via `automated.prune: true`. The opt-in is a deliberate decision, not a side-effect.
+
+## Consequences
+
+### Accepted
+
+- **Operational tax.** Every PR merge creates an `OutOfSync` Application that someone has to click through. With ~30 clusters × dozens of apps, this can pile up on busy days.
+- **Drift accumulation.** A PR that nobody clicks Sync on sits as `OutOfSync` indefinitely. Sometimes this is intentional (the PR was speculative); sometimes it's forgotten. Periodic audits ("which Applications have been `OutOfSync` for > 7 days?") aren't yet automated.
+- **Manual-sync bias** can mask incidents — an Application failing to sync (because of a values regression) may sit `OutOfSync` for a while before someone notices. Auto-sync would have surfaced it loudly via failed reconciles.
+
+### Mitigated
+
+- **The runbook** ([argocd-sync-failure.md](../../docs/platform/runbooks/argocd-sync-failure.md)) explicitly handles the "OutOfSync only, no error, sync hasn't run" branch as §4 — its own diagnostic path.
+- **Sanctity rule analogue** in the application-side `devops-argo-config` repo names this explicitly (R3); we inherit the principle.
+- **Skill** ([diagnose-scheduling.md](../../skills/infra/diagnose-scheduling.md)) outputs Layer 2 advisories ("recommend operator clicks Sync") rather than auto-Sync triggers.
+
+### Open
+
+- **Notification on long-`OutOfSync` Applications** — periodic alert / dashboard. Today operators just see this in the Argo UI.
+- **Should some infra apps move to auto-sync?** Specifically, sidecars whose blast radius is tiny (telemetry collectors, log agents). A periodic review hasn't been done.
+- **The opt-in list of currently auto-synced apps** isn't documented in this repo. Has to be inferred from the sister repo's `Application` files.
+- **Cluster-specific opt-ins** — auto-sync on dev clusters but manual on prod — would be a reasonable refinement but adds per-cluster `Application` divergence.
+
+## Alternatives considered
+
+| Alternative | Why not |
+|-------------|---------|
+| **Auto-sync everywhere by default.** | Loses the per-cluster operator gate; a bad merge cascades fleet-wide. |
+| **Auto-sync with `selfHeal: false` but `prune: true`.** | Still applies values changes immediately; doesn't help. |
+| **Manual sync but with auto-fallback after N hours.** | Argo CD doesn't offer this natively. Building it would require a controller. |
+| **Per-environment policy** (auto-sync on int, manual on prod). | Reasonable refinement; adds policy state to the sister repo. Could be future work. |
+
+## References
+
+- Sister-repo `Application` shapes: `github.com/Meesho/devops-infra-argo-config`.
+- Runbook §4: [argocd-sync-failure.md](../../docs/platform/runbooks/argocd-sync-failure.md).
+- Boundaries: [AGENT_BOUNDARIES.md](../../docs/global/AGENT_BOUNDARIES.md) (Layer 2 row "Sync `<release>` now in Argo CD").
+- Application-side analogue: `devops-argo-config`'s `SANCTITY_RULES.md R3`.