devops-lib-gcp/docs/adr/0007-gitops-via-argocd-4-step-sync-sequence.md

ADR-0007: GitOps Deployments via Strict 4-Step ArgoCD Sync Sequence

Status: Accepted Category: INFRA Date decided: Early on Date documented: 2026-05-12

Context

Meesho migrated GCP service deployments from direct kubectl apply / Helm install to GitOps via ArgoCD. The key requirement was continuous reconciliation — the cluster state should always reflect what's in Git, and any manual kubectl changes should be automatically reverted. The deployment pipeline needed to update two separate Git repositories (argo-config for ArgoCD Application manifests, helm-repo for Helm chart values) and trigger ArgoCD to sync, without leaving the cluster in an inconsistent intermediate state.

Decision

All GCP service deployments go through a strict 4-step sequence in deployArgoCD.groovy:

1. update_argo_repo — Push the updated ArgoCD Application manifest to devops-argo-config
2. refresh_app_of_apps — Trigger ArgoCD to sync the app-of-apps, creating any new Application objects
3. update_helm_repo — Push the new Helm chart (with the new image tag) to the Helm repo
4. refresh_and_sync — Trigger ArgoCD to sync the specific application

The order is non-interchangeable. Steps 2 and 4 cannot be swapped.

Alternatives Considered

• Direct kubectl apply: Rejected — any manual change to the cluster would persist indefinitely; no drift detection or automatic reconciliation.
• Helm install from Jenkins directly: Rejected — Helm state would live only in the cluster's release history, not in Git; no GitOps audit trail or rollback via git revert.
• Skipping step 2 (app-of-apps refresh): Not a conscious alternative — the hard requirement emerged from debugging. For new services, step 2 must run before step 3 because the ArgoCD Application object doesn't exist yet; if step 3 (Helm push) runs first, step 4 (sync) targets a non-existent application and fails silently.

Consequences

Positive:

• Every deployment is a Git commit — rollback is a git revert, and the cluster state is always reproducible from Git history.
• ArgoCD continuously reconciles cluster state — manual kubectl apply changes are automatically reverted, preventing configuration drift.
• Deployment failures are localised: the 4-step sequence makes it clear which step failed (argo-config push? app-of-apps refresh? Helm push? sync?) for faster debugging.

Negative:

• The 4-step sequence is opaque without documentation — engineers debugging a deploy failure must know which step corresponds to which operation.
• Steps 2 and 4 being non-interchangeable is tribal knowledge; swapping them for new services causes a silent sync failure that is hard to diagnose.
• ArgoCD dependency: if ArgoCD is degraded, all deployments are blocked regardless of build success.

Neutral:

• The sequence touches two separate Git repositories (devops-argo-config and helm-repo) in a single pipeline run — partial failures leave one repo updated and the other stale.

Constraints

ArgoCD was the organisational standard for GCP deployments. The 4-step sequence was designed to handle both the case of existing services (steps 1, 3, 4 are the hot path) and new services being onboarded for the first time (step 2 is required to create the Application object before step 4 can sync it).

Current Assessment

Still appropriate — no changes needed.

Notes

• Key file: src/com/meesho/stages/deployArgoCD.groovy (480+ lines) — all 4 steps are defined here
• The step ordering constraint is documented in CLAUDE.md and docs/tribal-knowledge.md (TK#10) as load-bearing tribal knowledge
• See also: docs/wiki/pages/deploy/argocd-sync.md for a detailed walkthrough of each step