devops-lib-gcp/docs/adr/0013-multi-zone-deployables-gated-out-of-direct-jenkins-argocd.md

ADR-0013: Multi-zone deployables gated out of direct Jenkins ArgoCD

Status: Accepted Category: RELIABILITY Date decided: Alongside the multi-zone initiative Date documented: 2026-05-13

Context

Meesho ran a multi-zone initiative to deploy production services across two GCP zones in parallel and split traffic between them. The initiative immediately exposed that not every workload type is safe to multiply across zones: schedulers, cron jobs, and consumers cause correctness issues when more than one zone runs them at the same time; cache and database service types are explicitly blocked by Turbo-Turtle's ValidateDeploymentConfigActivity; some workloads (e.g. Deepgram, with volume affinity that pins it to a single zone) can't go multi-zone at all. Meanwhile, deployArgoCD.groovy in this shared library only knows how to call argocd sync against a single application — it has no concept of split-by-service-type or per-zone sequencing.

Decision

deployArgoCD.groovy fetches the multizone-enabled-repos whitelist from Meesho/whitelists at the start of every build. If the current deployable is on the list, the pipeline aborts with "Multi-zone enabled for this deployable. Please use Ringmaster for deployment." rather than attempting a sync. Multi-zone-enabled services are required to go through Ringmaster, which owns the split-deploy logic, per-service-type validation, and zone-affinity awareness.

Alternatives Considered

No alternatives were explicitly evaluated by the team during this interview. The gate was introduced alongside the multi-zone initiative itself, not retrofitted after an incident.

Consequences

Positive:

• Direct Jenkins ArgoCD is structurally incapable of getting multi-zone wrong because it never gets the chance — the gate fails closed.
• The split-deploy logic and the service-type-validation logic live in exactly one place (Ringmaster); we do not maintain two copies.
• A team can flip a service to multi-zone by adding it to multizone-enabled-repos.yaml without modifying devops-lib — the policy change takes effect on the next build.

Negative:

• Engineers who routinely use cicd- Jenkins jobs see an unfamiliar refusal once their service is added to the whitelist; the error message is the only signal pointing them at Ringmaster.
• The library has no way to attempt a partial deploy or to surface what would-have-been-deployed; the gate is binary.

Neutral:

• The decision lives across two repos — this code in devops-lib, the source-of-truth list in Meesho/whitelists. See ADR-0003 for the broader policy-in-a-separate-repo pattern.

Constraints

• Multi-zone correctness requires service-type awareness that the Jenkins ArgoCD path does not have.
• The whitelist is fetched fresh per build (ADR-0004), so policy changes in Meesho/whitelists take effect on the next build with no library release.

Current Assessment

• Still appropriate — fail-closed gating is the right posture for a capability the library cannot safely implement.

Related Decisions

• ADR-0003: Policy Exceptions Controlled by a Separate Meesho/whitelists Repo — explains the whitelist-source-of-truth pattern this gate uses.
• ADR-0004: Whitelist Cloned Fresh on Every Build — No Caching — why this list takes effect immediately.
• ADR-0006: Ringmaster as Mandatory Build Trigger Gate — Ringmaster's parallel role on the trigger side.
• ADR-0011: Build-user identity routes post-build callbacks — the callback flow that Ringmaster uses on success.

Notes

• Key files: src/com/meesho/stages/deployArgoCD.groovy
• Whitelist source: https://github.com/Meesho/whitelists/blob/main/multizone-enabled-repos.yaml
• Discovery id: RELIABILITY-1