5.4 KiB
ADR-A1 — Cache upstream charts in helm-templates/ (vs. pull on the fly)
Status: Accepted (de facto — current state of the repo). Repo:
devops-infra-helm-charts. Related: docs/architecture.md, SANCTITY_RULES R7, update-chart-version.md, fork-upstream-chart.md.
Context
Argo CD can render a Helm release in two ways:
- Pull on the fly —
Application.spec.source.repoURLpoints at an upstream Helm registry;chart:names the chart. Argo CD pulls the chart at sync time. - Cache locally — the chart lives in a git repo (this one), and
Application.spec.source.pathpoints at it. Argo CD reads the chart files from git directly.
This repo has chosen caching. Every chart consumed by Meesho's GKE infra fleet has a directory in helm-templates/<chart>/ — usually a thin wrapper whose Chart.yaml declares the upstream chart as a dependency, with the resolved subchart materialised in Chart.lock and charts/<sub>-<ver>.tgz.
Decision
Cache upstream charts in helm-templates/ as thin wrapper directories with pinned dependencies[].version and a committed Chart.lock. Do not configure Argo CD to pull charts directly from upstream registries.
Rationale
-
Repeatable renders. A chart-version bump in this repo is a git diff; a chart-version "bump" via upstream pull is whatever the registry returns at sync time. Reproducibility on rollback requires the chart bytes to be in git.
-
Air-gapped reviewability. A reviewer can read the
Chart.yaml, theChart.lock, and the subchart.tgzto know exactly what will render. With on-the-fly pulls, the reviewer trusts the upstream registry hasn't moved a tag. -
Network independence at sync time. Argo CD's reconcile loop doesn't need outbound network to upstream registries. If GitHub is reachable, sync works; if it isn't, nothing's deploying anyway.
-
Supply-chain control. Pinning
dependencies[].versionplus committingChart.lockmeans the digest of each subchart.tgzis recorded. A registry compromise that re-publishes a tag with new content is detected by the lockfile mismatch. -
Forking is local. When upstream lacks a feature or has a bug, fork-upstream-chart is a local edit. No "wait for upstream merge"; the patch lives in our repo until upstream catches up.
-
Render-tooling unchanged. Reviewer-side
helm template helm-templates/<chart> -f overrides.yamlworks locally without any registry config. Pre-merge validation is justhelm templateagainst the directory.
Consequences
Accepted
- Repo bigger. 74 chart directories totalling tens of MBs of subchart
.tgzfiles. - Manual update cadence. A new upstream release isn't picked up automatically. Someone has to bump
dependencies[].versionand runhelm dependency update. (update-chart-version) - Forking risk. Editing
helm-templates/<chart>/templates/casually creates an accidental fork that gets clobbered nexthelm dependency update. (SANCTITY_RULES R7) - Lock-step requirement. A
Chart.yamlbump without a refreshedChart.lockis incomplete — Argo CD reads the lockfile, so the version change silently no-ops.
Mitigated
- Update-chart-version procedure documents the lock-step requirement explicitly.
- Versioned siblings (ADR-A2) handle major-version bumps without losing the old chart.
- Pre-merge
helm templatecatches values incompatibilities before merge.
Open
- Stale charts. Some directories in
helm-templates/have no current consumer (grep -rl '<chart>' helm-overridesempty). A periodic clean-up exercise has not been formalised. - Subchart
.tgzsize pollution in git history. Everyhelm dependency updatewrites a new.tgztocharts/. Over years, the repo's history grows accordingly. Whether to switch to a Helm-OCI-pull model is an open question. - Chart-bump notification. Nothing currently alerts the team when an upstream advisory affects a chart we have pinned at an old version. Manual diligence today.
Alternatives considered
| Alternative | Why not |
|---|---|
| Pull-on-the-fly from upstream registries. | Reproducibility, network dependency, supply-chain risk all worse. |
Fully vendor every chart's templates/ (no dependencies[], no subchart .tgz). |
Massive diff churn on every upstream release; worse fork hygiene. |
Use Helm OCI registries as a middle ground (pull .tgz from a private registry instead of git). |
Plausible Phase-2 work. Adds an extra service to maintain; doesn't solve forking. Not done today. |
Use kustomize instead of Helm. |
Most upstream charts are Helm; rewriting every chart's templates as Kustomize patches would be enormous. |
References
- Argo CD Helm chart source docs: https://argo-cd.readthedocs.io/en/stable/user-guide/helm/
- This repo's chart directory layout: docs/architecture.md §Module boundaries.
- Bump procedure: update-chart-version.
- Fork procedure: fork-upstream-chart.