added repo

2026-08-26 03:39:42 +05:30
parent 45c25a95af
commit b8575bb8b9
6889 changed files with 1217125 additions and 0 deletions
@@ -0,0 +1,214 @@
+# Wiki Entity — DevOps Infra Helm Charts
+
+> Architectural reference page for `devops-infra-helm-charts`. Operational depth lives in the procedures and runbooks under `docs/platform/`. This page is what you cite from other wiki pages when you mean "the infra Helm values repo."
+
+---
+
+## What it is
+
+The single GitOps source-of-truth for **what infrastructure tooling runs on Meesho's GKE fleet, where, and with what values**. Sister repo `github.com/Meesho/devops-infra-argo-config` is the routing layer — it holds the Argo CD `Application` / `ApplicationSet` manifests that point at paths in this repo.
+
+A merge to `main` is a deploy event: Argo CD on each cluster reconciles from `main`. **Most infra Applications use manual sync** ([ADR-A5](../analyses/ADR-A5-manual-sync-default-for-infra.md)), so a merge updates the Application resource but a human Sync click on the cluster's Argo CD UI deploys the workload.
+
+There is no application code, no build, no tests — only declarative YAML (Helm charts, values overrides, Kubernetes manifests) and two git-hook shell scripts.
+
+---
+
+## What it controls
+
+| Surface | Count |
+|---------|-------|
+| Cached / forked upstream Helm charts in `helm-templates/` | **74** |
+| Cluster directories under `helm-overrides/` | 30+ (16 BU prod + 5 GCP twins + dataplane `db-*` + int + dev + Aurva) |
+| Cluster-wide singletons under `manifests/` | StorageClasses (4), per-cluster PriorityClasses, Jenkins/JFrog filestore PV/PVCs |
+| Active pre-commit hooks | 1 (TruffleHog) |
+| No-op pre-commit hooks (gated paths absent) | 2 (CAC, Yaak) |
+| Post-commit hooks | 1 (Cursor metric collector — non-blocking) |
+
+---
+
+## Architecture
+
+### Two-repo GitOps split
+
+```text
+┌─────────────────────────────────┐         ┌──────────────────────────────────┐
+│  devops-infra-helm-charts       │         │  devops-infra-argo-config        │
+│  (this repo — values + charts)  │  ◄───── │  (sister repo — routing)         │
+│                                 │   path: │                                  │
+│  helm-templates/<chart>/        │         │  Application / ApplicationSet    │
+│  helm-overrides/<cluster>/<app>/│         │     spec.source.path:            │
+│  manifests/<singleton>/         │         │       helm-overrides/<...>       │
+└─────────────────────────────────┘         └──────────────────────────────────┘
+                                                       │
+                                                       ▼
+                                            ┌──────────────────────────────┐
+                                            │  Per-cluster Argo CD         │
+                                            │  (one per workload cluster)  │
+                                            │  reconciles main → cluster   │
+                                            └──────────────────────────────┘
+```
+
+### Deploy lifecycle
+
+```text
+edit helm-overrides/<cluster>/<app>/custom-values.yaml
+         │
+         ▼
+git commit  ──► pre-commit hook (TruffleHog secret scan)
+         │
+         ▼
+git push    ──► PR → review → merge to main
+         │
+         ▼
+Argo CD on each cluster reconciles main + sister-repo main
+         │
+         ▼
+Application sync: helm template <chart> -f <override>  →  apply (manual Sync click for most)
+         │
+         ▼
+post-commit hook ships Cursor AI metrics (background, non-blocking)
+```
+
+---
+
+## Cluster fleet
+
+All in `asia-southeast1` (zone-a or zone-c), fleet `meesho-admin-prd-0622`. See [docs/architecture.md](../../docs/architecture.md) for the full inventory.
+
+### BU prod clusters (`k8s-<bu>-prd-ase1[c]`)
+
+| Cluster type | Examples |
+|--------------|----------|
+| Standard GKE prod | `k8s-supply-prd-ase1`, `k8s-demand-prd-ase1`, `k8s-dataengg-prd-ase1`, `k8s-datascience-prd-ase1`, `k8s-farmiso-prd-ase1`, `k8s-ml-platform-prd-ase1`, `k8s-admin-prd-ase1`, `k8s-sec-admin-ase1`, `k8s-devops-admin-ase1` |
+| GKE Autopilot | `k8s-central-prd-ase1`, `k8s-dsgpu-prd-ase1`, `k8s-shared-int-ase1` |
+| Specialty | `k8s-central-mqkafka-prd-ase1`, `k8s-dengspark-prd-ase1`, `k8s-dengspark-di-prd-ase1`, `k8s-dengspark-notebook-prd-ase1`, `k8s-dscispark-prd-ase1` |
+| GCP zone-c twins | `k8s-supply-prd-ase1c`, `k8s-demand-prd-ase1c`, `k8s-dataengg-prd-ase1c`, `k8s-datascience-prd-ase1c`, `k8s-central-prd-ase1c` |
+
+### Other clusters
+
+| Pattern | Use |
+|---------|-----|
+| `k8s-shared-int-ase1` | Shared int (pre-prod) — only non-prod BU cluster |
+| `k8s-aurva-prd-ase1` | Aurva integration (minimal override set) |
+| `k8s-supply-dev-ase1` | Dev/sandbox supply |
+| `db-<numeric-id>-...` | Auto-named dataplane clusters (minimal: `kube-state-metrics` + `victoria-metrics-agent`) |
+
+---
+
+## Chart inventory
+
+Categorised view; full list in [docs/architecture.md §Helm chart inventory](../../docs/architecture.md).
+
+| Category | Charts |
+|----------|--------|
+| Argo / GitOps | `argo-cd`, `argo-cd-green` |
+| Ingress / edge | `contour`, `contour-v1.33.3`, `contour-ca-issuer`, `contour-cert-checker`, `ingress-nginx`, `cert-manager`, `external-dns`, `external-secrets` |
+| Observability — metrics | `prometheus-node-exporter`, `prometheus-stackdriver-exporter`, `kube-state-metrics`, `kube-events`, `victoria-metrics-{single,cluster,cluster-latest,agent,agent-latest,alert,alert-stateful,alerts-config,auth,mcp}`, `vm-alert-config`, `mimir-distributed`, `pmm`, `telegraf-operator` |
+| Observability — logs/traces/profiles | `fluentd`, `loki-distributed`, `tempo-distributed`, `pyroscope`, `alloy`, `opentelemetry-collector`, `opentelemetry-collector-latest`, `opentelemetry-operator`, `elastalert2`, `coroot-node-agent`, `deepfence-console`, `deepfence-router` |
+| UI / dashboards | `grafana`, `grafana-edge`, `grafana-mcp`, `kubernetes-dashboard`, `superset`, `uptime-kuma` |
+| Workflow / CI/CD | `jenkins`, `jfrog`, `sonarqube`, `sonarqube-old`, `flagger`, `keda`, `keda-2.17.1`, `kyverno`, `loadtester`, `temporal`, `dind`, `canary-bot-gcp`, `paused-container` |
+| Networking / DNS | `coredns`, `kube-dns`, `bifrost`, `conntrack-adjuster`, `node-thp-config` |
+| Data / search / DB | `clickhouse`, `etcd`, `vault`, `elasticsearch-mcp`, `eck-operator`, `athens-proxy` |
+| AI / 3rd-party | `aurva-dataplane`, `deepgram-onprem`, `rancher` |
+
+Versioned siblings (blue-green migration targets) are intentional, not duplicates — see [ADR-A2](../analyses/ADR-A2-blue-green-sibling-pattern.md).
+
+---
+
+## Upstreams (what this repo *needs*)
+
+| Upstream | Why we need it |
+|----------|----------------|
+| Sister repo `devops-infra-argo-config` | The routing layer. Without an `Application` / `ApplicationSet` over there, paths here are inert. |
+| Per-cluster Argo CD instances | Reconcile main into each cluster. Bootstrapping lives outside this repo. |
+| GCP Artifact Registry mirror (`asia-southeast1-docker.pkg.dev/meesho-devops-admin-0622/admin/sre/`) | All production image tags resolve here. |
+| External Secrets Operator (per-cluster `external-secrets` app) | Materialises GCP Secret Manager / Vault secrets into K8s `Secret`s referenced by `existingSecret:` keys. |
+| TruffleHog webhook (`observe.meeshogcp.in/api/webhook`) | Pre-commit secret-scan telemetry. |
+| Cursor metric API (`cursor-server.meeshogcp.in/api/v1/...`) | Post-commit (non-blocking) Cursor AI usage metrics. |
+| `cicd-scripts` repo | Source of pre/post-commit hook script logic. |
+| `registry-bootstrap` automation | Owns `repository.yaml`. |
+
+## Downstreams (what depends on this repo)
+
+| Downstream | Failure mode if this repo is broken |
+|------------|--------------------------------------|
+| Per-cluster Argo CD | If a chart's `Chart.lock` is missing or the values don't render, that cluster's Argo Application reports sync failure. |
+| Every infra workload (Contour, VictoriaMetrics, Argo CD, cert-manager, …) | A bad values change can take down ingress, observability, secret materialisation. |
+| Pulse alerting / on-call routing | Reads label metadata on alerts; chart-bump-induced label drift can break routing. |
+| `external-dns` / Cloud DNS | Sidecar `Service` resources here drive DNS records. |
+
+---
+
+## Key conventions (load-bearing)
+
+| Convention | What enforces it |
+|------------|------------------|
+| `helm-overrides/<cluster>/<app>/custom-values.yaml` is the values filename | Sister-repo `Application.spec.source.helm.valueFiles` references this name |
+| Cluster directory name == cluster name in Argo CD | Convention only — silent bind failure if mismatched |
+| `image.registry: asia-southeast1-docker.pkg.dev` | [SANCTITY_RULES R11](../../docs/global/SANCTITY_RULES.md) |
+| `nodeSelector` / `tolerations` per-cluster bespoke | [SANCTITY_RULES R5](../../docs/global/SANCTITY_RULES.md), [contour-nodeselector-tolerations-summary.md](../../contour-nodeselector-tolerations-summary.md) |
+| `fullnameOverride` is stable forever | [SANCTITY_RULES R9](../../docs/global/SANCTITY_RULES.md) |
+| Versioned chart siblings stay live during migrations | [SANCTITY_RULES R8](../../docs/global/SANCTITY_RULES.md), [ADR-A2](../analyses/ADR-A2-blue-green-sibling-pattern.md) |
+| `helm-templates/<chart>/templates/` is upstream — don't edit casually | [SANCTITY_RULES R7](../../docs/global/SANCTITY_RULES.md), [ADR-A1](../analyses/ADR-A1-cache-vs-upstream-charts.md) |
+
+---
+
+## Operational procedures
+
+| Task | Procedure |
+|------|-----------|
+| Onboard an app to a cluster | [onboard-app-to-cluster](../../docs/platform/procedures/onboard-app-to-cluster.md) |
+| Onboard a brand-new cluster's overrides | [onboard-new-cluster](../../docs/platform/procedures/onboard-new-cluster.md) |
+| Bump a chart's pinned version | [update-chart-version](../../docs/platform/procedures/update-chart-version.md) |
+| Intentionally fork a chart | [fork-upstream-chart](../../docs/platform/procedures/fork-upstream-chart.md) |
+| Migrate a chart blue-green | [blue-green-chart-migration](../../docs/platform/procedures/blue-green-chart-migration.md) |
+| Deboard a retired app | [deboard-app](../../docs/platform/procedures/deboard-app.md) |
+
+## Runbooks
+
+| Symptom | Runbook |
+|---------|---------|
+| Argo CD app errored / OutOfSync | [argocd-sync-failure](../../docs/platform/runbooks/argocd-sync-failure.md) |
+| Ingress (Contour) is down | [ingress-down](../../docs/platform/runbooks/ingress-down.md) |
+| Pods Pending / wrong-node scheduling | [pod-pending-scheduling](../../docs/platform/runbooks/pod-pending-scheduling.md) |
+
+---
+
+## Architecture decisions
+
+| ADR | Decision |
+|-----|----------|
+| [ADR-A1](../analyses/ADR-A1-cache-vs-upstream-charts.md) | Why we cache upstream charts in `helm-templates/` instead of pulling on the fly |
+| [ADR-A2](../analyses/ADR-A2-blue-green-sibling-pattern.md) | Why we use versioned chart siblings for migrations |
+| [ADR-A3](../analyses/ADR-A3-per-cluster-scheduling.md) | Why per-cluster `nodeSelector` / `tolerations` / `computeClass` is bespoke |
+| [ADR-A4](../analyses/ADR-A4-raw-manifest-sidecars-in-helm-overrides.md) | Why `helm-overrides/<cluster>/<app>/` mixes Helm values with raw sidecar manifests |
+| [ADR-A5](../analyses/ADR-A5-manual-sync-default-for-infra.md) | Why most infra Applications are manual-sync (no `automated`) |
+
+---
+
+## Open knowledge gaps
+
+1. **The bootstrap source for per-cluster Argo CD installs is outside this repo.** Likely Terraform-managed cluster config or a separate "argo-bootstrap" repo. Locating and documenting it is a follow-up.
+2. **Some `helm-templates/<chart>/` charts have no consumers** (`grep -rl '<chart>' helm-overrides` returns empty). Inventory and cleanup is a separate exercise.
+3. **The split between `helm-overrides/<cluster>/<app>/<x>.yaml` raw sidecars and pure-Helm values directories isn't formally documented per app.** The matching sister-repo `Application` is authoritative; this repo doesn't always make the shape obvious from a glance.
+4. **Cross-cluster project-replica pattern** (e.g. mrouter-equivalent for infra). Doesn't exist here in the same way it does in `devops-argo-config`, but the dataplane (`db-*`) clusters do share a structure that could be templated.
+
+---
+
+## Ownership
+
+- **Primary**: `siddharth.pal@meesho.com` (per `repository.yaml`)
+- **Secondary**: `samarth.nag@meesho.com`
+- **Team**: DevOps / Platform
+
+---
+
+## Related wiki entities
+
+- `[[DevOps Infra Argo Config]]` — sister repo (Argo `Application` / `ApplicationSet` routing).
+- `[[DevOps ArgoCD Config]]` — application-side GitOps (services, not infra).
+- `[[DevOps Helm Charts]]` — the *application*-side chart repo (services), distinct from this one.
+- `[[CI-CD Security Tools]]` / `[[Git Hooks Security Pipeline]]` — pre-commit hook source.
+- `[[Per-Cluster Deployment Contract]]` — the cross-repo contract for bringing up an app on a cluster.
+- `[[GitOps with ArgoCD]]` — overarching GitOps concept page.