added repo
This commit is contained in:
Your Name
@@ -0,0 +1,195 @@
|
||||
> Per AI Blitz Plan §platform.schemas. Layer: 1. Repo: devops-infra-helm-charts.
|
||||
|
||||
# Schema — Incubator-tool values + sidecar contract
|
||||
|
||||
> **Scope:** the values-side contract for incubator (newly-vendored) infrastructure tools that live under `helm-overrides/<cluster>/<incubator-tool>/`.
|
||||
>
|
||||
> **Out of scope (explicit):** the Argo CD `Application` / `ApplicationSet` manifest that registers the incubator tool with a cluster's Argo. Those manifests live in the **sister repo** `github.com/Meesho/devops-infra-argo-config`, NOT here. This file documents only what `devops-infra-helm-charts` is responsible for: the values + raw sidecar manifests Argo CD reads from this repo.
|
||||
|
||||
This schema complements [custom-values-schema.md](custom-values-schema.md) (general values shape) and [raw-manifest-sidecar-schema.md](raw-manifest-sidecar-schema.md) (sidecar manifest shapes). Read both first.
|
||||
|
||||
---
|
||||
|
||||
## What "incubator" means here
|
||||
|
||||
A tool is **incubator** while it is being trialled on one or two clusters before fleet-wide rollout. In this repo that maps to:
|
||||
|
||||
- A new chart directory under `helm-templates/<incubator-tool>/` (often a thin wrapper `Chart.yaml` with an upstream dep).
|
||||
- An override under `helm-overrides/<cluster>/<incubator-tool>/` for the trial cluster(s) only — typically `k8s-shared-int-ase1` (integration / pre-prod) first, sometimes one BU prod cluster as a canary.
|
||||
- Possibly raw sidecar manifests alongside the values file.
|
||||
|
||||
Once the tool graduates from incubator status, the override pattern is identical to other infra tools — the "incubator" label is operational, not structural. This schema codifies the conventions that keep early-stage adoption sane.
|
||||
|
||||
---
|
||||
|
||||
## Directory layout
|
||||
|
||||
```
|
||||
helm-overrides/<cluster>/<incubator-tool>/
|
||||
├── custom-values.yaml # Helm values (required if chart is Helm-based)
|
||||
├── computeclass/ # Optional — GKE Autopilot ComputeClass
|
||||
│ └── <name>-cc.yaml
|
||||
├── external-dns-services/ # Optional — Service for external-dns annotation
|
||||
│ └── <fqdn>.yaml
|
||||
├── external-secrets/ # Optional — ExternalSecret resources
|
||||
│ └── <name>.yaml
|
||||
└── <incubator-specific>.yaml # Tool-specific raw manifest (CRD, ConfigMap)
|
||||
```
|
||||
|
||||
The Argo `Application` for this directory (sister repo) determines whether files are Helm-rendered, raw-applied, or layered (see [raw-manifest-sidecar-schema.md §Layered-with-Helm vs standalone](raw-manifest-sidecar-schema.md)).
|
||||
|
||||
---
|
||||
|
||||
## `custom-values.yaml` keys an incubator chart should expose
|
||||
|
||||
A chart promoted to incubator status MUST surface (i.e. let the override file set without forking templates) at minimum:
|
||||
|
||||
| Key | Why required |
|
||||
|-----|--------------|
|
||||
| `image.registry`, `image.repository`, `image.tag` | Production overrides pin to `asia-southeast1-docker.pkg.dev/meesho-devops-admin-0622/admin/sre/<image>` (Meesho Artifact Registry mirror, not Docker Hub). |
|
||||
| `image.pullPolicy` | Default `IfNotPresent`. |
|
||||
| `resources.requests.cpu`, `.memory` | No defaults assumed; Autopilot scheduling requires explicit requests. |
|
||||
| `resources.limits.cpu`, `.memory` | Same. |
|
||||
| `nodeSelector` | Per-cluster scheduling (see [custom-values-schema.md](custom-values-schema.md)). |
|
||||
| `tolerations` | Per-cluster taints. |
|
||||
| `affinity` (optional) | Pod anti-affinity for replicated tools. |
|
||||
| `serviceAccount.create`, `.name`, `.annotations` | Workload Identity — `iam.gke.io/gcp-service-account` annotation goes here. |
|
||||
| `replicaCount` (or `autoscaling.{enabled,minReplicas,maxReplicas}`) | Sizing. |
|
||||
| `persistence.enabled`, `.storageClass`, `.size` | If stateful. `storageClass` MUST exist in `manifests/storageclass/`. See [storageclass-priorityclass-schema.md](storageclass-priorityclass-schema.md). |
|
||||
| `priorityClassName` | If the tool needs preemption priority — must reference a class in `manifests/priorityclass/<cluster>/`. |
|
||||
| `podLabels`, `podAnnotations` | For prometheus.io scrape annotations and team-attribution labels. |
|
||||
| `extraEnv` (or `env`) | For environment-specific knobs the chart's templates don't already accept. |
|
||||
|
||||
If the upstream chart doesn't expose these — that's a chart-bug. **Do not** fork the chart in `helm-templates/` to add them ([NEVER-DO list](../../../CLAUDE.md)). Either upstream-PR the chart or wrap with a small Meesho-owned chart that re-exposes the keys.
|
||||
|
||||
---
|
||||
|
||||
## Values-file conventions for incubator tools
|
||||
|
||||
1. **Pin every image tag.** Never `latest`, never an unpinned SHA. This is the same rule as production overrides; incubator status does not relax it.
|
||||
2. **Set explicit `nodeSelector` and `tolerations`** authored from scratch using sibling apps on the same cluster. Cross-cluster copying is forbidden — see [custom-values-schema.md](custom-values-schema.md).
|
||||
3. **Use `fullnameOverride` deliberately or not at all.** Once set, do not change ([NEVER-DO list](../../../CLAUDE.md)). Incubator graduations to other clusters should reuse the same `fullnameOverride` string for portability.
|
||||
4. **Don't enable `autoscaling`** in the first incubator deploy. Pin `replicaCount: 1` (or 2 for HA-mandatory) until you have a load profile.
|
||||
5. **Don't expose the tool externally** in the incubator phase. No `external-dns-services/` until it's promoted to a cluster's stable inventory.
|
||||
6. **Annotate the values file** with a top-of-file YAML comment: `# Incubator: cluster=<...>, owner=<...>, graduation-target=<date>`. Comment is plain text; not parsed; serves as reviewer signal.
|
||||
7. **Confine secrets to `ExternalSecret`** under `external-secrets/`. No inline `existingSecret:` referencing a hand-applied Secret.
|
||||
|
||||
---
|
||||
|
||||
## Raw sidecar manifests in incubator directories
|
||||
|
||||
Concrete shapes already documented elsewhere; this section calls out which ones routinely appear with incubators.
|
||||
|
||||
### `computeclass/*-cc.yaml` (GKE Autopilot only)
|
||||
|
||||
Required when the override's `nodeSelector` uses `cloud.google.com/compute-class: <name>` and that class doesn't already exist on the cluster. The `metadata.name` MUST equal the value referenced. See [raw-manifest-sidecar-schema.md §ComputeClass](raw-manifest-sidecar-schema.md).
|
||||
|
||||
Example incubator pattern:
|
||||
|
||||
```yaml
|
||||
apiVersion: autoscaling.gke.io/v1
|
||||
kind: ComputeClass
|
||||
metadata:
|
||||
name: <incubator-tool>-cc
|
||||
namespace: <ns>
|
||||
spec:
|
||||
priorities:
|
||||
- machineFamily: c4d
|
||||
minCores: 2
|
||||
minMemory: 8Gi
|
||||
nodePoolAutoCreation:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
Per-cluster only — never copy between clusters.
|
||||
|
||||
### `external-dns-services/*.yaml`
|
||||
|
||||
Skip in the first incubator deploy. Only add once the tool has a stable internal-only DNS need. See [raw-manifest-sidecar-schema.md §Service for external-dns binding](raw-manifest-sidecar-schema.md).
|
||||
|
||||
### `external-secrets/*.yaml`
|
||||
|
||||
Always present if the tool needs secrets. Use the existing per-cluster `SecretStore` / `ClusterSecretStore`; do not author new stores in an incubator PR. See [raw-manifest-sidecar-schema.md §ExternalSecret](raw-manifest-sidecar-schema.md).
|
||||
|
||||
### `elastic-cluster/argo-launch.yaml`-style operator-managed CRD launches
|
||||
|
||||
If the incubator tool is an operator (e.g. ECK, Pyroscope), a sidecar CRD instance often lives alongside the operator's chart values:
|
||||
|
||||
```
|
||||
helm-overrides/<cluster>/<incubator-operator>/
|
||||
├── custom-values.yaml # operator chart values
|
||||
└── <crd-instance>/
|
||||
└── argo-launch.yaml # the actual workload CRD instance
|
||||
```
|
||||
|
||||
Argo CD applies both in one Application (directory loader). The CRD instance must:
|
||||
|
||||
- Reference an `apiVersion` whose CRD the operator has already installed.
|
||||
- Pin its own image / version explicitly.
|
||||
- Avoid hand-rolling values that the operator templates would otherwise compute.
|
||||
|
||||
---
|
||||
|
||||
## Argo CD — sister repo contract (for cross-reference only)
|
||||
|
||||
The Application that points at this directory lives in `github.com/Meesho/devops-infra-argo-config`. For incubator tools the Application typically:
|
||||
|
||||
- Has `syncPolicy.automated` **disabled** (manual sync — incubator-grade safety; see [argocd.md](../../global/coding-guidelines/argocd.md)).
|
||||
- Is named `<incubator-tool>-<cluster>` to be unambiguous.
|
||||
- Lives in the cluster's namespace under `apps/<cluster>/`.
|
||||
|
||||
This file does NOT instruct on authoring the Application — that PR is in the sister repo. Cross-link the sister-repo PR in the values-side PR description.
|
||||
|
||||
---
|
||||
|
||||
## Validation
|
||||
|
||||
Same as any Helm override. Before opening a PR:
|
||||
|
||||
```bash
|
||||
yamllint helm-overrides/<cluster>/<incubator-tool>/custom-values.yaml
|
||||
|
||||
helm template <release> helm-templates/<incubator-tool> \
|
||||
-f helm-overrides/<cluster>/<incubator-tool>/custom-values.yaml > /tmp/render.yaml
|
||||
|
||||
# Validate any sidecar manifests against the cluster's CRDs
|
||||
kubectl --context=<ctx> --dry-run=server \
|
||||
-f helm-overrides/<cluster>/<incubator-tool>/<sidecar>.yaml apply
|
||||
```
|
||||
|
||||
For tool-specific CRDs that aren't in vanilla Kubernetes, prefer `--dry-run=server` (the cluster validates against the registered CRD) over `--dry-run=client` (only client-side schema, often misses required fields).
|
||||
|
||||
---
|
||||
|
||||
## Graduation: incubator → stable
|
||||
|
||||
When the tool has been stable for some period (usually 2–4 weeks) and is ready to fleet-roll:
|
||||
|
||||
1. Remove the top-of-file `# Incubator:` comment.
|
||||
2. Open new override directories in target clusters — author from scratch each time, do not copy.
|
||||
3. Open the matching sister-repo `Application` PRs (one per new cluster) or extend the `ApplicationSet`.
|
||||
4. If the chart was a thin wrapper, audit `Chart.yaml` `dependencies[].version` is current.
|
||||
5. Promote the chart's documentation in `helm-templates/<incubator-tool>/README.md` (if a fork was needed) or note in PR description that no fork was needed.
|
||||
|
||||
---
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
1. **Forking the chart in `helm-templates/<incubator-tool>/templates/`** to add a missing values key. Wrap or upstream-PR; don't fork. See NEVER-DO.
|
||||
2. **Copying the entire incubator directory across clusters** for graduation. Per-cluster scheduling is bespoke.
|
||||
3. **Enabling autoscaling on day one.** No load profile = thrashy autoscaler.
|
||||
4. **Promoting from `k8s-shared-int-ase1` straight to a critical BU cluster.** Insert a low-traffic prod canary first.
|
||||
5. **Inlining secrets** "just for the trial." TruffleHog will block; even if it didn't, the leak is real.
|
||||
6. **Setting `fullnameOverride`** without a graduation plan. The string travels — bad ones travel forever.
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- Schema: [custom-values-schema.md](custom-values-schema.md) — general values shape.
|
||||
- Schema: [raw-manifest-sidecar-schema.md](raw-manifest-sidecar-schema.md) — sidecar shapes.
|
||||
- Schema: [storageclass-priorityclass-schema.md](storageclass-priorityclass-schema.md) — cluster-singleton resources.
|
||||
- Procedure: [../procedures/onboard-app-to-cluster.md](../procedures/onboard-app-to-cluster.md) — the procedure used to land an incubator override.
|
||||
- Procedure: [../procedures/fork-upstream-chart.md](../procedures/fork-upstream-chart.md) — only when a fork is genuinely required.
|
||||
- Coding guideline: [../../global/coding-guidelines/helm-values.md](../../global/coding-guidelines/helm-values.md).
|
||||
- Coding guideline: [../../global/coding-guidelines/argocd.md](../../global/coding-guidelines/argocd.md) — sister-repo Application conventions.
|
||||
Reference in New Issue
Block a user