devops-infra-argo-config-gcp/docs/platform/procedures/add-new-cluster.md

# Procedure: Onboard a New Cluster

> Step-by-step guide for adding a new Kubernetes cluster to the ArgoCD infrastructure GitOps control plane.
>
> **Layer:** 1-T (Tool-Mediated). Two files must be created: one incubator file and one values file.
>
> **Blast radius:** New cluster only. Existing clusters are unaffected.
>
> **Prerequisite:** The GKE cluster must already be provisioned via `terraform-google-modules` and registered as an ArgoCD cluster destination.

---

## Inputs

| Input | Example | Where it comes from |
| ----- | ------- | ------------------- |
| Cluster name | `k8s-dsgpu-prd-ase1` | GKE cluster provisioning (Terraform) |
| Environment | `prd` | Cluster naming convention |
| ArgoCD namespace | See table below | Depends on environment |
| Target branch | See table below | Depends on environment |
| Initial tools | `keda`, `contour`, `external-secrets`, etc. | Platform team decision |

**Environment → branch / ArgoCD namespace mapping:**

| Environment | Branch | ArgoCD namespace | Values dir |
| ----------- | ------ | ---------------- | ---------- |
| prd | `main` | `argocd-prd` | `values/prd/` |
| stg / dev | `develop` | `argocd-dev` | `values/dev/` |
| int | `pre-prod` | `argocd-shared-int` | `values/int/` |

---

## Step 1: Create the incubator file

Create `incubator/<env>/<cluster>.yaml`:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: incubator-infra-<cluster>
  namespace: <argocd-namespace>       # argocd-prd | argocd-dev | argocd-shared-int
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: 'https://github.com/Meesho/devops-infra-argo-config'
    targetRevision: <branch>          # main (prd) | develop (stg/dev) | pre-prod (int)
    path: generic-argo-apps-chart
    helm:
      valueFiles:
      - ../values/<env>/incubator-infra-<cluster>-values.yaml
  destination:
    name: in-cluster
    namespace: <argocd-namespace>     # argocd-prd | argocd-dev | argocd-shared-int
```

**Key points:**
- `metadata.name`: `incubator-infra-<cluster>` (matches the file name minus `.yaml`).
- `spec.project`: `default` (the incubator itself runs in the ArgoCD admin cluster).
- `spec.source.path`: Always `generic-argo-apps-chart`.
- `helm.valueFiles`: Relative path to the values file from the chart directory.
- `spec.destination.name`: `in-cluster` (the incubator runs on the ArgoCD admin cluster, not the target cluster).

---

## Step 2: Create the values file

Create `values/<env>/incubator-infra-<cluster>-values.yaml`:

```yaml
clusterSpec:
  destination:
    server: ""
    name: "<cluster>"

argocdSpec:
  namespace: <argocd-namespace>       # argocd-prd | argocd-dev | argocd-shared-int

teamSpec:
  devops:
    source:
      repoURL: https://github.com/Meesho/devops-infra-helm-charts
      targetRevision: <branch>        # main (prd) | develop (stg/dev) | pre-prod (int)
      path: helm-templates
      valueFiles: ../../helm-overrides/<cluster>
    labels:
      bu: infra
      team: devops
      env: <env>
      cluster: <cluster>

appSpec: []
```

Start with an empty `appSpec` list. Add tools in a follow-up PR after the cluster is bootstrapped.

---

## Step 3: Register the cluster in the admin cluster's values file

The admin cluster (`k8s-admin-prd-ase1`) may need an entry to reference the new cluster's ArgoCD Application. Check if the admin values file at `values/prd/incubator-infra-k8s-admin-prd-ase1-values.yaml` needs an `appSpec` entry for the new cluster's ArgoCD instance.

---

## Step 4: Create external-name-service files (if zone-c)

For multi-zone clusters (e.g., `-ase1c`), create:

1. `external-name-service-incubator/<env>/<zone>/external-name-service-incubator-<cluster>.yaml`
2. Update external-name-service values for the zone if needed.

---

## Step 5: Validate locally

```bash
# Render the incubator Application
helm template generic-argo-apps-chart/ \
  -f values/<env>/incubator-infra-<cluster>-values.yaml

# Verify YAML syntax
yamllint values/<env>/incubator-infra-<cluster>-values.yaml
yamllint incubator/<env>/incubator-infra-<cluster>.yaml
```

---

## Step 6: Open PR

- Both files (incubator + values) in the same PR.
- Target branch: matches environment — `main` (prd), `develop` (stg/dev), `pre-prod` (int).
- Required: Platform team review.

---

## Step 7: Post-merge bootstrap (manual, platform team)

After the PR merges:

1. The incubator Application auto-syncs to the ArgoCD admin cluster.
2. ArgoCD reads the values file and renders child Applications (initially none if `appSpec` is empty).
3. Verify in ArgoCD UI that the incubator Application is healthy.
4. Add initial tools via [add-tool-to-cluster.md](add-tool-to-cluster.md).

---

## Naming convention reference

| Component | Convention | Example |
| --------- | ---------- | ------- |
| Incubator file | `incubator/<env>/incubator-infra-<cluster>.yaml` | `incubator/prd/incubator-infra-k8s-dsgpu-prd-ase1.yaml` |
| Values file | `values/<env>/incubator-infra-<cluster>-values.yaml` | `values/prd/incubator-infra-k8s-dsgpu-prd-ase1-values.yaml` |
| Incubator Application name | `incubator-infra-<cluster>` | `incubator-infra-k8s-dsgpu-prd-ase1` |

---

## Checklist

- [ ] GKE cluster is provisioned and registered in ArgoCD
- [ ] Incubator file created with correct name and paths
- [ ] Values file created with correct `clusterSpec.destination.name`
- [ ] `valueFiles` path in incubator points to correct values file
- [ ] `teamSpec.source.valueFiles` points to correct `helm-overrides/<cluster>` directory
- [ ] Labels match the cluster name and environment
- [ ] Both files pass `yamllint`
- [ ] Pre-commit hooks pass