devops-infra-argo-config-gcp/docs/platform/procedures/add-tool-to-cluster.md

Procedure: Add, Update, or Remove a Tool from a Cluster

Step-by-step guide for modifying appSpec entries in cluster values files.

Layer: 1-T (Tool-Mediated). Use argo-app-tool where available; manual YAML edits acceptable with checklist.

Blast radius: Single cluster, single tool. Auto-sync means changes deploy immediately after merge to the env branch (main for prd, develop for stg, pre-prod for int).

---

Prerequisites

Before starting, confirm:

1. The tool's chart exists in devops-infra-helm-charts/helm-templates/<chartDir>/ on the correct branch.
2. The tool's values override exists in devops-infra-helm-charts/helm-overrides/<cluster>/<valuesDir>/custom-values.yaml on the correct branch.
3. You know which cluster and environment the tool should deploy to.

Both this repo and devops-infra-helm-charts use the same branch convention:

Environment	Branch
Production (prd)	main
Staging (stg)	develop
Integration (int)	pre-prod

If either the chart or override doesn't exist on the target branch, the work starts in devops-infra-helm-charts first. See devops-infra-helm-charts procedures.

---

Add a new tool

Step 1: Identify the target values file

values/<env>/incubator-infra-<cluster>-values.yaml

Example: values/prd/incubator-infra-k8s-central-prd-ase1-values.yaml

Step 2: Verify chart and values exist in the sister repo

# Chart directory
ls /path/to/devops-infra-helm-charts/helm-templates/<chartDir>/

# Values override
ls /path/to/devops-infra-helm-charts/helm-overrides/<cluster>/<valuesDir>/custom-values.yaml

If either is missing, stop. Create them in devops-infra-helm-charts first.

Step 3: Compute the auto-generated Application name

Apply the munging rules from coding-guidelines/infra-argo.md:

<name>-<mungedCluster>-<env>

Example: kyverno on k8s-farmiso-prd-ase1 → kyverno-farmiso-prd

Verify:

• Name is <= 253 characters.
• Name doesn't collide with an existing Application (grep the values file).

Step 4: Add the appSpec entry

Append to the appSpec list in the values file:

  - name: kyverno
    namespace: kyverno-farmiso-prd
    chartDir: kyverno
    valuesDir: kyverno

Only add nameOverride if Step 3 revealed a collision or length issue.

Only add additionalValueFiles if the tool requires region-shared overlays.

Step 5: Validate locally

# Render the generic chart with the updated values
helm template generic-argo-apps-chart/ \
  -f values/prd/incubator-infra-<cluster>-values.yaml | grep -A 20 "name: kyverno"

# Verify no YAML syntax errors
yamllint values/prd/incubator-infra-<cluster>-values.yaml

Step 6: Open PR

• Target branch: main (prd) / develop (stg) / pre-prod (int)
• Required: Platform team review
• Pre-commit hooks must pass (TruffleHog, CAC, Yaak)

Step 7: After merge

ArgoCD auto-syncs from the env branch. The new Application will appear in ArgoCD within minutes. Verify in the ArgoCD UI that:

• The Application is created with the expected name.
• It syncs successfully.
• The target namespace is created.

---

Update an existing tool

Change chart version

Update chartDir to point to the new chart directory:

  - name: contour-internal-0
    chartDir: contour-v1.33.3          # was: contour

Verify the new chartDir exists in helm-templates/.

Change values directory

Update valuesDir:

  - name: vmagent
    valuesDir: victoria-metrics-agent-new   # was: victoria-metrics-agent

Verify the new valuesDir exists in helm-overrides/<cluster>/.

Change namespace

Update namespace. This is a destructive operation — ArgoCD will create the new namespace and deploy there, but the old namespace's resources are not automatically cleaned up.

  - name: ai-gateway
    namespace: ai-gateway-prd          # was: ai-gateway

After merge, manually clean up the old namespace if no other tools use it.

---

Remove a tool from a cluster

Step 1: Delete the appSpec entry

Remove the entire - name: ... block from the values file.

Step 2: Verify no other entries depend on it

Check if any other appSpec entries reference the same namespace or have dependencies on this tool.

Step 3: Open PR

After merge, the ArgoCD Application will be deleted by the finalizer (resources-finalizer.argocd.argoproj.io), which will also clean up the deployed resources.

Warning: If the Application has CreateNamespace=true in syncOptions and the namespace is shared with other tools, removing the Application will NOT delete the namespace. Namespace cleanup is manual.

---

Multi-cluster rollout

When adding a tool to multiple clusters:

1. Add the appSpec entry to each cluster's values file in the same PR.
2. Verify chart/values exist for each cluster — valuesDir names may differ per cluster.
3. Each cluster should have its own override directory in devops-infra-helm-charts/helm-overrides/<cluster>/<valuesDir>/.

---

Checklist

• chartDir exists in devops-infra-helm-charts/helm-templates/
• valuesDir exists in devops-infra-helm-charts/helm-overrides/<cluster>/
• Auto-generated Application name <= 253 characters
• No name collision with existing appSpec entries
• nameOverride used only if justified
• helm template renders without errors
• yamllint passes
• Pre-commit hooks pass