devops-infra-argo-config-gcp/skills/infra/fleet-wide-rollout.md

Skill: Fleet-Wide Tool Rollout

Parameterized agent task for adding a new tool to multiple clusters in a single PR.

Layer: 1-T (Tool-Mediated). Agent generates the diff and opens a PR.

Procedure: docs/platform/procedures/fleet-wide-tool-rollout.md

---

When to use

Trigger on requests like:

• "Add <tool> to all prd clusters"
• "Roll out <tool> to demand, supply, central, farmiso"
• "Onboard <tool> fleet-wide"
• "Deploy <tool> across all data clusters"

Do NOT use for single-cluster additions — use add-tool.md instead.

---

Inputs

Parameter	Required	Example	Source
tool_name	yes	coroot	User request
target_clusters	yes	["k8s-demand-prd-ase1", "k8s-supply-prd-ase1"] or "all-prd"	User request
env	yes	prd	Derived from cluster names
chart_dir	yes	coroot	Must exist in devops-infra-helm-charts/helm-templates/
values_dir_pattern	yes	coroot or coroot-<cluster>	Depends on whether overrides are cluster-specific
namespace_pattern	yes	prd-<tool> or <tool>-<mungedCluster>	Convention from procedure

---

Steps (deterministic)

Step 0: Resolve target clusters

If target_clusters = "all-prd":

ls values/prd/

Otherwise: use the provided list. Derive env from cluster names.

Step 1: Verify chartDir exists

ls /path/to/devops-infra-helm-charts/helm-templates/<chart_dir>/

Halt if missing.

Step 2: Verify valuesDir exists for each cluster

for cluster in <target_clusters>; do
  ls /path/to/devops-infra-helm-charts/helm-overrides/$cluster/<values_dir>/custom-values.yaml \
    && echo "OK: $cluster" || echo "MISSING: $cluster"
done

Halt if any cluster is missing its override. Report which clusters are missing so the user can create them in devops-infra-helm-charts first.

Step 3: Check for existing entries (idempotency)

for f in <target_values_files>; do
  grep "name: <tool_name>" "$f" && echo "Already exists in $f"
done

Skip clusters that already have the tool.

Step 4: Compute Application names, check collisions

For each cluster, compute <tool_name>-<mungedCluster>-<env> and verify no collision.

Step 5: Append appSpec entries

For each cluster values file, append:

  - name: <tool_name>
    namespace: <namespace>
    chartDir: <chart_dir>
    valuesDir: <values_dir>

Use the cluster-specific valuesDir if overrides are cluster-named.

Step 6: Validate all modified files

for f in <modified_files>; do
  helm template generic-argo-apps-chart/ -f "$f" | grep -c "kind: Application"
  yamllint "$f"
done

All must pass.

Step 7: Open PR

Title: onboard <tool_name> to [all prd | <cluster-group>] clusters

PR body checklist:

• Clusters modified: [list]
• chartDir exists: devops-infra-helm-charts/helm-templates/<chart_dir>/
• valuesDir verified for each cluster
• Application names computed, no collisions
• helm template passes for all clusters
• Pre-commit hooks pass

---

Output

Multiple modified values/<env>/<cluster>-values.yaml files, one new appSpec entry per file.

---

Gotchas

1. valuesDir names vary per cluster. Some tools use <tool> as the valuesDir (same for all clusters); others use cluster-specific directories like coroot-central. Verify each one — do not assume uniformity.
2. Auto-sync = simultaneous fleet deploy. All clusters deploy on merge. If you want staged rollout, open separate PRs — one per cluster group.
3. Skip already-deployed clusters silently. If a cluster already has the tool (Step 3), skip it with a note in the PR body — don't add a duplicate entry.
4. Namespace convention matters. Confirm the namespace pattern with the tool owner. Some tools use prd-<tool> (env-prefixed), others use <tool>-<mungedCluster>. Inconsistency across clusters is a drift risk.