119 lines
3.8 KiB
Markdown
119 lines
3.8 KiB
Markdown
# Procedure: Deboard a Tool from a Cluster
|
|
|
|
> Step-by-step guide for safely removing an `appSpec` entry from a cluster values file.
|
|
>
|
|
> **Layer:** 1-T (Tool-Mediated). Agent generates the diff and opens a PR.
|
|
>
|
|
> **Blast radius:** Single cluster. After merge, ArgoCD deletes the Application and its resources via finalizer.
|
|
>
|
|
> Per AI Blitz Plan §5.2.
|
|
|
|
---
|
|
|
|
## Warning: deletion is destructive
|
|
|
|
Removing an `appSpec` entry causes ArgoCD to delete the child Application, which triggers the `resources-finalizer.argocd.argoproj.io` to **delete all Kubernetes resources** the Application manages. This includes Deployments, Services, ConfigMaps, PVCs, and the namespace (if `CreateNamespace=true` and the namespace is not shared).
|
|
|
|
**Confirm with the tool owner before proceeding.**
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
1. Confirm the tool is no longer needed on this cluster.
|
|
2. Confirm no other system depends on the tool's namespace (e.g., shared `victoriametrics` namespace — removing one app doesn't remove the namespace, but verify).
|
|
3. If the tool has persistent storage (PVC), confirm data can be discarded or has been backed up.
|
|
|
|
---
|
|
|
|
## Steps
|
|
|
|
### Step 1: Find the appSpec entry
|
|
|
|
```bash
|
|
grep -n -A 5 'name: <tool>' values/<env>/incubator-infra-<cluster>-values.yaml
|
|
```
|
|
|
|
Record the line numbers of the entire entry block.
|
|
|
|
### Step 2: Check for namespace sharing
|
|
|
|
```bash
|
|
# Does any other appSpec entry use the same namespace?
|
|
grep 'namespace: <namespace>' values/<env>/incubator-infra-<cluster>-values.yaml
|
|
```
|
|
|
|
If another entry shares the namespace, removing this entry will NOT delete the namespace — only the Application's resources. This is usually safe, but confirm.
|
|
|
|
### Step 3: Compute the Application name that will be deleted
|
|
|
|
```text
|
|
<name>-<mungedCluster>-<env>
|
|
```
|
|
|
|
Use this to verify the Application in ArgoCD before the PR merges.
|
|
|
|
### Step 4: Remove the appSpec entry
|
|
|
|
Delete the entire entry block from the values file:
|
|
|
|
```yaml
|
|
# Remove this block entirely:
|
|
- name: <tool>
|
|
namespace: <namespace>
|
|
chartDir: <chartDir>
|
|
valuesDir: <valuesDir>
|
|
```
|
|
|
|
Do not leave empty lines or dangling list markers.
|
|
|
|
### Step 5: Validate
|
|
|
|
```bash
|
|
helm template generic-argo-apps-chart/ \
|
|
-f values/<env>/incubator-infra-<cluster>-values.yaml
|
|
|
|
yamllint values/<env>/incubator-infra-<cluster>-values.yaml
|
|
```
|
|
|
|
Confirm the removed Application no longer appears in the rendered output.
|
|
|
|
### Step 6: Open PR
|
|
|
|
- Title: `deboard <tool> from <cluster>`
|
|
- PR body must include: what the tool was doing, why it's being removed, who confirmed the removal, and namespace cleanup plan.
|
|
- Required: Platform team review + tool owner acknowledgment
|
|
|
|
### Step 7: Post-merge cleanup
|
|
|
|
After merge and ArgoCD sync completes:
|
|
|
|
1. Verify in ArgoCD that the Application has been deleted.
|
|
2. Verify the namespace is gone (or still exists if shared — expected).
|
|
3. If `custom-values.yaml` in `devops-infra-helm-charts` is no longer needed, clean it up in a separate PR.
|
|
|
|
---
|
|
|
|
## Special case: decommissioning a cluster
|
|
|
|
If you are removing **all** tools from a cluster as part of cluster decommission:
|
|
|
|
1. Do **not** delete the incubator file in this PR — that is a separate step requiring confirmation.
|
|
2. First empty the `appSpec` list: `appSpec: []`
|
|
3. After all tools are confirmed deleted, open a second PR to delete the incubator and values files.
|
|
|
|
See escalation matrix: [docs/global/escalation-matrix.md](../../global/escalation-matrix.md) — cluster decommission requires coordinated sign-off.
|
|
|
|
---
|
|
|
|
## Checklist
|
|
|
|
- [ ] Tool owner confirmed removal
|
|
- [ ] Namespace sharing checked — no unintended namespace deletion
|
|
- [ ] Persistent storage impact assessed
|
|
- [ ] Application name identified for post-merge verification
|
|
- [ ] `helm template` renders without errors
|
|
- [ ] `yamllint` passes
|
|
- [ ] Pre-commit hooks pass
|
|
- [ ] PR body includes removal rationale and namespace cleanup plan
|