gitadmin/devops-infra-argo-config-gcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added repo

Browse Source
This commit is contained in:
Your Name
2026-08-26 04:03:34 +05:30
parent 2389ec1fd6
commit 1055e1394f
150 changed files with 12395 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/global/AGENT_BOUNDARIES.md
+94
View File
@@ -0,0 +1,94 @@
# Agent Boundaries — `devops-infra-argo-config`
> Authoritative document defining which operations agents can perform in this repository.
>
> **Audience:** Claude Code agents, human reviewers of agent-generated PRs.
>
> **Model:** 3-Layer Operating Model per AI Blitz Plan §4.2.
---
## The 3-Layer Model
| Layer | What agents do | Safety gate |
| ----- | -------------- | ----------- |
| **Layer 1-T** (Tool-Mediated) | Generate YAML diff via tools, open PR. No direct YAML edits. | PR review by platform team. |
| **Layer 2** (Agent-Readable Advisory) | Research, analyze, suggest. Human executes. | Human judgment. |
| **Layer 3** (Agent-Blocked) | Refuse the write. Direct user to correct owner. | Hard stop. |
**This repo is Layer 1-T.** All YAML changes should go through `incubator-tool` or `argo-app-tool` where available. Direct YAML edits bypass tool validation but are acceptable when tools are not available, provided the PR checklist is followed.
---
## Per-operation classification
### Layer 1-T — Standard operations
| Operation | Files affected | Blast radius | Tool | Approval |
| --------- | ------------- | ------------ | ---- | -------- |
| Add appSpec entry to values file | `values/<env>/<cluster>-values.yaml` | Single cluster, single tool | `argo-app-tool` | Platform team PR review |
| Remove appSpec entry from values file | `values/<env>/<cluster>-values.yaml` | Single cluster, single tool | `argo-app-tool` | Platform team PR review |
| Update appSpec entry (chartDir, valuesDir, namespace) | `values/<env>/<cluster>-values.yaml` | Single cluster, single tool | `argo-app-tool` | Platform team PR review |
| Add `nameOverride` to appSpec entry | `values/<env>/<cluster>-values.yaml` | Single cluster, single tool | `argo-app-tool` | Requires justification in PR |
| Create incubator file for new cluster | `incubator/<env>/<cluster>.yaml` | New cluster bootstrap | `incubator-tool` | Platform team PR review |
| Create values file for new cluster | `values/<env>/<cluster>-values.yaml` | New cluster bootstrap | `argo-app-tool` | Platform team PR review |
| Add external-name-service files | `external-name-service-*/**` | Cross-cluster DNS routing | Manual | Platform team PR review |
| Modify AppProject | `projects/*.yaml` | All Applications referencing the project | Manual | Platform team + security review |
### Layer 1-T — HIGH RISK operations
These are Layer 1-T (agent may generate the diff + open PR) but carry elevated risk:
| Operation | Why high risk | Extra requirement |
| --------- | ------------- | ----------------- |
| Edit `generic-argo-apps-chart/` | Auto-sync means a bad template change breaks ALL clusters immediately | Explicit platform-team sign-off; test with `helm template` against multiple values files |
| Modify `clusterSpec.destination` in values file | Redirects all tools to a different cluster | Verify cluster name exists in GKE |
| Change `teamSpec.source.targetRevision` | Switches chart source branch — affects all tools on the cluster | Must match env branch: `main` (prd), `develop` (stg/dev), `pre-prod` (int) |
| Change `teamSpec.source.repoURL` | Switches chart source repo entirely | Requires platform-team approval |
| Bulk-add appSpec entries to multiple clusters | Fleet-wide tool rollout | Each cluster's entry must be individually verified |
### Layer 2 — Advisory only (no agent write)
| Operation | Agent action |
| --------- | ------------ |
| Run `argocd app sync <appName>` | Suggest the command; do not execute. Infra apps auto-sync from `main`. |
| Run `argocd app diff <appName>` | Suggest the command if cluster access is available. |
| Investigate sync failure | Read logs, analyze values, suggest fix. See [runbooks/argocd-sync-failure.md](../platform/runbooks/argocd-sync-failure.md). |
| Check which clusters have a tool | Run `grep -rl 'name: <tool>' values/` — read-only. |
| Audit AppProject scope | Read `projects/` and list dependent Applications — advisory only. |
### Layer 3 — Hard stops (agent must refuse)
| Operation | Why blocked | Redirect |
| --------- | ----------- | -------- |
| Edit `repository.yaml` | Owned by `registry-bootstrap` automation | Direct user to upstream automation |
| Run `kubectl apply` against a cluster | Out of scope — this repo is GitOps, not in-cluster mutation | Direct to incident response procedures |
| Push directly to `main` | Branch protection enforced at org level | Open PR instead |
| Delete an incubator file | Orphans the cluster's tooling | Require confirmation that cluster is decommissioned |
| Force-push to any branch | Destructive, irreversible | Never do this |
| Skip pre-commit hooks (`--no-verify`) | Bypasses TruffleHog, CAC, Yaak | Fix the hook failure instead |
| Add secrets to YAML | TruffleHog will block; if bypassed, requires history purge | Use external-secrets operator |
---
## Cross-cut verification checklist (every Layer 1-T PR)
Before merging any agent-generated PR:
1. Tool was used (or manual edit follows tool conventions).
2. `chartDir` exists in `devops-infra-helm-charts/helm-templates/`.
3. `valuesDir` exists in `devops-infra-helm-charts/helm-overrides/<cluster>/`.
4. Auto-generated Application name is <= 253 characters.
5. No name collision with existing appSpec entries in the same values file.
6. `nameOverride` is used only when justified (R7).
7. `spec.project` references an existing AppProject (`sre` or `sec`).
---
## Escalation procedure
When an operation falls outside Layer 1-T boundaries:
1. Refuse the write.
2. Cite this document and the specific layer/operation.
3. Suggest the human ask the platform team or file a CMR (Change Management Request).
docs/global/SANCTITY_RULES.md
+118
View File
@@ -0,0 +1,118 @@
# Sanctity Rules — `devops-infra-argo-config`
> Non-negotiable rules for this repository. Violations are hard stops — not warnings, not suggestions.
>
> **Audience:** Every agent and human who opens a PR against this repo.
>
> **Enforcement:** Pre-commit hooks (TruffleHog, CAC validate, Yaak) + branch protection on `main`. A merge to `main` **immediately deploys** — infra apps are auto-sync.
---
## R1: `main` is production
A merge to `main` triggers immediate ArgoCD auto-sync across all managed clusters. There is no staging gate between merge and deploy.
- No experiments on `main`.
- No force-pushes to `main`.
- Every PR must pass pre-commit hooks and receive platform-team review before merge.
---
## R2: The incubator ↔ values ↔ generic-chart contract is sacred
Each cluster is bootstrapped by exactly three things working together:
1. **Incubator file** (`incubator/<env>/<cluster>.yaml`) — parent ArgoCD Application that points at the generic chart + the cluster's values file.
2. **Values file** (`values/<env>/<cluster>-values.yaml`) — defines `clusterSpec`, `teamSpec`, `argocdSpec`, and the `appSpec[]` list.
3. **Generic chart** (`generic-argo-apps-chart/`) — renders one child ArgoCD Application per `appSpec` entry.
Breaking any leg of this triangle breaks the cluster's tooling. The naming conventions in CLAUDE.md are load-bearing — they are identifiers, not cosmetic.
---
## R3: `appSpec` entries must reference existing chart and values in `devops-infra-helm-charts`
Every `appSpec[].chartDir` must correspond to a directory under `devops-infra-helm-charts/helm-templates/`. Every `appSpec[].valuesDir` must correspond to a directory under `devops-infra-helm-charts/helm-overrides/<cluster>/`. If either doesn't exist, the ArgoCD Application will fail to render.
**Verify before adding:**
```bash
# chartDir exists?
ls devops-infra-helm-charts/helm-templates/<chartDir>/
# valuesDir exists?
ls devops-infra-helm-charts/helm-overrides/<cluster>/<valuesDir>/custom-values.yaml
```
---
## R4: Child Application names are auto-generated — do not invent names
The generic chart template generates Application names as:
```
<appSpec.name>-<mungedCluster>-<env>
```
Where cluster munging strips: `k8s-`, `prd-`, `int-`, `dev-`, `-ase1`; maps `-ase1c` to `-c`.
Example: `keda` on `k8s-central-prd-ase1` becomes `keda-central-prd`.
Only use `nameOverride` when:
- The auto-generated name would exceed 253 characters (Kubernetes limit).
- There is a genuine name collision between two appSpec entries.
---
## R5: `repository.yaml` is automation-owned
Owned by `registry-bootstrap`. Direct edits will be overwritten. To change ownership metadata, go to the upstream automation.
---
## R6: Secrets stay out of git
No credentials, tokens, API keys, or passwords in any YAML file. The TruffleHog pre-commit hook will block the commit. If a secret is accidentally committed, it must be force-purged from history — this requires platform-team involvement.
---
## R7: `nameOverride` is exceptional, not routine
Most PRs should not include `nameOverride`. When reviewing, flag any `nameOverride` that lacks a comment explaining why the auto-generated name was insufficient.
---
## R8: One incubator file per cluster, one values file per cluster
The 1:1 mapping between incubator files and values files is structural. Do not create multiple incubator files for the same cluster or share a values file across clusters.
---
## R9: Cluster folder/file names are routing keys
The cluster name in the values file (`clusterSpec.destination.name`) must match the incubator file name and the `helm-overrides/` folder name in the sister repo. These names are used by ArgoCD for cluster routing — a mismatch means tools deploy to the wrong cluster or fail silently.
---
## R10: Pre-commit hooks must pass
Run `pre-commit run --all-files` before pushing. Never use `--no-verify` to skip hooks. If a hook fails, fix the root cause.
The hooks are: TruffleHog (secrets scan), CAC validate (config validation), Yaak (additional linting).
---
## R11: Branch protection trumps everything
All changes go via PR with required review. Direct pushes to `main` are blocked at the GitHub org level. No exceptions for "quick fixes" — the blast radius of a bad merge is immediate cluster-wide impact.
---
## R12: AppProject wildcards are dangerous
The `sre` and `sec` projects currently allow `*` for source repos and destinations. Widening or modifying AppProject scope requires auditing every Application that references the project. A misconfigured project could allow unauthorized deployments.
---
## How to cite these rules
In PR reviews or agent halt messages, cite as: *"Blocked by R3 (SANCTITY_RULES.md) — chartDir must exist in devops-infra-helm-charts before adding appSpec."*
docs/global/coding-guidelines/argocd.md
+135
View File
@@ -0,0 +1,135 @@
> Per [AI Blitz Plan §6](../AGENT_BOUNDARIES.md). Layer: 1-T.
# Coding Guidelines — ArgoCD Application Manifests
Conventions for ArgoCD `Application` YAML files in `incubator/<env>/`.
---
## Required fields
Every incubator Application must have all of these:
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: incubator-infra-<cluster> # Must match filename without .yaml
namespace: <argocd-namespace> # argocd-prd | argocd-dev | argocd-shared-int
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: 'https://github.com/Meesho/devops-infra-argo-config'
targetRevision: <branch> # main | develop | pre-prod
path: generic-argo-apps-chart
helm:
valueFiles:
- ../values/<env>/incubator-infra-<cluster>-values.yaml
destination:
name: in-cluster
namespace: <argocd-namespace>
```
---
## `metadata.name` rules
- Must equal the filename without `.yaml` — this is ArgoCD's tracking key (R9).
- Convention: `incubator-infra-<gke-cluster-name>` (e.g. `incubator-infra-k8s-central-prd-ase1`).
- Changing this after the Application is registered breaks ArgoCD tracking; treat as immutable.
---
## `metadata.namespace` (ArgoCD namespace)
| Environment | Namespace |
| ----------- | --------- |
| prd | `argocd-prd` |
| stg / dev | `argocd-dev` |
| int | `argocd-shared-int` |
The `metadata.namespace` and `spec.destination.namespace` must be the same value.
---
## `spec.source.repoURL` allow-list
Only one value is permitted:
```text
https://github.com/Meesho/devops-infra-argo-config
```
Changing this is a **Layer 3** operation. See [AGENT_BOUNDARIES.md](../AGENT_BOUNDARIES.md).
---
## `spec.source.targetRevision`
Must match the environment branch:
| Environment | `targetRevision` |
| ----------- | ---------------- |
| prd | `main` |
| stg / dev | `develop` |
| int | `pre-prod` |
Never pin to a feature branch. Feature branches are not monitored by auto-sync.
---
## `spec.source.path`
Always `generic-argo-apps-chart`. This is the Helm chart that renders child Applications from the values file.
---
## `spec.destination.name`
Always `in-cluster`. Incubator Applications run on the ArgoCD admin cluster itself, not on the target GKE cluster.
---
## `spec.destination.namespace`
Must match `metadata.namespace` — same ArgoCD namespace.
---
## Finalizer
The finalizer `resources-finalizer.argocd.argoproj.io` must be present. It causes ArgoCD to cascade-delete child Applications when the parent incubator is deleted. Removing it creates orphaned child Applications.
---
## `helm.valueFiles` path convention
Paths are relative to the chart root (`generic-argo-apps-chart/`), so `..` is required to reach the repo root:
```text
generic-argo-apps-chart/ (chart root = ArgoCD's working dir for this source)
../values/<env>/incubator-infra-<cluster>-values.yaml
└── resolves to: values/<env>/incubator-infra-<cluster>-values.yaml at repo root
```
---
## Naming invariant
Directory and values path vary by env — but the invariant itself is universal:
```text
prd (main):
Filename: incubator/prd/incubator-infra-k8s-central-prd-ase1.yaml
metadata.name: incubator-infra-k8s-central-prd-ase1
helm.valueFiles: ../values/prd/incubator-infra-k8s-central-prd-ase1-values.yaml
stg (develop):
Filename: incubator/infra/incubator-infra-k8s-central-stg-ase1.yaml
metadata.name: incubator-infra-k8s-central-stg-ase1
helm.valueFiles: ../values/dev/ incubator-infra-k8s-central-stg-ase1-values.yaml
```
All three must be consistent. Any mismatch causes a routing failure.
docs/global/coding-guidelines/helm.md
+141
View File
@@ -0,0 +1,141 @@
> Per [AI Blitz Plan §6](../AGENT_BOUNDARIES.md). Layer: 1-T.
# Coding Guidelines — Helm Values Files
Conventions for cluster values files (`values/<env>/incubator-infra-<cluster>-values.yaml`).
---
## File naming
```text
values/<values-dir>/incubator-infra-<gke-cluster-name>-values.yaml
```
The values directory name does **not** always match the environment name:
| Environment | Branch | Values dir | Example |
| ----------- | ------ | ---------- | ------- |
| prd | `main` | `values/prd/` | `values/prd/incubator-infra-k8s-central-prd-ase1-values.yaml` |
| stg | `develop` | `values/dev/` | `values/dev/incubator-infra-k8s-central-stg-ase1-values.yaml` |
| dev | `develop` | `values/dev/` | `values/dev/incubator-infra-k8s-central-dev-ase1-values.yaml` |
| int | `pre-prod` | `values/int/` | `values/int/incubator-infra-k8s-shared-int-ase1-values.yaml` |
| admin | `main` / `pre-prod` | `values/admin/` | `values/admin/incubator-infra-k8s-devops-admin-ase1-values.yaml` |
Note: stg and dev clusters both live under `values/dev/` on the `develop` branch.
---
## Top-level structure
Four keys in this exact order — no additions, no reordering:
```yaml
clusterSpec:
argocdSpec:
teamSpec:
appSpec:
```
---
## `clusterSpec`
```yaml
clusterSpec:
destination:
server: "" # Always empty string — name-based routing only
name: "k8s-central-prd-ase1" # Must match GKE cluster name and helm-overrides/ folder
```
- `server` is always `""`. Never set a URL here.
- `name` is immutable after cluster registration. Changing it redirects all tooling.
---
## `argocdSpec`
```yaml
argocdSpec:
namespace: argocd-prd # argocd-prd | argocd-dev | argocd-shared-int
```
| Environment | Namespace |
| ----------- | --------- |
| prd | `argocd-prd` |
| stg / dev | `argocd-dev` |
| int | `argocd-shared-int` |
---
## `teamSpec`
```yaml
teamSpec:
devops:
source:
repoURL: https://github.com/Meesho/devops-infra-helm-charts
targetRevision: main # main | develop | pre-prod
path: helm-templates
valueFiles: ../../helm-overrides/<cluster>
labels:
bu: infra
team: devops
env: prd # prd | stg | dev | int | admin
cluster: k8s-central-prd-ase1 # Must match clusterSpec.destination.name
```
**`repoURL` allow-list:** Only `https://github.com/Meesho/devops-infra-helm-charts` is permitted.
**`targetRevision`** must match the environment:
| Environment | `targetRevision` |
| ----------- | ---------------- |
| prd | `main` |
| stg / dev | `develop` |
| int | `pre-prod` |
**`valueFiles` path:** Relative from `helm-templates/` root (two levels up) to the cluster's override directory. Always `../../helm-overrides/<cluster>`.
**`labels.cluster`** must equal `clusterSpec.destination.name` exactly — it is used in Application name generation.
---
## `appSpec`
```yaml
appSpec:
- name: keda # Short, lowercase, hyphen-separated tool name
namespace: keda-central-prd # Target K8s namespace
chartDir: keda # Directory under helm-templates/ in helm-charts repo
valuesDir: keda # Directory under helm-overrides/<cluster>/ in helm-charts repo
```
- One entry = one ArgoCD child Application.
- Generated name: `<name>-<mungedCluster>-<env>` (e.g. `keda-central-prd`).
- `chartDir` and `valuesDir` **must exist** in `devops-infra-helm-charts` before adding the entry (R3).
### Optional fields
```yaml
- name: coredns
namespace: kube-system
chartDir: coredns
valuesDir: coredns
nameOverride: coredns-central-prd # Only for name collisions or >253 char names (R7)
additionalValueFiles:
- ../../helm-templates/coredns/gcp-ase1a-values.yaml
```
---
## Common mistakes
| Mistake | Impact | Fix |
| ------- | ------ | --- |
| `server: <url>` instead of `""` | May route to wrong cluster | Always use `""` |
| Wrong `targetRevision` for env | Tools pull from wrong chart branch | Use `main`/`develop`/`pre-prod` per env |
| `chartDir` not in `helm-templates/` | ArgoCD render failure | Verify in `devops-infra-helm-charts` first |
| `valuesDir` missing `custom-values.yaml` | ArgoCD render failure | Add `custom-values.yaml` to the override dir |
| `labels.cluster` ≠ `clusterSpec.destination.name` | Application name munging uses wrong cluster | Keep them identical |
| Duplicate `name` in `appSpec` | Application name collision | Use `nameOverride` for multi-instance tools |
docs/global/coding-guidelines/infra-argo.md
+149
View File
@@ -0,0 +1,149 @@
# Coding Guidelines — Values Files and appSpec Entries
> YAML authoring conventions for `devops-infra-argo-config`.
>
> **Scope:** `values/<env>/<cluster>-values.yaml` files and `appSpec` entries within them.
---
## Values file structure
Every values file has exactly four top-level keys in this order:
```yaml
clusterSpec: # Cluster identity — destination for ArgoCD
argocdSpec: # ArgoCD namespace
teamSpec: # Source repo, labels, team identity
appSpec: # List of tools to deploy to this cluster
```
Do not add other top-level keys. Do not reorder these keys.
---
## `clusterSpec` conventions
```yaml
clusterSpec:
destination:
server: "" # Leave empty — use name-based routing
name: "k8s-central-prd-ase1" # Must match the cluster name in GKE
```
- `server` is always empty string `""` — name-based routing is the standard.
- `name` must exactly match the GKE cluster name and the `helm-overrides/` folder in the sister repo.
---
## `argocdSpec` conventions
```yaml
argocdSpec:
namespace: argocd-prd # prd → argocd-prd | stg/dev → argocd-dev | int → argocd-shared-int
```
---
## `teamSpec` conventions
```yaml
teamSpec:
devops:
source:
repoURL: https://github.com/Meesho/devops-infra-helm-charts
targetRevision: main # Env-specific: prd=main, stg/dev=develop, int=pre-prod
path: helm-templates # Chart root in the helm-charts repo
valueFiles: ../../helm-overrides/k8s-central-prd-ase1 # Relative path to overrides
labels:
bu: infra # Always "infra" for this repo
team: devops # Always "devops" — maps to sre AppProject
env: prd # prd, int, dev, or admin
cluster: k8s-central-prd-ase1 # Must match clusterSpec.destination.name
```
- `targetRevision` must match the environment branch: `main` (prd), `develop` (stg/dev), `pre-prod` (int). Deviating from this requires explicit justification.
- `valueFiles` is a relative path from the chart source to the cluster's override directory in `devops-infra-helm-charts`.
- Labels are used by the generic chart template for Application naming and metadata.
---
## `appSpec` entry conventions
Each entry in `appSpec` defines one ArgoCD child Application:
```yaml
appSpec:
- name: keda # Short tool name (used in Application name generation)
namespace: keda-central-prd # Target namespace (auto-created by ArgoCD)
chartDir: keda # Directory under helm-templates/ in helm-charts repo
valuesDir: keda # Directory under helm-overrides/<cluster>/ in helm-charts repo
```
### Required fields
| Field | Description | Convention |
| ----- | ----------- | ---------- |
| `name` | Short tool identifier | Lowercase, hyphen-separated. Used in rendered Application name. |
| `namespace` | Kubernetes namespace for the tool | Pattern: `<tool>-<mungedCluster>` or shared namespace (e.g., `victoriametrics`, `kube-system`) |
| `chartDir` | Chart directory name in `helm-templates/` | Must exist in `devops-infra-helm-charts` |
| `valuesDir` | Override directory name in `helm-overrides/<cluster>/` | Must exist in `devops-infra-helm-charts` |
### Optional fields
| Field | When to use |
| ----- | ----------- |
| `nameOverride` | Only when the auto-generated name exceeds 253 chars or collides with another entry |
| `additionalValueFiles` | When a tool needs region-shared overlays (e.g., CoreDNS GCP zone values) |
### Ordering
New appSpec entries should be appended at the end of the list. Do not sort alphabetically — the order reflects deployment history and makes diffs cleaner.
---
## Namespace naming patterns
| Pattern | When |
| ------- | ---- |
| `<tool>-<mungedCluster>` | Default. Example: `keda-central-prd`, `contour-external-central-prd` |
| Shared namespace | When multiple tools share a namespace. Example: `victoriametrics` for all VM tools, `kube-system` for system tools |
| Tool-specific with role | Multi-instance tools. Example: `contour-internal-0-central-prd`, `contour-internal-1-central-prd` |
---
## Application name generation
The generic chart template generates names as:
```
<name>-<mungedCluster>-<env>
```
**Cluster munging rules** (applied in order by the Helm template):
1. Replace `dp-` with placeholder, `backup` with placeholder
2. Strip: `p-`, `prd-`, `int-`, `dev-`, `-cluster`
3. Replace: `prod-ops` → `infra`, `-ase1c` → `-c`, `-ase1` → (empty), `k8s-` → (empty)
4. Restore placeholders
**Example:** `k8s-central-prd-ase1` → `central-prd` → Application name: `keda-central-prd`
---
## YAML formatting
- 2-space indentation (no tabs).
- No trailing whitespace.
- Single newline at end of file.
- Quote strings only when YAML requires it (e.g., empty strings `""`).
- Use block style for lists (one `- ` per line), not flow style.
---
## Common mistakes
| Mistake | Why it's wrong | Fix |
| ------- | -------------- | --- |
| Adding `nameOverride` without justification | Breaks naming consistency; see R7 | Remove unless name > 253 chars or collision |
| `chartDir` that doesn't exist in helm-charts | ArgoCD will fail to render the Application | Verify with `ls helm-templates/<chartDir>/` |
| Duplicate `name` in same appSpec list | Two Applications will have the same name → conflict | Use unique tool names or `nameOverride` for multi-instance |
| Changing `targetRevision` away from env branch | All tools on the cluster pull from the wrong branch | Must match env: `main` (prd), `develop` (stg/dev), `pre-prod` (int) |
docs/global/escalation-matrix.md
+60
View File
@@ -0,0 +1,60 @@
# Escalation Matrix — `devops-infra-argo-config`
> When an agent or human hits a boundary, blocker, or incident in this repo, use this matrix to determine who to contact and how fast.
>
> **Audience:** Agents (to know when to halt and hand off), on-call humans (to know who owns what).
>
> Per [AI Blitz Plan §5.2](AGENT_BOUNDARIES.md). Layer: **1-T**.
---
## Escalation triggers
| Situation | Urgency | Who | How |
| --------- | ------- | --- | --- |
| Agent would need to edit `repository.yaml` | Non-urgent | Platform team | Slack `#devops-tech` — this is automation-owned |
| Agent would need to push to `main`/`develop`/`pre-prod` directly | Non-urgent | Platform team | Open PR instead; if truly urgent, page on-call |
| `generic-argo-apps-chart/` change needed | Non-urgent | Platform team senior review | PR + explicit sign-off from two platform team members |
| Helm render failure after PR merge, tools broken | **Urgent** | On-call platform engineer | PagerDuty `Devops` + Slack `#devops-tech` |
| ArgoCD sync stuck on a cluster > 10 minutes | **Urgent** | On-call platform engineer | PagerDuty `Devops` + Slack `#devops-tech` |
| Accidental secret committed to git | **Critical** | Platform team lead + security | Slack `#devops-tech` immediately; do NOT merge; requires history purge |
| Incubator file deleted, cluster tooling orphaned | **Critical** | On-call platform engineer | PagerDuty `Devops` + Slack `#devops-tech` — restore file immediately |
| `clusterSpec.destination.name` changed to wrong cluster | **Critical** | On-call platform engineer | Revert PR immediately; tools may be deploying to wrong cluster |
| `teamSpec.source.repoURL` changed | **Critical** | On-call platform engineer | Revert PR immediately; all cluster tools affected |
| AppProject (`sre`/`sec`) modified | **Urgent** | Platform team + security | Security review required before merge |
| Pre-commit hooks consistently failing on valid YAML | Non-urgent | Platform team | Slack `#devops-tech` — may be hook version issue |
| Cluster decommission requested | Non-urgent | Platform team + cluster owner | Coordinated removal: incubator + values files + helm-overrides cleanup |
---
## Urgency definitions
| Level | Meaning | Response time |
| ----- | ------- | ------------- |
| **Critical** | Production tooling broken or security incident | Page immediately; respond in < 15 min |
| **Urgent** | Cluster sync stuck, tools not deploying | Page if outside business hours; Slack if in-hours; respond in < 1 hour |
| **Non-urgent** | Question, blocked agent, process clarification | Slack message; respond in same business day |
---
## Contact channels
| Channel | Purpose |
| ------- | ------- |
| Slack `#devops-tech` | Day-to-day questions, production incidents, and security-sensitive issues (secret exposure, AppProject changes) |
| PagerDuty `Devops` | On-call page for critical/urgent issues outside business hours |
---
## Agent halt protocol
When an agent reaches a Layer 3 boundary or an unresolvable blocker:
1. **Stop** — do not attempt a workaround.
2. **State** what operation was attempted and why it's blocked (cite rule from [SANCTITY_RULES.md](SANCTITY_RULES.md) or [AGENT_BOUNDARIES.md](AGENT_BOUNDARIES.md)).
3. **Recommend** the correct human path (e.g., "Open a PR targeting platform team review" or "Page on-call via PagerDuty").
4. **Do not retry** the blocked operation with slightly different parameters.
Example halt message:
> Blocked by R5 (SANCTITY_RULES.md): `repository.yaml` is owned by `registry-bootstrap` automation and must not be hand-edited. To change ownership metadata, the request must go to the upstream automation team via Slack `#devops-tech`.