# Skill: build-failure-debugger

**One-line description:** Given a Jenkins job log from a failed devops-lib build, identify exactly which stage failed, trace the code path through devops-lib that produced the failure, and output a concrete fix with the exact file and line driving the error.

**Owner:** DevOps Platform team

---

## Why this skill exists

Jenkins job logs are long, noisy, and mixed-language (Groovy stack traces, Maven output, Docker build output, ArgoCD sync output). When a build fails, an engineer must:

1. Scroll through 2000+ lines to find the failure point
2. Know that "Build" maps to `buildMaven.groovy`, "Deploying to ArgoCD" maps to `deployArgoCD.groovy`, etc.
3. Know which code path was taken based on `build_tool` and `cicd_environment`
4. Know which external system (GAR, SonarQube, ArgoCD, Helm repo) caused the failure vs which devops-lib logic caused it

The mapping from Jenkins stage names to devops-lib source files is non-obvious and not documented anywhere. A "SonarQube Quality Gate" failure could mean: the threshold was breached, the project doesn't exist in Sonar yet, or a whitelist entry is missing. Three different fixes, identical log output.

This skill knows the stage-to-file mapping, the code paths for each failure mode, and the set of conditions that trigger each error.

---

## Trigger

- User pastes a Jenkins job log (or the relevant failure excerpt)
- `"this Jenkins build failed, help me debug it"` with log attached
- `"sonar quality gate failing for auth-service on develop"`
- `"build stuck at ArgoCD sync for catalog-service in prd"`
- Proactively: when a user shares a build URL, fetch the log and diagnose

---

## Input

- Jenkins job log (pasted text or URL to the Jenkins job)
- Optionally: the service `config.yaml` (to know `build_tool`, `skip_sonar`, `deployArgo`, etc.)

The skill can infer build_tool and environment from the log itself if config.yaml is not provided (Jenkins prints `build_tool` and `cicd_environment` at pipeline start via `constructParam.run()`).

---

## Expected output

Given: Jenkins log showing failure in "Deploying to ArgoCD" stage for `catalog-service` on `develop`

```
Build Failure Debug — catalog-service | branch: develop | env: stg

Stage failed:  Deploying to ArgoCD
devops-lib:    src/com/meesho/utilities/deployArgoCD.groovy

Step that failed: update_helm_repo (step 3 of 4)
Error in log:  ERROR: Helm push failed — chart version 0.1.142 already exists

Root cause: The Helm chart version was not bumped between this push and the previous
  build. deployArgoCD.groovy uses the image SHA as the chart version suffix, but the
  SHA collision occurred because the source didn't change (config-only build — latest
  tag was reused from GAR without a new image build).

Fix: Force a source change to generate a new image SHA, OR bump the chart version
  manually in devops-argo-config.

deployment_order check: 'catalog-service-stg' — verify this matches the app name
  in devops-argo-config (wrong name → silent sync failure at step 4, not step 3).

Relevant code:
  deployArgoCD.groovy:update_helm_repo() — pushes chart with SHA-derived version
  constructParam.groovy:getConfigOnlyChange() — sets SKIP_BUILD flag when only YAMLs changed
```

---

## Stage-to-file mapping (built-in knowledge)

| Jenkins stage name | devops-lib file |
|---|---|
| `Build` | `buildMaven.groovy` / `buildGo.groovy` / `buildNode.groovy` / etc. (by `build_tool`) |
| `Docker Build & Push` | `buildMaven.groovy:dockerBuildAndPush()` / equivalent in each build stage |
| `SonarQube Analysis` | `buildMaven.groovy:sonar_scan()` / `buildGo.groovy:sonar_scan()` |
| `SonarQube Quality Gate` | `constructParam.groovy:waitForQualityGate()` |
| `CAC Validation` | `buildMaven.groovy:cac_validation()` / `buildGo.groovy:cac_validation()` |
| `Deploying to ArgoCD` | `deployArgoCD.groovy` (4-step: update_argo_repo → refresh_app_of_apps → update_helm_repo → refresh_and_sync) |
| `Notify` | `notify.groovy` / `notifySlack.groovy` |
| `AppConfig Validation` | `constructParam.groovy:validateAppConfig()` |

---

## Failure mode taxonomy

The skill classifies every failure into one of these categories before diagnosis:

| Category | Signal in log | devops-lib location |
|---|---|---|
| Build compilation | `BUILD FAILURE` / `go build failed` | build<Lang>.groovy |
| Docker push to GAR | `denied` / `UNAUTHORIZED` on push | build<Lang>.groovy:dockerBuildAndPush() |
| Sonar threshold | `Quality Gate status: FAILED` | constructParam.groovy:waitForQualityGate() |
| Sonar project missing | `Project not found` | buildGo.groovy:sonar_scan() auto-create logic |
| ArgoCD step 1 (argo_repo update) | `git push failed` in argo-config repo | deployArgoCD.groovy:update_argo_repo() |
| ArgoCD step 2 (app-of-apps refresh) | `app not found` / `no Application object` | deployArgoCD.groovy:refresh_app_of_apps() — new service, ArgoCD app not yet created |
| ArgoCD step 3 (helm push) | `chart version already exists` | deployArgoCD.groovy:update_helm_repo() |
| ArgoCD step 4 (sync) | `OutOfSync` / `Helm values error` | deployArgoCD.groovy:refresh_and_sync() |
| Wrong deployment_order | Silent sync on wrong app name | config.yaml:deployment_order vs devops-argo-config |
| Multizone gate | `must be deployed via Ringmaster` | constructParam.groovy:isMultizoneEnabled() |

---

## Dependencies

- Full read access to devops-lib source (stage-to-file mapping, failure message strings)
- Jenkins log (pasted by engineer or fetched via Jenkins API)
- Optionally: `Meesho/whitelists` (to check if skip_sonar or multizone entries explain the failure)
- Optionally: `Meesho/devops-argo-config` (to validate `deployment_order` app names for ArgoCD failures)

---

## Design notes

- The skill must handle truncated logs (Jenkins UI often shows the last N lines). It should ask for the full log if the failure point is not visible.
- For hotfix builds: sonar and quality gate failures are expected to be suppressed; if they appear, the hotfix path was not taken — the skill should check the branch name pattern.
- The skill should distinguish between a devops-lib bug (code path is wrong) vs a configuration error (wrong value in config.yaml or whitelist) vs an external system error (GAR down, Sonar unreachable).

---

## Open questions

- Should it auto-fetch the Jenkins log via the Jenkins API if given a job URL?
- For ArgoCD step 4 failures (Helm values schema errors): should it parse the exact Helm error and cross-reference with the deployment.yaml schema?
- Should it suggest a rerun command or ArgoCD force-sync as a recovery action?