devops-lib-gcp/docs/architecture.md

<!--
Auto-generated by /meesho-init Phase 8.
Long-form architecture for devops-lib. CLAUDE.md links here but does not
duplicate the contents.
-->

# devops-lib — Architecture

A Jenkins shared library that every Meesho service repo consumes via
`@Library('devops-lib') _` in its `Jenkinsfile`. It encapsulates the
per-language build, the Helm/ArgoCD deploy ceremony, secret + key handling,
notification routing, and security-scan integration.

## 1. High-level shape

```
service Jenkinsfile                 (lives in service repo)
   │
   └─ buildPipeline { repo_name: ..., build_tool: ..., maintainer: ... }
                                                      │
                  ┌───────────────────────────────────┘
                  ▼
            vars/<entry>.groovy                       ← global step
                  │  delegates to
                  ▼
        src/com/meesho/stages/<stage>.groovy          ← stage object
                  │  uses
                  ▼
        src/com/meesho/utilities/<helper>.groovy
                  │  loads
                  ▼
        resources/com/meesho/<template>.{yaml,Dockerfile}
```

### Module boundaries

| Directory | Owns | Outside callers |
|---|---|---|
| `vars/` | Public Jenkins global steps (the "API") | Service Jenkinsfiles |
| `src/com/meesho/stages/` | Stage logic — build, deploy, notify, security scan | `vars/` |
| `src/com/meesho/utilities/` | Pure helpers — git, params, node-pool selection, templating | Stages |
| `resources/com/meesho/` | Per-language Dockerfile + Helm template + values.yaml | Stages (rendered into service workspace) |
| `resources/org/meesho/` | Jenkins agent pod templates (`*-pod.yaml`) | `vars/` (via `libraryResource`) |
| `resources/com/meesho/validate_configs.py` | Helm-values schema validator (monolithic Python) | Stages (invoked via `sh`) |

### Architectural philosophy signals

- **Convention over configuration.** Almost every per-service behaviour is
  encoded in `vars/<entry>.groovy` + the language switch in
  `buildObjHelper.groovy`. New tech stacks require new files, not new flags.
- **CPS-aware Groovy.** Helpers that touch non-serialisable Java APIs (regex
  engines, template engines) sit behind `@NonCPS` boundaries
  (`constructTemplate._construct`). Cross those boundaries inside a
  `parallel` block and you'll get a serialisation error in production.
- **Per-call freshness vs caching.** `getWhitelistedRepos()` re-clones
  `Meesho/whitelists` on every invocation (no caching). The five whitelist
  checks each clone independently. This is by design: a DevOps whitelist
  change must take effect on the *next* build without a library release.
- **Hard-coded `'Meesho'` org.** Many helpers embed the org as a literal —
  parametrising it has knock-on effects across every service Jenkinsfile.
- **Single Python file.** `validate_configs.py` (1207 lines) holds every
  CAC/values-schema rule; splitting it has been flagged
  ([BUGS_AND_IMPROVEMENTS_REPORT §10](../BUGS_AND_IMPROVEMENTS_REPORT.md)).

## 2. Entry points (vars/)

| `vars/<file>.groovy` | Used by | Notes |
|---|---|---|
| `buildPipeline.groovy` | The legacy entry — service `Jenkinsfile` calls `buildPipeline { ... }` | Pins to `node('slave02')` (hard-coded — flagged) |
| `eksCICD.groovy` | EKS-targeted services | Calls `commonCICDFlow()` |
| `gkeCICD.groovy` | GKE-targeted services | Mirror of `eksCICD` for GCP |
| `cloudFunctionCICD.groovy` | GCP cloud-function deploys | Currently a stub (`sh 'ls -al'; echo 'Hello World'` — flagged in BUGS report) |
| `onlyPushtoJfrog.groovy` | One-off artifact push without full pipeline | JDK 11 / JDK 21 selections fall through to JDK 8 (flagged) |
| `createEKSconfigs.groovy` | Bootstrap EKS pod-template configs | |
| `gcpMigration.groovy` | One-off migration helper | |
| `log.groovy` | `log.info` wrapper used across stages | Wraps `echo` — there is no real logger |
| `stageName.groovy` | Stage-name helper | |
| `buildDockerGroovyGke.groovy` | Docker build helper for GKE | |

## 3. Stage layer (src/com/meesho/stages/)

`buildObjHelper.groovy:run(String build_tool)` is the dispatch switch:

```
build_tool   →  Stage class
─────────────────────────────────
maven        →  buildMaven
maven-*      →  buildMaven
gradle       →  buildGradle
docker       →  buildDocker
python-*     →  buildPython
node-*       →  buildNode
go*          →  buildGo
php          →  buildPhp
<other>      →  defaultBuild   (silent no-op — flagged)
```

Other stage files (orthogonal to the build dispatch):

- `checkOut.groovy` — Jenkins SCM checkout. Contains a method named
  `chekoutSubmodule` (misspelled — flagged) at line 20.
- `deployArgoCD.groovy` — the deploy ceremony.
- `deployRingmaster.groovy` — Ringmaster / Turbo-Turtle callback.
- `deployJar.groovy` — non-container artifact deploy.
- `notify.groovy` — Slack notification + tracking-API callback. Reads
  `config.maintainer` and `config.notify_channel` (defaults to
  `ci-cd-status`).
- `securityScan.groovy` — invokes the in-house scanner. Hard-codes
  `final String url = '172.31.5.29:63232/scans'` (P0 violation — flagged).
- `automationTest.groovy` — integration-test trigger.
- `hotFix.groovy` — sets `env.hot_fix = true` to skip Sonar / quality gate.
- `helmGenerator.groovy` — renders Helm values; line 103 contains a typo
  (`catch (Exceptione)` — flagged).
- `multiBranchPipeline.groovy` — wires `GitHubSCMSource`.

### deployArgoCD step order (load-bearing)

```
update_argo_repo          (push new app definition)
   ↓
refresh_app_of_apps       (sync app-of-apps so the new Application object exists)
   ↓
update_helm_repo          (compute xms/xmx from memory_limit, push values)
   ↓
refresh_and_sync          (per-service sync)
```

Steps 2 and 4 are not interchangeable — see [tribal-knowledge §10](tribal-knowledge.md).

Canary enforcement (lines 408-430): if `priority_v2 ∈ {sp0, up0}` and
`envrn == 'prd'`, the deploy fails fast unless canary is properly
configured (`canary.enabled=true`, `skipAnalysis=false`,
`enableManualPromotion=true`). No whitelist or bypass.

JVM memory (lines 220-255): `xms = xmx = memory_limit * 0.5`. There is no
`memory_request * 0.75` formula, no 64m rounding (the older claim in
tribal-knowledge has been reconciled).

## 4. Utilities (src/com/meesho/utilities/)

| File | Owns |
|---|---|
| `constructParam.groovy` | The big one. Loads CAC config, sets `env.BU`, `env.GCPProject`, `env.GCPLBProject`, computes `cicd_environment`, runs the five whitelist gates (`skip-sonar`, `app-config-disabled`, `multizone-enabled`, `allowedNonDevelopPrDeployment`, `ValidateCacConfig`). |
| `gitActions.groovy` | Clone, fetch, status — hard-codes the `Meesho` GitHub org in clone URLs and PR-merge URLs. |
| `nodePoolSelection.groovy` | Maps `cicd_environment` → node-pool selector. `dev`/`ftr`/`stg` all collapse to `${BU}-shared`. |
| `constructTemplate.groovy` | `@NonCPS` template engine wrapper over `SimpleTemplateEngine`. |
| `addSSHKey.groovy` | Writes a Jenkins SSH credential to `./id_github_jenkins`. **Does not** cat the key to stdout (reconciled — the historical PR #634 leak has been remediated). |
| `getDockerParams.groovy` | Helper to assemble `docker run` bindings. |
| `getYamlParameter.groovy` | Reads a single key from a YAML file. |
| `validateBuTeam.groovy` | Cross-checks the BU against team ownership. |

## 5. Resources

### `resources/com/meesho/`

Templates rendered into the service workspace:

- Per-language `Dockerfile` (`java-Dockerfile`, `go-Dockerfile`,
  `node-Dockerfile`, `php-Dockerfile`, `python-{2.7,3.7,3.10.12,3.13}-Dockerfile`)
- Per-language Helm values (`go-values.yaml`, `node-values.yaml`,
  `python-values.yaml`, `php-values.yaml`, `values.yaml`, `cron-values.yaml`)
- Per-language deployment manifests (`deployment.yaml`, `go-deployment.yaml`,
  `node-deployment.yaml`, `php-deployment.yaml`, `python-deployment.yaml`,
  `gradle-deployment.yaml`)
- ArgoCD Application template (`argoApp.yaml`)
- `Dockerfile` and `Jenkinsfile` fallbacks (rare path)
- `config.yaml` — default service shape consumed by `constructParam`
- `validate_configs.py` / `validate_configs_v2.py` — the schema validator
  (v2 is the eventual replacement; both are referenced today)

### `resources/org/meesho/`

- `dev-pod.yaml`, `stg-pod.yaml`, `prd-pod.yaml` — Jenkins agent pod
  templates loaded via `libraryResource("org/meesho/${env.INFRA_ENV}-pod.yaml")`.
- `templates/maven-3.3-jdk-8.sh`, `templates/node-12.22.sh` — bootstrap
  shell scripts copied into the agent.

## 6. Downstream services

| Caller | Downstream | Endpoint / mechanism | Resilience |
|---|---|---|---|
| `deployArgoCD.groovy` | ArgoCD | `argocd login ${env.argoURL}:443 --grpc-web`, `argocd app sync`, `argocd app refresh` | `--http-retry-max 3 --retry-backoff-duration 1m` |
| `deployRingmaster.groovy` | Ringmaster *or* Turbo-Turtle | `POST http://turbo-turtle.meeshogcp.in/...` (chosen by `getUserId() == "ringmaster-bot"`) | None — direct `curl` |
| `notify.groovy` | Slack | `slackSend channel: ..., message: ...` | None |
| `notify.groovy` | Deployment Tracker | `POST https://deployment-tracker.meeshoint.in/...` (and `.prd.meesho.int`) | None |
| `buildMaven.groovy` / `buildNode.groovy` / `buildGo.groovy` | JFrog | `mvn deploy` / `npm publish` / artifact upload | Branch-gated: master, main, gcp-main, gcp-master |
| `buildMaven.groovy` / `buildGradle.groovy` etc. | S3 | `aws s3 cp …` | Same branch gate |
| `buildNode.groovy` | Docker registry | `docker push` | `retryDockerPush` retry wrapper |
| `buildMaven.groovy` etc. | SonarQube | `withSonarQubeEnv { … }` → `sonarqube-prd` | Skipped via `skip-sonar-whitelist.yaml` for Maven prd |
| `constructParam.groovy` | Vault | `vault-prd.meeshogcp.in`, `vault-dev.meeshogcp.in` | None |
| `securityScan.groovy` | In-house scanner | `POST http://172.31.5.29:63232/scans` (P0 — should be DNS, flagged) | None |

## 7. Critical invariants & gotchas

(Most are also enumerated in [docs/tribal-knowledge.md](tribal-knowledge.md);
this section captures the ones that change the *shape* of the code.)

1. **`config.yaml` is read once by `constructParam.run()`.** Everything
   downstream reads from `env.*` it set. Don't introduce a second config
   read; mutate `env.*` instead.
2. **`hot_fix` shortcuts.** `env.hot_fix = true` (set by `hotFix.groovy:11`)
   skips Sonar, quality gates, and several validation steps in
   `buildMaven` / `buildGo`. Use it deliberately, not as a "skip everything"
   knob.
3. **`branch_name = 'repo'`** appears as a hard-coded string in
   `buildGradle.groovy:252,285` (flagged in BUGS report). Do not assume
   `branch_name` is dynamic — the comparison against `'master'/'main'`
   never matches there.
4. **`rm -rf *`** appears in six locations across `buildPython`, `buildMaven`,
   `buildGradle` (flagged). Be aware of the working directory when adding
   stages near these — there is no directory guard.
5. **`generic catch (Exception e)`** is used ~87 times across stages
   (flagged). New code should prefer typed exceptions, but existing handlers
   suppress everything — be cautious assuming a stage "succeeded".

## 8. Configuration touch points

| Config | Where read | Drives |
|---|---|---|
| `config.bu` | `constructParam:178,255,308,318` | Helm chart path, ArgoCD namespace, GCP project name |
| `env.INFRA_ENV` | `eksCICD:57`, `createEKSconfigs:5`, `onlyPushtoJfrog:4` | Jenkins agent pod template |
| `env.CHANGE_ID` | `constructParam:107-110` | PR vs branch detection; `cicd_environment` remap (`prd→int` for main/master PRs, `stg→ftr` for develop PRs) |
| `env.hot_fix` | `buildGo:25-28`, `buildMaven:37-40`, `hotFix:11` | Skip Sonar / quality gate |
| `param.build_tool` | `buildObjHelper:run`, `vars/buildPipeline:12` | Stage class selection |
| `param.maintainer` | `notify:9,31` | Slack mention |
| `param.skip_test` | `buildMaven:17,206-208` | Maven `-DskipTests` |
| `param.skip_sonar` | `buildMaven:18,240-243` | Sonar bypass |
| `param.skip_security_scan` | `securityScan:7-9` | Security scan bypass |
| `param.skip_notify` | `notify:10,26-28` | Slack bypass |
| `param.notify_channel` | `notify:11` | Slack channel (default `ci-cd-status`) |
| `param.push_to_jfrog` | `buildMaven:19,377` | Allow non-default-branch JFrog push |
| `param.push_to_s3` | `buildMaven:20,462` | Allow non-default-branch S3 push |

## 9. What's missing (and known)

`BUGS_AND_IMPROVEMENTS_REPORT.md` is the authoritative catalogue. Highlights
that affect *how* you should approach changes:

- **No test suite** (P0). Don't fabricate test commands.
- **No retry on most stages** (P1) — only ArgoCD sync and Docker push retry.
- **Hard-coded IPs** (P0) — `securityScan.groovy:11` is a known violation
  pending remediation.
- **Inconsistent logging** — mix of `log.info()` (which is a thin wrapper)
  and bare `echo`. Prefer `log.info` for new code.
- **Commented-out blocks** — `buildPython:13-38`, `buildNode:27-31`,
  `buildGradle:470-477` carry large dead sections. Don't extend them; if a
  block is genuinely dead, delete it in a separate PR.

## See also

- [`docs/acronyms.md`](acronyms.md)
- [`docs/tribal-knowledge.md`](tribal-knowledge.md)
- [`../review-learnings.md`](../review-learnings.md)
- [`../BUGS_AND_IMPROVEMENTS_REPORT.md`](../BUGS_AND_IMPROVEMENTS_REPORT.md)