gitadmin/devops-lib-gcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added files

Browse Source
This commit is contained in:
Your Name
2026-08-26 02:02:24 +05:30
parent 58ee8a276a
commit 3419cfba0c
200 changed files with 22132 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/wiki/pages/01-ARCHITECTURE.md
+86
View File
@@ -0,0 +1,86 @@
<!-- m-wiki: type=top-level slug=architecture topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Architecture
devops-lib is a Jenkins Shared Library that implements the entire CI/CD pipeline for all Meesho microservices. Consumer services load it via `@Library('devops-lib@main')` and delegate their full build–deploy–notify lifecycle to it through a single `eksCICD(repo)` call.
## TL;DR
- One entry point (`vars/eksCICD.groovy`) routes to GCP (pod-based) or AWS (EKS node) infra via `CLOUD_PROVIDER`.
- A sequential flow: checkout → config parse → build stage → ArgoCD deploy → Ringmaster/Turbo-Turtle notify.
- Language-specific build logic lives in `src/com/meesho/stages/build*.groovy`; infra and policy env vars in `src/com/meesho/utilities/constructParam.groovy`.
- All deployments go through ArgoCD; no `kubectl apply` ever runs directly.
- Secrets and whitelists are injected at runtime — never hard-coded.
## Mental model
Think of `eksCICD` as a dispatcher: it doesn't contain any build or deploy logic itself. It (1) authenticates the triggering user, (2) selects the right infrastructure pod or node, and (3) hands off to `commonCICDFlow`, which assembles and runs the actual pipeline stages in sequence.
Each stage is a separate Groovy class with a `run(Map config)` method. `constructParam.run()` populates `env.*` variables (registry URL, vault endpoint, ArgoCD credentials, etc.) so every downstream stage can read a consistent environment without reconfiguring itself.
## Structure / data flow
```
Consumer Jenkinsfile
└─ @Library('devops-lib@main') → eksCICD(repo)
│
├─ [auth guard] allowedUsers check
│
├─ CLOUD_PROVIDER=GCP → gcpInfra() → podTemplate(INFRA_ENV-pod.yaml) → node(POD_LABEL)
│ └─ container('devops-tools') → commonCICDFlow(repo)
│
└─ CLOUD_PROVIDER=AWS → awsInfra() → node('EKS') → commonCICDFlow(repo)
│
▼
commonCICDFlow(repo)
├─ checkOut.run(repo) ← clone service + submodules
├─ getYamlParameter.getParam() ← parse config.yaml
├─ buildObjHelper.run(build_tool) ← dispatch to language builder
├─ constructParam.run(param) ← resolve env.* vars
├─ hotFix.run(repo_name) ← skip tests/sonar on hotfix/*
└─ buildObj.run(param) ← build + deploy + notify
│
├─ build (Maven/Go/Node/…)
├─ deployArgoCD.run()
└─ notify.run() → deployRingmaster.run()
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `call` | `vars/eksCICD.groovy:call` | Top-level entry — auth guard + infra routing |
| `gcpInfra` | `vars/eksCICD.groovy:gcpInfra` | Loads pod YAML from `resources/org/meesho/` and wraps in `podTemplate` |
| `awsInfra` | `vars/eksCICD.groovy:awsInfra` | Runs on static EKS node labelled `EKS` |
| `commonCICDFlow` | `vars/eksCICD.groovy:commonCICDFlow` | Orchestrates the full stage sequence |
| `run` | `src/com/meesho/stages/buildObjHelper.groovy:run` | Dispatches to language builder by `build_tool` |
| `run` | `src/com/meesho/utilities/constructParam.groovy:run` | Sets all `env.*` vars for build + deploy |
| `run` | `src/com/meesho/stages/deployArgoCD.groovy:run` | 4-step ArgoCD deploy per deployable |
| `run` | `src/com/meesho/stages/notify.groovy:run` | Slack + Ringmaster/Turbo-Turtle callback |
## Sharp edges
- **Authorization is strict**: only `ringmaster-bot`, `turbo-turtle`, and the `allowedUsers` list can trigger builds. Builds not in this list are rejected immediately with a message directing to Ringmaster.
- **`env.*` mutation is CPS-bound**: `constructParam.run()` sets `env.*` in a CPS method. Any utility that needs `@NonCPS` cannot read from `env.*` inside the annotation — use method parameters instead.
- **GCP vs AWS env diverge**: `constructParam.run()` has two separate `accountDetails` maps for GCP and AWS. Registry URL, bucket name, and ArgoCD cluster coordinates differ between clouds.
- **`useSidecar` flag**: services that need a sidecar container can pass `useSidecar: true` in the repo Map. This switches the pod template from `INFRA_ENV-pod.yaml` to `INFRA_ENV-sidecar-pod.yaml`.
## Related concepts
- [Build dispatch](build/build-dispatch.md) — how `buildObjHelper` matches `build_tool` strings
- [Environment mapping](05-ENVIRONMENT-MAPPING.md) — branch/PR → `cicd_environment` table
- [Config policy](06-CONFIG-POLICY.md) — `constructParam` env resolution + whitelist gates
- [Deploy ArgoCD](04-DEPLOY-ARGOCD.md) — 4-step deploy sequence
- [Infra pods](09-INFRA-PODS.md) — GCP pod spec selection
- [ADR index](adr/adr-index.md) — the "why" behind devops-lib's core design decisions
- [Security overview](security/security-overview.md) — trust boundaries and credential handling
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](01-ARCHITECTURE.md) · [Index](../index.md) · [Next →](02-ENTRYPOINTS.md)
docs/wiki/pages/01-overview.md
+57
View File
@@ -0,0 +1,57 @@
<!-- m-wiki: type=top-level slug=overview topic=null base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:README.md, code:vars/buildPipeline.groovy, code:src/com/meesho/stages/buildObjHelper.groovy, code:resources/com/meesho/config.yaml] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: top-level. 4 sources.
# Overview
**devops-lib** is the Jenkins shared library backing every Meesho service's CI/CD pipeline. Service repos import it via `@Library('devops-lib') _` at the top of their `Jenkinsfile`, then call one of the `vars/` globals (most commonly `buildPipeline { ... }` or `eksCICD { ... }`).
It is **not** a service, has no `Dockerfile`, no `Makefile`, no test suite, no local run command. The only "runtime" is Jenkins itself: a Jenkins controller loads the library, evaluates the Jenkinsfile, and executes stages on a pod template selected by `env.INFRA_ENV`.
## How a service repo consumes it
```groovy
// <service-repo>/Jenkinsfile
@Library('devops-lib') _
buildPipeline {
repo_name = 'order-service'
build_tool = 'maven' // or 'go', 'node', 'python', 'gradle', 'docker', 'php'
maintainer = 'roshan.v' // Slack handle for #ci-cd-status mentions
skip_test = false
skip_sonar = false
push_to_jfrog = false // default-branch-only push otherwise
deployArgo = true
}
```
The map flows into [`vars/buildPipeline.groovy`](../../../vars/buildPipeline.groovy) which:
1. Pins to `node('slave02')` ([line 16](../../../vars/buildPipeline.groovy)).
2. Sets `env.msg = 'Job Passed'` and wraps everything in `ansiColor` + `timestamps`.
3. Calls `checkOut` → `buildObjHelper.run(param.build_tool)` → the selected stage class's `run(param)` → `notify`.
## What lives where
| Path | Contains | Read by |
|---|---|---|
| [`vars/`](../../../vars/) | 10 entry-point Groovy scripts — the "public API" | Service Jenkinsfiles |
| [`src/com/meesho/stages/`](../../../src/com/meesho/stages/) | 19 stage classes (build, deploy, notify, scan) | `vars/` entry points |
| [`src/com/meesho/utilities/`](../../../src/com/meesho/utilities/) | 8 helpers — `constructParam`, `gitActions`, `nodePoolSelection`, `constructTemplate`, etc. | Stages |
| [`resources/com/meesho/`](../../../resources/com/meesho/) | Per-language Dockerfile + values.yaml + deployment.yaml; the Python validator | Stages (rendered into the service workspace) |
| [`resources/org/meesho/`](../../../resources/org/meesho/) | `{dev,stg,prd}-pod.yaml` Jenkins agent pod templates | `vars/` (via `libraryResource`) |
## Why "no local build / no test suite"
This is a Groovy library loaded by Jenkins, not a JVM app. There is no `build.gradle`, no `pom.xml`, no `package.json` at the repo root — Jenkins discovers `vars/` and `src/` by convention. The only way to "test" a change is to push the branch and point a Jenkins job at `@Library('devops-lib@<branch>')`.
`BUGS_AND_IMPROVEMENTS_REPORT.md` flags the absent test suite as a P0 gap.
## Critical conventions to know before changing anything
- `vars/` files are the public API. Renaming or removing a global is a breaking change for every service Jenkinsfile in the org.
- The `build_tool` switch is in [`src/com/meesho/stages/buildObjHelper.groovy`](../../../src/com/meesho/stages/buildObjHelper.groovy). Unknown values fall through to `defaultBuild` silently.
- The deploy step order (`update_argo_repo` → `refresh_app_of_apps` → `update_helm_repo` → `refresh_and_sync`) is load-bearing — see [04-deploy-flow](04-deploy-flow.md).
- `constructTemplate._construct()` is `@NonCPS` ([constructTemplate.groovy:13-20](../../../src/com/meesho/utilities/constructTemplate.groovy)) — do not call it across a `parallel` boundary.
See also: [02-entry-points](02-entry-points.md), [03-build-dispatch](03-build-dispatch.md), [04-deploy-flow](04-deploy-flow.md), [05-cross-cutting](05-cross-cutting.md), [`docs/tribal-knowledge.md`](../../tribal-knowledge.md), [`docs/acronyms.md`](../../acronyms.md).
docs/wiki/pages/02-ENTRYPOINTS.md
+72
View File
@@ -0,0 +1,72 @@
<!-- m-wiki: type=top-level slug=entrypoints topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Entry Points
The `vars/` directory contains every Groovy script that is callable from a consumer Jenkinsfile. Each file in `vars/` becomes a global function in the Jenkins pipeline namespace.
## TL;DR
- `eksCICD` is the primary entry point used by all microservices.
- `buildPipeline` is a legacy wrapper kept for backward compatibility.
- `gkeCICD`, `gcpMigration`, `cloudFunctionCICD` are specialised for specific infra targets.
- `createEKSconfigs` and `onlyPushtoJfrog` are utility entry points, not full CI/CD flows.
- `log`, `stageName`, `automationTest` are helper utilities exposed as global functions.
## Mental model
Any `.groovy` file placed in `vars/` is automatically loaded by Jenkins as a global variable/function. Consumer Jenkinsfiles call these directly: `eksCICD(repo)`, `buildPipeline(repo)`, etc. There is no package declaration in `vars/` files — they are scripts, not classes.
## Structure / data flow
```
vars/
├─ eksCICD.groovy Primary entry — all active microservices
├─ buildPipeline.groovy Legacy alias → wraps eksCICD
├─ gkeCICD.groovy GKE-specific pipeline (minimal stub)
├─ gcpMigration.groovy AWS→GCP migration helper
├─ cloudFunctionCICD.groovy Cloud Functions CI/CD
├─ createEKSconfigs.groovy EKS kubeconfig bootstrapper
├─ onlyPushtoJfrog.groovy Pushes JAR to JFrog without full pipeline
├─ automationTest.groovy Automation test runner entry
├─ buildDockerGroovyGke.groovy Docker build for GKE target
├─ log.groovy Global log.info/log.error/log.warn helpers
└─ stageName.groovy Returns stage name string for display
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `call` | `vars/eksCICD.groovy:call` | Main CI/CD flow — auth guard, infra routing |
| `call` | `vars/buildPipeline.groovy:18` | Legacy entry — delegates to eksCICD internals |
| `call` | `vars/gkeCICD.groovy:1` | GKE variant (thin stub) |
| `call` | `vars/cloudFunctionCICD.groovy:1` | Cloud Functions deployment flow |
| `call` | `vars/createEKSconfigs.groovy:1` | EKS kubeconfig bootstrap utility |
| `call` | `vars/log.groovy:1` | Global logging helper (info/error/warn) |
## Sharp edges
- **`vars/` scripts run in the Jenkins CPS interpreter.** Any non-serializable Java object (iterators, closures with complex state) passed from `vars/` to a `@NonCPS` method will throw `NotSerializableException` at runtime.
- **`buildPipeline` is legacy** — do not add new consumers to it. All new services should use `eksCICD`.
- **`log.groovy` shadows Jenkins' built-in `echo`** in some contexts — if you see unexpected log formatting, check whether `log.info` or `echo` was used.
- **`automationTest.groovy` is independent** — it does not go through `eksCICD`; automation test repos call it directly.
- **`scm` is only available in consumer Jenkinsfiles**: the `scm` variable (branch, remote URL, credentials) is injected by the GitHub Branch Source plugin into consumer Jenkinsfiles only — not into `vars/` or `src/` of the shared library. Referencing `scm.branches` in library code causes `MissingPropertyException` at runtime. Use `env.BRANCH_NAME`, `env.GIT_URL`, or `env.CHANGE_*` instead.
See also: [SCM variable scope](build/scm-variable-scope.md)
## Related concepts
- [Architecture](01-ARCHITECTURE.md) — how eksCICD orchestrates the pipeline
- [Build stages](03-BUILD-STAGES.md) — what runs after the entry point selects infra
- [Environment mapping](05-ENVIRONMENT-MAPPING.md) — branch-to-env resolution
- [SCM variable scope](build/scm-variable-scope.md) — why `scm` is unavailable in library code
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](01-ARCHITECTURE.md) · [Index](../index.md) · [Next →](03-BUILD-STAGES.md)
docs/wiki/pages/02-entry-points.md
+40
View File
@@ -0,0 +1,40 @@
<!-- m-wiki: type=top-level slug=entry-points topic=null base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:vars/buildPipeline.groovy, code:vars/eksCICD.groovy, code:vars/gkeCICD.groovy, code:vars/onlyPushtoJfrog.groovy, code:vars/createEKSconfigs.groovy, code:vars/cloudFunctionCICD.groovy, code:vars/log.groovy, code:vars/stageName.groovy] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: top-level. 8 sources.
# Entry points (vars/)
Every file under [`vars/`](../../../vars/) is automatically exposed by Jenkins as a global step bearing the file's name (no `.groovy` suffix). Service Jenkinsfiles call them like top-level functions; renaming a file is a breaking change for every consumer.
| File | Call signature | What it does |
|---|---|---|
| [`buildPipeline.groovy`](../../../vars/buildPipeline.groovy) | `def call(Map param)` | Legacy primary entry. Pins to `node('slave02')` ([line 16](../../../vars/buildPipeline.groovy)), runs checkout → buildObjHelper dispatch → optional automation tests → notify. |
| [`eksCICD.groovy`](../../../vars/eksCICD.groovy) | `def call(Map repo)` | Authorized-user gate ([lines 12-26](../../../vars/eksCICD.groovy) — explicit allowedUsers list), then routes GCP → pod template OR AWS → EKS node selector, then `commonCICDFlow()`. Used by most modern services. |
| [`gkeCICD.groovy`](../../../vars/gkeCICD.groovy) | `def Podcall(Map stepParams)` | Sparse legacy entry — Maven build + Docker push with hardcoded pod template. New services should not use this. |
| [`createEKSconfigs.groovy`](../../../vars/createEKSconfigs.groovy) | `def call(Map params)` | EKS Helm config generator. Validates BU/team and renders config files. Pod from `libraryResource("org/meesho/${env.INFRA_ENV}-pod.yaml")` ([line 5](../../../vars/createEKSconfigs.groovy)). |
| [`onlyPushtoJfrog.groovy`](../../../vars/onlyPushtoJfrog.groovy) | declarative pipeline (no `call()`) | One-off artifact push without the full pipeline. Choice parameter declares `['jdk8', 'jdk11', 'jdk17', 'jdk21']` ([line 16](../../../vars/onlyPushtoJfrog.groovy)) but the JAVA_HOME switch ([lines 48-53](../../../vars/onlyPushtoJfrog.groovy)) only handles `jdk17` explicitly — jdk11 and jdk21 fall through to the jdk8 default. ⚠ flagged in BUGS report. |
| [`gcpMigration.groovy`](../../../vars/gcpMigration.groovy) | `def call(Map params)` | One-off migration helper that emits a Jenkinsfile + Helm chart for a GCP-main branch. |
| [`buildDockerGroovyGke.groovy`](../../../vars/buildDockerGroovyGke.groovy) | `def run(Map config)` | Multi-module Docker build helper for GKE with artifact-version detection. |
| [`cloudFunctionCICD.groovy`](../../../vars/cloudFunctionCICD.groovy) | `def cloudFunctionCICDFlow()` | **Skeleton only.** Body is `sh 'ls -al'; echo 'Hello World'` ([lines 19-30](../../../vars/cloudFunctionCICD.groovy)) — flagged in BUGS report as not-yet-implemented. |
| [`log.groovy`](../../../vars/log.groovy) | `def info/warning/error(msg)` | ANSI-colour logging wrapper. See [concepts/observability](concepts/observability.md). |
| [`stageName.groovy`](../../../vars/stageName.groovy) | `def call(String description)` | Labels stages with environment + step counter. Reads `cicd_environment` → `INFRA_ENV` → `BUILD_ENV` → `ENVIRONMENT` (default `ftr`). |
## The user-authorization gate (eksCICD only)
[`eksCICD.groovy:12-26`](../../../vars/eksCICD.groovy) holds an explicit allowlist of users (`'turbo-turtle'`, `'ringmaster-bot'`, and a handful of named engineers). Builds triggered by anyone outside that list are **failed with a hard `error()`** and a banner directing them to Ringmaster. `buildPipeline` does **not** enforce this gate — services on the legacy entry point can be triggered directly.
## Pod template wiring
`env.INFRA_ENV` selects the Jenkins agent template via `libraryResource("org/meesho/${env.INFRA_ENV}-pod.yaml")`. Used in [`eksCICD.groovy:57`](../../../vars/eksCICD.groovy), [`createEKSconfigs.groovy:5`](../../../vars/createEKSconfigs.groovy), and [`onlyPushtoJfrog.groovy:4`](../../../vars/onlyPushtoJfrog.groovy). The resolution table:
| `env.INFRA_ENV` | Resource | Use |
|---|---|---|
| `dev` | [`resources/org/meesho/dev-pod.yaml`](../../../resources/org/meesho/dev-pod.yaml) | Dev / sandbox |
| `stg` | [`resources/org/meesho/stg-pod.yaml`](../../../resources/org/meesho/stg-pod.yaml) | Staging |
| `prd` | [`resources/org/meesho/prd-pod.yaml`](../../../resources/org/meesho/prd-pod.yaml) | Production |
All three are single-container pods (`devops-tools` from the `asia-southeast1-docker.pkg.dev/meesho-devops-admin-0622/admin/devops/build-tools:lunar-v1.0.1*` image), with `nodeSelector: {dedicated: jenkins}` and the matching toleration.
Don't add inline `podTemplate` blocks in stage code — they bypass the central agent inventory.
See also: [03-build-dispatch](03-build-dispatch.md), [`docs/architecture.md`](../../architecture.md).
docs/wiki/pages/03-BUILD-STAGES.md
+78
View File
@@ -0,0 +1,78 @@
<!-- m-wiki: type=top-level slug=build-stages topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Build Stages
Build logic lives in `src/com/meesho/stages/`. `buildObjHelper.groovy` is the dispatch router — it inspects the `build_tool` string from `config.yaml` and returns the appropriate builder class instance.
## TL;DR
- `buildObjHelper.run(build_tool)` returns an object with a `.run(param)` method.
- Matching uses Groovy's `switch/case` with regex patterns (e.g., `~/^go.*/`).
- Eight language builders exist: Maven, Go, Gradle, Node, Python, PHP, Rust, Docker.
- Each builder follows the same interface: `def run(Map config)`.
- Config-only change detection runs inside each builder before invoking the compiler/test runner — if only YAML files changed, the build step is skipped.
## Mental model
`buildObjHelper` is a factory. It reads one string (`build_tool`) and returns an object. The caller (`commonCICDFlow`) doesn't need to know which concrete class was returned — it just calls `.run(param)`. This is an interface-based dispatch pattern in Groovy without formal interfaces.
Each builder is responsible for the complete build lifecycle of its language: compile, test, Sonar scan, Docker image build, and pushing to the registry. After building, each builder typically calls `deployArgoCD.run()` or similar to kick off the deploy phase.
## Structure / data flow
```
config.yaml: build_tool: "go-1.22"
│
▼
buildObjHelper.run("go-1.22")
switch "go-1.22":
case ~/^go.*/ → return new buildGo()
│
▼
buildGo.buildDckr(param)
├─ [config-only check] git diff configs/ → skip build if no src change
├─ go build / go test
├─ sonar scan (constructParam.skipSonarCheckForGo)
├─ docker build + push (constructTemplate + dockerUtilities)
└─ deployArgoCD.run(...) or skip if deployArgo=false
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `run` | `src/com/meesho/stages/buildObjHelper.groovy:run` | Regex dispatch to builder class |
| `run` | `src/com/meesho/stages/buildMaven.groovy:run` | Maven/Java builder |
| `buildDckr` | `src/com/meesho/stages/buildGo.groovy:buildDckr` | Go builder |
| `buildDckr` | `src/com/meesho/stages/buildNode.groovy:buildDckr` | Node.js builder |
| `buildDckr` | `src/com/meesho/stages/buildPython.groovy:buildDckr` | Python builder |
| `run` | `src/com/meesho/stages/buildGradle.groovy:run` | Gradle/Java builder |
| `buildDckr` | `src/com/meesho/stages/buildPhp.groovy:buildDckr` | PHP builder |
| `buildDckr` | `src/com/meesho/stages/buildRust.groovy:buildDckr` | Rust builder |
| `run` | `src/com/meesho/stages/buildDocker.groovy:run` | Docker-only builder |
| `run` | `src/com/meesho/stages/hotFix.groovy:run` | Sets hot_fix flag — skips tests/sonar |
| `run` | `src/com/meesho/stages/checkOut.groovy:run` | Git checkout stage |
## Sharp edges
- **The dispatch is regex-based, not exact-match for most languages.** `~/^go.*/` matches `go`, `go-1.22`, `go-1.21`, etc. Typos in `config.yaml` silently fall through to `defaultBuild()` — there is no explicit error for unrecognized `build_tool` values. A mis-spelled `build_tool` produces a silent no-op build that succeeds with no artifact.
- **Go and Node entry points are `buildDckr`, not `run`**: `buildGo.groovy` and `buildNode.groovy` expose `buildDckr(Map config)` as their primary method. `buildMaven.groovy`, `buildGradle.groovy`, and `buildPython.groovy` use `run(Map config)`. This asymmetry is historical.
- **`hotFix.run()` runs before `buildObj.run()`** in `commonCICDFlow`. It sets `env.hot_fix = true` for hotfix/* branches, which causes each builder to skip tests and Sonar.
- **`buildPython` has multiple variants.** `python-3.10`, `python-3.12`, etc. all match `~/^python-.*/` and resolve to the same `buildPython` class, which reads `dockerBuildVersion` internally to select the right base image.
## Related concepts
- [Build dispatch](build/build-dispatch.md) — detailed regex pattern table for buildObjHelper
- [Config-only detection](build/config-only-detection.md) — how builders skip builds when only YAML changed
- [Docker tagging](build/docker-tagging.md) — `getDockerParams.getTag()` tag format
- [Dockerfile templates](08-DOCKERFILE-TEMPLATES.md) — how builder selects and renders the right Dockerfile
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](02-ENTRYPOINTS.md) · [Index](../index.md) · [Next →](04-DEPLOY-ARGOCD.md)
docs/wiki/pages/03-build-dispatch.md
+66
View File
@@ -0,0 +1,66 @@
<!-- m-wiki: type=top-level slug=build-dispatch topic=null base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:src/com/meesho/stages/buildObjHelper.groovy, code:src/com/meesho/stages/buildMaven.groovy, code:src/com/meesho/stages/buildGo.groovy, code:src/com/meesho/stages/buildNode.groovy, code:src/com/meesho/stages/buildPython.groovy, code:src/com/meesho/stages/buildGradle.groovy] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: top-level. 6 sources.
# Build dispatch
[`buildObjHelper.groovy`](../../../src/com/meesho/stages/buildObjHelper.groovy) is the central switch from the `build_tool` parameter (set in the service Jenkinsfile) to the stage class that actually runs. `vars/buildPipeline.groovy:12` calls `buildObjHelper.run(param.build_tool)` and assigns the return to `buildObj`; everything from then on flows through that object.
## The switch (buildObjHelper.groovy:5-30)
| `build_tool` value | Stage class | Source |
|---|---|---|
| `maven` | `buildMaven` | [`src/com/meesho/stages/buildMaven.groovy`](../../../src/com/meesho/stages/buildMaven.groovy) |
| `maven-*` (e.g. `maven-3.3-jdk-17`) | `buildMaven` | same |
| `docker` | `buildDocker` | [`src/com/meesho/stages/buildDocker.groovy`](../../../src/com/meesho/stages/buildDocker.groovy) |
| `python-*` (e.g. `python-3.10`) | `buildPython` | [`src/com/meesho/stages/buildPython.groovy`](../../../src/com/meesho/stages/buildPython.groovy) |
| `node-*` (e.g. `node-16`) | `buildNode` | [`src/com/meesho/stages/buildNode.groovy`](../../../src/com/meesho/stages/buildNode.groovy) |
| `go*` (e.g. `go1.21`) | `buildGo` | [`src/com/meesho/stages/buildGo.groovy`](../../../src/com/meesho/stages/buildGo.groovy) |
| `gradle` | `buildGradle` | [`src/com/meesho/stages/buildGradle.groovy`](../../../src/com/meesho/stages/buildGradle.groovy) |
| `php` | `buildPhp` | [`src/com/meesho/stages/buildPhp.groovy`](../../../src/com/meesho/stages/buildPhp.groovy) |
| any other value | `defaultBuild` (silent no-op) | (default case at line 28) |
There is **no `sbt`** case and **no `rust`** case, despite what older docs may have implied (both have been reconciled out — see [`review-learnings.md`](../../../review-learnings.md) for the audit trail).
## Per-language quirks worth knowing
### `buildMaven`
- CAC validation runs before build ([`buildMaven.groovy:186-201`](../../../src/com/meesho/stages/buildMaven.groovy)).
- Sonar scan triggered with `withSonarQubeEnv` ([line 245](../../../src/com/meesho/stages/buildMaven.groovy)) against `sonarqube-prd` via [`constructParam.groovy:172`](../../../src/com/meesho/utilities/constructParam.groovy).
- Quality-gate timeout: 600s on prd, 360s elsewhere ([lines 286-296](../../../src/com/meesho/stages/buildMaven.groovy)).
- JFrog push gated on branch ∈ `{master, main}` OR `push_to_jfrog=true` ([lines 377, 410](../../../src/com/meesho/stages/buildMaven.groovy)). S3 push gated on branch ∈ `{master, main, gcp-main, gcp-master}` OR `push_to_s3=true` ([lines 462, 497](../../../src/com/meesho/stages/buildMaven.groovy)).
### `buildGo`
- `sonar_scan()` ([lines 212-270](../../../src/com/meesho/stages/buildGo.groovy)) downloads the Go binary AND the sonar-scanner zip via `curl` inside the build stage itself ([lines 227, 236](../../../src/com/meesho/stages/buildGo.groovy)) — there is no fileExists check on `sonar-project.properties` before the scan ([line 220](../../../src/com/meesho/stages/buildGo.groovy)).
- Go test failures are caught and logged but **do not propagate** as pipeline failure ([lines 232-235](../../../src/com/meesho/stages/buildGo.groovy)): `echo "Go tests failed, but the pipeline will continue."`.
- `env.hot_fix = true` (set by [`hotFix.groovy:11`](../../../src/com/meesho/stages/hotFix.groovy)) skips Sonar AND the quality gate ([lines 25-28](../../../src/com/meesho/stages/buildGo.groovy)).
- Multi-module builds run in `parallel { }` ([lines 161-179](../../../src/com/meesho/stages/buildGo.groovy)).
- Docker push wrapped by `retryDockerPush` (5 attempts, 3s sleep) — see [05-cross-cutting](05-cross-cutting.md).
### `buildNode`
- `getNpmRc` ([lines 63-68](../../../src/com/meesho/stages/buildNode.groovy)) fetches `MEESHO_NPMRC_SECRET` from AWS Secrets Manager or Vault and writes it to `.npmrc` in the workspace.
- Inline `.env` writes for `GITHUB_TOKEN` and `SONAR` credentials ([lines 283-312](../../../src/com/meesho/stages/buildNode.groovy)) — note: there is **no** `truncate -s 0 .env` cleanup pattern; that was a historical PR concern that has been resolved.
- `DOCKER_BUILDKIT=0` is set explicitly ([lines 387, 392, 400, 405](../../../src/com/meesho/stages/buildNode.groovy)) — BuildKit is **disabled**, not enabled. The `BUILDKIT` acronym entry in [`docs/acronyms.md`](../../acronyms.md) was reconciled accordingly.
- Docker push via `retryDockerPush` ([line 389](../../../src/com/meesho/stages/buildNode.groovy)).
### `buildPython`
- Docker build with module support ([lines 89-210](../../../src/com/meesho/stages/buildPython.groovy)) — no Sonar scan (unlike Maven / Go / Node).
- Duplicate-key bug in `docker_bindings` map at [lines 75 and 78](../../../src/com/meesho/stages/buildPython.groovy) — second assignment overwrites the first. Flagged in BUGS report.
### `buildGradle`
- `branch_name = 'repo'` hard-coded at [lines 252 and 285](../../../src/com/meesho/stages/buildGradle.groovy) — the subsequent comparison against `'master'/'main'` therefore never matches the real branch. Flagged in BUGS report; do not assume `branch_name` is dynamic there.
- Uses `xq` to query `build.gradle` ([lines 255-256, 260, 288-289, 293](../../../src/com/meesho/stages/buildGradle.groovy)) — `xq` is an XML query tool and Gradle files are Groovy/Kotlin DSL, not XML. Flagged.
## Adding a new build_tool
1. Create `src/com/meesho/stages/build<Lang>.groovy` with a `run(Map param)` method.
2. Add a `case` in [`buildObjHelper.groovy`](../../../src/com/meesho/stages/buildObjHelper.groovy) — either a literal (`case 'rust'`) or a regex (`case ~/^rust-.*/`) for versioned variants.
3. Add `resources/com/meesho/<lang>-Dockerfile`, `<lang>-values.yaml`, `<lang>-deployment.yaml`.
4. There is no unit test to add — push the branch and validate via `@Library('devops-lib@<branch>')` in a sandbox service Jenkinsfile.
See also: [04-deploy-flow](04-deploy-flow.md), [05-cross-cutting](05-cross-cutting.md), [`docs/architecture.md`](../../architecture.md).
docs/wiki/pages/04-DEPLOY-ARGOCD.md
+88
View File
@@ -0,0 +1,88 @@
<!-- m-wiki: type=top-level slug=deploy-argocd topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Deploy via ArgoCD
All Meesho microservice deployments go through ArgoCD. `deployArgoCD.groovy` orchestrates a 4-step sequence per deployable: it first commits an ArgoCD Application manifest to `devops-argo-config`, then triggers a Helm values update in `devops-helm-charts`, and finally hard-refreshes and syncs the ArgoCD application.
## TL;DR
- 4 steps per deployable: `update_argo_repo` → `refresh_app_of_apps` → `update_helm_repo` → `refresh_and_sync`.
- Every step commits to a separate Git repo (argo-config or helm-charts), opens a PR, merges it, and deletes the branch.
- The deploy waits for user input (checkbox UI) to choose which deployables to deploy, with a 300s timeout.
- Multizone deployables are **blocked** — they must use Ringmaster.
- JVM heap (`xms`/`xmx`) defaults to 50% of `memory_limit`, overridable via `deployment_args`.
## Mental model
`deployArgoCD` treats Git as the deployment API. Every config change becomes a PR in `devops-argo-config` or `devops-helm-charts`, which ArgoCD polls and syncs. The pipeline commits the changes atomically for one deployable at a time in a `for` loop.
The user input step allows partial deploys — you can select "All" or individual apps from the checkbox list. If you skip input (timeout or `skip_user_input=true`), all apps in `deployment_order` are deployed.
## Structure / data flow
```
deployArgoCD.run(repo_name, deployment_order, tag, ...)
│
├─ [whitelist check] constructParam.allowedNonDevelopPrDeploymentToIntRepos()
├─ [user input] wait_for_user_input(deployment_order) — 300s timeout
│
└─ for each deployable:
├─ constructParam.isMultizoneEnabled(deployment) → ERROR if true
├─ constructParam.perDeploymentVars(value_binding) ← sets env.argoURL, env.argoIncubator, etc.
│
├─ stage: update_argo_repo()
│ ├─ render argoApp.yaml template → devops-argo-config/applications_v2/<cluster>/<team>-<app>.yaml
│ └─ commit → PR → merge → delete branch
│
├─ stage: refresh_app_of_apps()
│ └─ argocd app sync <argoIncubator>
│
├─ stage: update_helm_repo()
│ ├─ render values.yaml template → devops-helm-charts/<env>/bu/team/app/values.yaml
│ ├─ [canary enforcement] enforce skipAnalysis=false for sp0/up0 services
│ ├─ [dependabot check] block if CRITICAL CVEs on prd sp0-sp1 deploys
│ └─ commit → PR → merge → delete branch
│
└─ stage: refresh_and_sync()
├─ argocd app get --hard-refresh <env>-<app_name>
└─ argocd app sync <env>-<app_name>
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `run` | `src/com/meesho/stages/deployArgoCD.groovy:run` | Main entry — user input + per-deployable loop |
| `update_argo_repo` | `src/com/meesho/stages/deployArgoCD.groovy:update_argo_repo` | Renders ArgoCD Application YAML and commits to argo-config |
| `refresh_app_of_apps` | `src/com/meesho/stages/deployArgoCD.groovy:refresh_app_of_apps` | Syncs the incubator app-of-apps |
| `update_helm_repo` | `src/com/meesho/stages/deployArgoCD.groovy:update_helm_repo` | Renders Helm values and commits to helm-charts |
| `refresh_and_sync` | `src/com/meesho/stages/deployArgoCD.groovy:refresh_and_sync` | Hard-refreshes and syncs the ArgoCD app |
| `enable_backward_compatibility` | `src/com/meesho/stages/deployArgoCD.groovy:enable_backward_compatibility` | Fills in missing deployment.yaml keys with defaults |
| `dependabotCriticalCheck` | `src/com/meesho/stages/deployArgoCD.groovy:dependabotCriticalCheck` | Blocks deploy if CRITICAL CVEs found |
| `calculate_active_processors` | `src/com/meesho/stages/deployArgoCD.groovy:calculate_active_processors` | Converts cpu_request string to JVM -XX:ActiveProcessorCount |
## Sharp edges
- **Branch naming is environment-derived**: `helm_branch_name` maps `main/master→main`, `develop→develop`, PR→`feature` or `pre-prod`. This is independent of `cicd_environment` — confusion between the two causes PR target mismatches.
- **Feature deployments use ingress namespacing**: for `envrn=ftr`, Helm values go into `values_properties.yaml` under `<ingress_val>/` subdirectory, and the app branch is prefixed with `<ingress_val>-`.
- **AppConfig gate**: `appConfigDisabledForbidden()` blocks stg deploys for Maven/Gradle repos that have `appConfigEnabled=false` and are not in the `app-config-disabled` whitelist.
- **xms/xmx auto-calculation**: for Maven/Gradle, heap is set to 50% of `memory_limit`. This can be overridden by `Xms<val>` or `Xmx<val>` tokens in `deployment_args`. If a service hard-codes `-Xmx` in `JAVA_OPTS`, the auto-calculated value will collide; use `jvm_memory_override: true` in `deployment.yaml` to suppress auto-calc.
- **Canary is mandatory for sp0/up0**: `deployArgoCD.groovy:run` (`src/com/meesho/stages/deployArgoCD.groovy:run`) blocks a non-canary `prd` deploy when `priority_v2` is `sp0` or `up0`. There is no whitelist or flag to bypass this check — it happens before any Helm update.
- **The 4-step order is load-bearing**: steps 2 (`refresh_app_of_apps`) and 4 (`refresh_and_sync`) are not interchangeable. Skipping step 2 on a first deploy means the ArgoCD Application object hasn't been created yet, causing step 4 to target a non-existent app.
## Related concepts
- [ArgoCD sync](deploy/argocd-sync.md) — detailed step-by-step sync sequence
- [Ringmaster integration](deploy/ringmaster-integration.md) — when deployRingmaster runs instead
- [Whitelist system](policy/whitelist-system.md) — multizone + allowedNonDevelop gates
- [Node pool selection](infra/node-pool-selection.md) — how nodeSelectorValue is computed
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](03-BUILD-STAGES.md) · [Index](../index.md) · [Next →](05-ENVIRONMENT-MAPPING.md)
docs/wiki/pages/04-deploy-flow.md
+87
View File
@@ -0,0 +1,87 @@
<!-- m-wiki: type=top-level slug=deploy-flow topic=null base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:src/com/meesho/stages/deployArgoCD.groovy, code:src/com/meesho/stages/deployRingmaster.groovy, code:src/com/meesho/stages/notify.groovy, code:src/com/meesho/stages/helmGenerator.groovy] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: top-level. 4 sources.
# Deploy flow
The deploy phase is owned by [`deployArgoCD.groovy`](../../../src/com/meesho/stages/deployArgoCD.groovy) (the ArgoCD ceremony) and [`deployRingmaster.groovy`](../../../src/com/meesho/stages/deployRingmaster.groovy) (the callback to the higher-level deploy controllers). Two things to never mess with: the four-step order, and the canary gate.
## The four-step ArgoCD ceremony — load-bearing
`deployArgoCD.run()` ([lines 73-107](../../../src/com/meesho/stages/deployArgoCD.groovy)) calls these in strict order:
```
1. update_argo_repo (line 91) pushes the ArgoApplication YAML into devops-argo-config
so the app-of-apps registry sees the new app.
2. refresh_app_of_apps (line 94) argocd app sync ${appofapps} — materialises the new
Application object before the per-service sync needs it.
3. update_helm_repo (line 97) computes xms/xmx, renders values.yaml, pushes into
devops-helm-charts.
4. refresh_and_sync (line 100) argocd app sync ${app_name} --hard-refresh — the
actual service rollout. Uses --http-retry-max 3
--retry-backoff-duration 1m (lines 506, 526).
```
Steps 2 and 4 are **not interchangeable**. On first-deploy of a brand-new service, the Application object doesn't exist yet — step 2 creates it (as a downstream effect of the app-of-apps sync), step 4 reads it. Swap the order and step 4 fails on a missing Application. See [`docs/tribal-knowledge.md`](../../tribal-knowledge.md) §10 for the post-incident note.
## JVM memory auto-calculation
`update_helm_repo` ([lines 220-255](../../../src/com/meesho/stages/deployArgoCD.groovy)) derives JVM memory flags from the pod's `memory_limit`:
```groovy
memory_value = memory_limit * 0.5
xms = "${memory_value}M"
xmx = "${memory_value}M"
```
Both flags are set equal, derived from `memory_limit` (not `memory_request`), with no 0.75 multiplier and no 64m rounding. The historical claim "`xmx = memory_request * 0.75, xms = xmx * 0.5`" was wrong and has been reconciled out of [`docs/tribal-knowledge.md`](../../tribal-knowledge.md).
**Do not hard-code `-Xmx` in `JAVA_OPTS`** — the auto-computed value will collide with it, and the last value seen by the JVM wins depending on arg order. There is no `jvm_memory_override` flag (that referenced flag does not exist in code).
## Canary enforcement — sp0 / up0 in prd
[`deployArgoCD.groovy:408-430`](../../../src/com/meesho/stages/deployArgoCD.groovy):
```
enforceCanary = (priority_v2 ∈ {sp0, up0})
∧ (envrn == 'prd')
∧ ¬(service is canary | cron | worker | scheduler | consumer | node | headless)
```
When `enforceCanary` is true, the deploy hard-errors unless the Helm values declare:
- `canary.enabled = true`
- `canary.skipAnalysis = false`
- `canary.enableManualPromotion = true`
There is **no whitelist** and **no bypass flag**. A service that needs to skip canary on a high-priority prd path has to either (a) be classified out of `sp0/up0`, or (b) match one of the exempted service types listed above.
## Ringmaster vs Turbo-Turtle routing
[`deployRingmaster.groovy:55-89`](../../../src/com/meesho/stages/deployRingmaster.groovy):
```groovy
def build_user = currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
if (build_user == "ringmaster-bot") {
callApi(url, header, jsonData) // → ringmaster endpoint
} else {
// → http://turbo-turtle.meeshogcp.in (line 71)
// http://turbo-turtle.admin.meeshogcp.in (line 74)
sh "curl -s -X POST -H '$newCICD_Header' -w '\\n%{response_code}' $newCICD_URL -d '$newCICD_JSON'"
}
```
The string `"ringmaster-bot"` is a **load-bearing constant**. Renaming the bot user silently routes every Ringmaster callback to Turbo-Turtle, which rejects them.
The JSON payload is passed **inline** via `-d '$newCICD_JSON'` ([line 80](../../../src/com/meesho/stages/deployRingmaster.groovy)). Earlier documentation claimed a temp-file + `curl -d @<file>` pattern with `finally`-block cleanup — that pattern does **not** exist in the current code and was reconciled out of `docs/tribal-knowledge.md`.
## Deployment-tracker callback (notify.groovy)
[`notify.groovy:108-152`](../../../src/com/meesho/stages/notify.groovy) POSTs to one of:
- `http://deployment-tracker.meeshoint.in/api/1.0/deployment-tracker/jenkins/create` (AWS)
- `http://deployment-tracker.prd.meesho.int/api/1.0/deployment-tracker/jenkins/create` (GCP)
Only triggered on branches `main` / `master` / `gcp-main` / `gcp-master` ([line 34](../../../src/com/meesho/stages/notify.groovy)). Payload includes repository, team, link, job_name, tag, status, commit_id, error_msg. No retry — a single failed POST means the dashboard miscounts that deploy.
See also: [05-cross-cutting](05-cross-cutting.md), [concepts/whitelists](concepts/whitelists.md), [`docs/tribal-knowledge.md`](../../tribal-knowledge.md).
docs/wiki/pages/05-ENVIRONMENT-MAPPING.md
+70
View File
@@ -0,0 +1,70 @@
<!-- m-wiki: type=top-level slug=environment-mapping topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Environment Mapping
`cicd_environment` is the single string that controls which registry, vault, ArgoCD cluster, and GCS bucket the pipeline uses. It is derived from the branch name (for push builds) or the PR target branch (for PR builds).
## TL;DR
- Push to `main`/`master`/`gcp-main` → `prd`.
- Push to `develop` → `stg`.
- PR targeting `main` → `int` (pre-prod).
- PR targeting `develop` → `ftr` (feature).
- Any other branch → `ftr` (fallback).
- Hotfix branches skip tests and Sonar but still map to `prd`.
- `env.CHANGE_ID` being set signals a PR build and flips the mapping table.
## Mental model
`constructParam.run()` switches on `env.CHANGE_ID` to decide which `environment_map` to use. Without `CHANGE_ID` (push build), branches map to their canonical environments. With `CHANGE_ID` (PR build), the same branch names map to the pre-prod equivalents: `main→int`, `develop→ftr`. This double-mapping is why `develop` can be either `stg` (push) or `ftr` (PR).
## Structure / data flow
```
env.CHANGE_ID not set (push build):
environment_map = {
master/main/gcp-main/farmiso-main/gcp-master → prd
develop/gcp-dev → stg
}
branch not in map → ftr (default)
env.CHANGE_ID set (PR build):
branch_name = env.CHANGE_TARGET
environment_map = {
master/main/gcp-main/farmiso-main/gcp-master → int
develop/gcp-dev → ftr
}
env.cicd_environment = environment_map[branch_name]
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `run` | `src/com/meesho/utilities/constructParam.groovy:run` | Sets `env.cicd_environment` + all env vars |
| `run` | `src/com/meesho/stages/deployArgoCD.groovy:run` | Has its own `branch_param_map` for helm/argo branch naming |
| `run` | `src/com/meesho/stages/hotFix.groovy:run` | Detects `hotfix/*` branch and sets `env.hot_fix=true` |
## Sharp edges
- **Two separate maps exist**: `constructParam` has one map for `cicd_environment`; `deployArgoCD` has a different `branch_param_map` for the Helm/Argo Git branch names (`main`, `develop`, `feature`, `pre-prod`). These are not the same and must not be conflated.
- **`gcp-dev` maps to `stg`**: services on the `gcp-dev` branch deploy to staging, not a separate dev environment.
- **Toolchain override**: if `env.INFRA_ENV == 'toolchain'` and the service is Node, `branch_name` is forced to `develop` so the image goes to `stg` toolchain registry regardless of actual branch.
- **`farmiso-main`** is treated as an alias for `main` — it maps to `prd` on both push and PR builds to `main`.
- **Use `env.CHANGE_ID` to detect PR context — not `env.BRANCH_NAME =~ /PR-/`**: the `BRANCH_NAME =~ /PR-/` pattern breaks on non-GitHub SCMs and re-triggered builds. `env.CHANGE_ID` is the canonical PR-build detector set by the GitHub Branch Source plugin and is used throughout `constructParam.groovy:run` (`src/com/meesho/utilities/constructParam.groovy:run`).
## Related concepts
- [Config policy](06-CONFIG-POLICY.md) — `constructParam` sets env vars after mapping
- [Architecture](01-ARCHITECTURE.md) — where `commonCICDFlow` calls `constructParam`
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](04-DEPLOY-ARGOCD.md) · [Index](../index.md) · [Next →](06-CONFIG-POLICY.md)
docs/wiki/pages/05-cross-cutting.md
+84
View File
@@ -0,0 +1,84 @@
<!-- m-wiki: type=top-level slug=cross-cutting topic=null base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:src/com/meesho/utilities/constructParam.groovy, code:src/com/meesho/utilities/nodePoolSelection.groovy, code:src/com/meesho/utilities/gitActions.groovy, code:src/com/meesho/stages/deployArgoCD.groovy, code:src/com/meesho/stages/hotFix.groovy, code:vars/log.groovy] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: top-level. 6 sources.
# Cross-cutting patterns
## Retries
| Caller | Downstream | Retry |
|---|---|---|
| [`deployArgoCD.groovy:506,526`](../../../src/com/meesho/stages/deployArgoCD.groovy) | ArgoCD sync | `argocd app sync --http-retry-max 3 --retry-backoff-duration 1m` |
| [`buildNode.groovy:389`](../../../src/com/meesho/stages/buildNode.groovy), [`buildGo.groovy:172`](../../../src/com/meesho/stages/buildGo.groovy) | Docker registry push | `retryDockerPush` — 5 attempts with 3s sleep between |
| `buildMaven`, `buildGradle`, `buildPython`, `buildPhp` | various | **No retry** |
| [`securityScan.groovy:12`](../../../src/com/meesho/stages/securityScan.groovy) | In-house scanner | **No retry** |
| [`notify.groovy:108-152`](../../../src/com/meesho/stages/notify.groovy) | Deployment Tracker | **No retry** — single failed POST = missed deploy in the dashboard |
`BUGS_AND_IMPROVEMENTS_REPORT.md` lists "no retry mechanism" as P1; the reconciled-in nuance is that retry IS present for ArgoCD sync + Docker push, but not for most other downstream calls.
## BU multi-tenancy
`env.BU` and `config.bu` flow from CAC config (read once by [`constructParam.groovy`](../../../src/com/meesho/utilities/constructParam.groovy)) and drive:
| Use | Code site |
|---|---|
| ArgoCD namespace | `argocd-${env.BU}-prd` — [`constructParam.groovy:318`](../../../src/com/meesho/utilities/constructParam.groovy) |
| GCP cluster name | `k8s-${env.BU}-prd-ase1` — [`constructParam.groovy:308`](../../../src/com/meesho/utilities/constructParam.groovy) |
| GCP project | `meesho-${config.bu}-prd-0622` — [`constructParam.groovy:178`](../../../src/com/meesho/utilities/constructParam.groovy) |
| Helm chart path | `${env.helmChartsPath}/${config.bu}/...` — [`helmGenerator.groovy:150`](../../../src/com/meesho/stages/helmGenerator.groovy), [`deployArgoCD.groovy:293`](../../../src/com/meesho/stages/deployArgoCD.groovy) |
| Non-prd node pool | `${env.BU}-shared` (all `dev`/`ftr`/`stg` traffic for the BU collapses to one pool) — [`nodePoolSelection.groovy:133-135`](../../../src/com/meesho/utilities/nodePoolSelection.groovy) |
**Implication for non-prd capacity planning:** a single noisy service in `bu=supply` degrades every other `supply` service on staging, because they all share `supply-shared`. See [`docs/tribal-knowledge.md`](../../tribal-knowledge.md) §8.
## Feature flags / toggle conditions
| Flag | Set by | Effect | Read at |
|---|---|---|---|
| `env.hot_fix` | [`hotFix.groovy:11`](../../../src/com/meesho/stages/hotFix.groovy) | Skips Sonar + quality gate; sets canary `skipAnalysis` | [`buildGo.groovy:25-28`](../../../src/com/meesho/stages/buildGo.groovy), [`buildMaven.groovy:37-40`](../../../src/com/meesho/stages/buildMaven.groovy), [`deployArgoCD.groovy:355`](../../../src/com/meesho/stages/deployArgoCD.groovy) |
| `config.skip_sonar` | Service Jenkinsfile param | Skips quality-gate check (gated by whitelist for Maven prd) | [`buildMaven.groovy:18,240-243`](../../../src/com/meesho/stages/buildMaven.groovy), [`constructParam.groovy:51-56`](../../../src/com/meesho/utilities/constructParam.groovy) |
| `config.skip_security_scan` | Service Jenkinsfile param | Skips POST to security-scan endpoint | [`securityScan.groovy:7-9`](../../../src/com/meesho/stages/securityScan.groovy) |
| `config.push_to_jfrog` | Service Jenkinsfile param | Allow non-default-branch JFrog push (default branches: master, main) | [`buildMaven.groovy:19,377`](../../../src/com/meesho/stages/buildMaven.groovy) |
| `config.push_to_s3` | Service Jenkinsfile param | Allow non-default-branch S3 push (default branches: master, main, gcp-main, gcp-master) | [`buildMaven.groovy:20,462`](../../../src/com/meesho/stages/buildMaven.groovy) |
| `env.INFRA_ENV == 'toolchain'` | CAC config | Skips Docker push, uses latest-tag logic | [`buildGo.groovy:40-58`](../../../src/com/meesho/stages/buildGo.groovy), [`buildNode.groovy:215-239`](../../../src/com/meesho/stages/buildNode.groovy) |
| `env.CHANGE_ID` | GitHub Branch Source plugin | PR-build detection; remaps `cicd_environment` (`prd→int` for main/master PRs, `stg→ftr` for develop PRs) | [`constructParam.groovy:107-110`](../../../src/com/meesho/utilities/constructParam.groovy) |
**Do not use `env.BRANCH_NAME =~ /PR-/`** as a PR-build detector — it breaks on re-triggered builds and non-GitHub SCMs.
## Parallel execution
The pipeline is **almost entirely linear**. The one confirmed `parallel { }` block is in multi-module Go builds at [`buildGo.groovy:161-179`](../../../src/com/meesho/stages/buildGo.groovy):
```groovy
for (m in modules) {
moduleBuilds["build-${moduleName}"] = { ... }
}
parallel moduleBuilds
```
Anything called from inside that closure must respect the `@NonCPS` rule (see [`constructTemplate.groovy:13-20`](../../../src/com/meesho/utilities/constructTemplate.groovy) — the template engine wrapper is `@NonCPS` because `SimpleTemplateEngine` is non-serialisable).
## Exception handling — inconsistent on purpose-ish
`catch (Exception e)` appears ~87 times across stages. Three observed patterns:
| Pattern | Example | Behaviour |
|---|---|---|
| Catch + log + **rethrow** | [`buildNode.groovy:20-26`](../../../src/com/meesho/stages/buildNode.groovy) | Hard-fail the stage. Most common. |
| Catch + log + **swallow** (loop body) | [`deployArgoCD.groovy:103-106`](../../../src/com/meesho/stages/deployArgoCD.groovy) | Continue with the next deployment in the loop. Intentional for multi-deploy resilience. |
| Catch + log + **swallow** (silent fall-through) | [`buildNode.groovy:199-200`](../../../src/com/meesho/stages/buildNode.groovy) | Recovers with a sensible default. Use with caution — readers don't always realise the stage "succeeded" while masking a real error. |
When adding new error handling, prefer the rethrow pattern unless the loop-resilience semantics are an explicit requirement.
## Validation: `validate_configs.py`
[`resources/com/meesho/validate_configs.py`](../../../resources/com/meesho/validate_configs.py) (1207 lines) is the monolithic CAC schema validator, invoked from Groovy via `sh`. Key validation classes:
- **Schema** ([line 949](../../../resources/com/meesho/validate_configs.py)) — `application-{dev,int,stg,prd}.yml` against `application-schema.yml`.
- **Cross-env endpoint check** ([line 1010](../../../resources/com/meesho/validate_configs.py)) — prevents stg configs from referencing prd endpoints.
- **Secrets detection** ([line 1099](../../../resources/com/meesho/validate_configs.py)) — via `detect_secrets` library.
- **DB URL patterns** ([lines 41-59](../../../resources/com/meesho/validate_configs.py)) — PostgreSQL/MySQL/MongoDB/Redis/SQLite URI shapes.
- **Zookeeper endpoint patterns** — env-specific allow-lists.
`validate_configs_v2.py` (1261 lines) is the eventual replacement; both are referenced today.
See also: [concepts/whitelists](concepts/whitelists.md), [concepts/secrets-and-auth](concepts/secrets-and-auth.md), [concepts/observability](concepts/observability.md).
docs/wiki/pages/06-CONFIG-POLICY.md
+82
View File
@@ -0,0 +1,82 @@
<!-- m-wiki: type=top-level slug=config-policy topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Config and Policy
`constructParam.groovy` does two things: it resolves all `env.*` variables from `config.yaml` and the current branch/environment, and it enforces five policy gates by fetching `Meesho/whitelists` at runtime.
## TL;DR
- `config.yaml` is the service's contract — `getYamlParameter.getParam()` reads it before any build runs.
- `constructParam.run()` sets 15+ `env.*` vars (registry, vault, sonar, GCPProject, etc.) based on `cicd_environment`.
- Five whitelist files are fetched from `Meesho/whitelists` at runtime via `git clone` — not bundled.
- Whitelisted policies: skip-sonar, app-config-disabled, multizone, allowedNonDevelopPrToInt, ValidateCacConfig.
- `ValidateCacConfig` gate triggers `validate_configs_v2.py` on Go and Maven PRs.
## Mental model
`constructParam` is the policy layer. All build and deploy stages read `env.*` but never set it — they are consumers. `constructParam` is the sole producer. This ensures every stage shares a consistent view of which cloud, environment, registry, and policy applies to this build.
The whitelist pattern allows DevOps to grant exceptions without modifying any service's code — a repo is added to `Meesho/whitelists/skip-sonar-whitelist.yaml` and the next build automatically picks up the exception.
## Structure / data flow
```
config.yaml (in service repo):
repo_name, build_tool, dockerBuildVersion, bu, team,
deployment_order, notify_channel, skip_sonar, deployArgo, appConfigEnabled
getYamlParameter.getParam(repo_name) → param Map
constructParam.run(param):
├─ environment_map → env.cicd_environment
├─ GCP accountDetails[env.cicd_environment]:
│ env.GCPProject, env.registry, env.buildRegistry
│ env.vaultURL/Token, env.sonarURL/Token
│ env.objBucket, env.DOCKER_HOST
│
├─ [whitelist gate] skipSonarCheckForbidden() → clone Meesho/whitelists/skip-sonar-whitelist.yaml
├─ [whitelist gate] appConfigDisabledForbidden() → app-config-disabled.yaml
├─ [whitelist gate] isMultizoneEnabled() → multizone-enabled-repos.yaml
├─ [whitelist gate] allowedNonDevelopPrDeploymentToIntRepos() → allowedNonDevelopPrDeploymentToInt.yaml
└─ [whitelist gate] ValidateCacConfigForRepo() → ValidateCacConfig.yaml
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `run` | `src/com/meesho/utilities/constructParam.groovy:run` | Main env var setter |
| `perDeploymentVars` | `src/com/meesho/utilities/constructParam.groovy:perDeploymentVars` | Sets per-deployable ArgoCD vars |
| `getWhitelistedRepos` | `src/com/meesho/utilities/constructParam.groovy:getWhitelistedRepos` | Clones Meesho/whitelists and reads a YAML file |
| `skipSonarCheckForbidden` | `src/com/meesho/utilities/constructParam.groovy:skipSonarCheckForbidden` | Blocks Maven prd builds with skip_sonar=true if not whitelisted |
| `appConfigDisabledForbidden` | `src/com/meesho/utilities/constructParam.groovy:appConfigDisabledForbidden` | Blocks stg deploys if appConfig disabled and not whitelisted |
| `isMultizoneEnabled` | `src/com/meesho/utilities/constructParam.groovy:isMultizoneEnabled` | Returns true for deployables in multizone whitelist |
| `ValidateCacConfigForRepo` | `src/com/meesho/utilities/constructParam.groovy:ValidateCacConfigForRepo` | Returns true if repo must run CAC validation |
| `getParam` | `src/com/meesho/utilities/getYamlParameter.groovy:getParam` | Reads a YAML file from the workspace |
## Sharp edges
- **Each whitelist call does a fresh `git clone`**: `getWhitelistedRepos()` clones `Meesho/whitelists` into a `whitelist/` subdirectory every time it's called. Five separate calls = five clones in the same build. Network latency here directly adds to build time. This is deliberate — each clone captures the latest whitelist state so a DevOps policy change takes effect on the very next build without a library release. Never cache across calls.
- **Go sonar-skip logic is in `constructParam.groovy`, not `buildGo.groovy`**: `skipSonarCheckForGo(Map config)` (`src/com/meesho/utilities/constructParam.groovy:skipSonarCheckForGo`) centralises all skip-sonar policy. Embedding whitelist checks inline in language build stages is the wrong pattern.
- **`perDeploymentVars` must run before ArgoCD steps**: it sets `env.argoURL`, `env.argoCreds`, `env.argoAppNS`, and `env.argoIncubator` per deployable. Calling ArgoCD stages before this results in empty ArgoCD credentials.
- **`bu` drives GCP project name**: `prodGCPProject = "meesho-${config.bu}-prd-0622"`. An invalid or misspelled `bu` in `config.yaml` produces a nonexistent GCP project name.
- **Toolchain env skips several policies**: when `env.INFRA_ENV == 'toolchain'`, skip_sonar is forced true and vault/sonar are pointed at `toolchain-dind-dev-svc`.
## Related concepts
- [Whitelist system](policy/whitelist-system.md) — detailed whitelist file inventory
- [CAC validation](policy/cac-validation.md) — what happens when ValidateCacConfig=true
- [Multi-tenancy](policy/multi-tenancy.md) — BU/team mapping and initials
- [Environment mapping](05-ENVIRONMENT-MAPPING.md) — how cicd_environment is determined
- [Security overview](security/security-overview.md) — credential handling, trust boundaries, security rules for new code
- [ADR index](adr/adr-index.md) — architectural decisions behind the whitelist and policy model (ADR-0003, ADR-0004)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](05-ENVIRONMENT-MAPPING.md) · [Index](../index.md) · [Next →](07-LANGUAGE-BUILDS.md)
docs/wiki/pages/07-LANGUAGE-BUILDS.md
+87
View File
@@ -0,0 +1,87 @@
<!-- m-wiki: type=top-level slug=language-builds topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Language Builds
Each language builder in `src/com/meesho/stages/` implements a `run(Map config)` method. They share a common structure: read config → config-only check → compile/test → Sonar → Docker build → push → deploy.
## TL;DR
- **Maven** (`buildMaven`): Java services using pom.xml. Handles multi-module layouts. Skips tests/sonar on hotfix. Pushes JAR to JFrog optionally.
- **Go** (`buildGo`): Uses Athens proxy (`env.goProxyUrl`) for module caching. Sonar skipped if whitelisted or toolchain env.
- **Node** (`buildNode`): Multiple variants (`node-18`, `node-20`, etc.). Toolchain-aware image path includes `TOOLCHAIN_ENV` segment.
- **Python** (`buildPython`): `python-3.10`, `python-3.12`, etc. — version in `dockerBuildVersion`, not separate files.
- **Gradle** (`buildGradle`): Android/Kotlin server builds.
- **PHP** (`buildPhp`): PHP services, uses `php-values.yaml` Helm template.
- **Rust** (`buildRust`): Rust services.
- **Docker** (`buildDocker`): Docker-only builds — no compiler, just image assembly.
## Mental model
Every builder reads `config.build_tool` (via `buildObjHelper` dispatch) and `config.dockerBuildVersion`. The `dockerBuildVersion` controls which Dockerfile template is selected from `resources/com/meesho/` — the template name roughly mirrors the version string (e.g., `maven-21`, `go-1.22`, `node-20`).
Config-only change detection is embedded in each builder. If `git diff HEAD~1 -- configs/` shows only YAML changes, the builder skips compilation and fetches the last image tag from GAR instead.
## Structure / data flow
```
buildMaven.run(config):
env.TAG = "v${pom-version}" or hotfix → "v${version}-HOT"
├─ [config-only] check s3/GAR for existing artifact
├─ stage: Test → mvn test -P${java_version}
├─ stage: Sonar → mvn sonar:sonar (skip if skipSonarCheckForbidden)
├─ stage: Docker Build → constructTemplate.renderTemplate(binding, 'maven-Dockerfile')
├─ stage: Docker Push → dockerUtilities.buildAndPush(...)
└─ deployArgoCD.run(...) or deployJar.run(...)
buildGo.buildDckr(config):
env.TAG = getDockerParams.getTag(repo_name)
├─ [config-only] skip build if only configs/ changed
├─ stage: Build/Test → go build, go test (with Athens proxy)
├─ stage: Sonar → sonar-scanner (skip if skipSonarCheckForGo)
├─ stage: Docker Build → constructTemplate.renderTemplate(binding, 'go-Dockerfile')
└─ deployArgoCD.run(...)
buildNode.buildDckr(config):
env.TAG = getDockerParams.getTag(repo_name)
├─ npm install + npm run build
├─ stage: Sonar → sonar-scanner (node sonar/sonar-scanner.ts)
├─ stage: Docker Build → constructTemplate.renderTemplate(binding, 'node-Dockerfile')
└─ deployArgoCD.run(...)
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `run` | `src/com/meesho/stages/buildMaven.groovy:run` | Maven build lifecycle |
| `buildDckr` | `src/com/meesho/stages/buildGo.groovy:buildDckr` | Go build lifecycle |
| `buildDckr` | `src/com/meesho/stages/buildNode.groovy:buildDckr` | Node build lifecycle |
| `buildDckr` | `src/com/meesho/stages/buildPython.groovy:buildDckr` | Python build lifecycle |
| `run` | `src/com/meesho/stages/buildGradle.groovy:run` | Gradle build lifecycle |
| `buildDckr` | `src/com/meesho/stages/buildRust.groovy:buildDckr` | Rust build lifecycle |
| `retryDockerPush` | `src/com/meesho/utilities/dockerUtilities.groovy:retryDockerPush` | Docker push with retry |
| `getTag` | `src/com/meesho/utilities/getDockerParams.groovy:getTag` | Image tag: v{version}-{sha}-{epoch} |
## Sharp edges
- **`python-3.10.12` is aliased to `python-3.7` in Helm values**: `deployArgoCD.update_helm_repo` remaps `python-3.10.12` → `python-3.7` for the `dockerBuildVersion` key passed to the Helm template. Other python versions pass through as-is.
- **Maven multi-module builds**: `getDockerParams.getModules()` reads `<modules>` from `pom.xml`. If modules exist, a Docker image is built per module.
- **toolchain env Node builds** include `TOOLCHAIN_ENV` in the image path: `${cicd_environment}/${TOOLCHAIN_ENV}/${team}/${repo}`. Standard builds omit the `TOOLCHAIN_ENV` segment.
- **`skip_s3_check=true` forces a fresh build** even if an artifact already exists for the same commit.
## Related concepts
- [Build dispatch](build/build-dispatch.md) — regex routing to the correct builder
- [Config-only detection](build/config-only-detection.md) — when builds are skipped
- [Dockerfile templates](08-DOCKERFILE-TEMPLATES.md) — template rendering
- [Docker tagging](build/docker-tagging.md) — tag format used by all builders
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](06-CONFIG-POLICY.md) · [Index](../index.md) · [Next →](08-DOCKERFILE-TEMPLATES.md)
docs/wiki/pages/08-DOCKERFILE-TEMPLATES.md
+84
View File
@@ -0,0 +1,84 @@
<!-- m-wiki: type=top-level slug=dockerfile-templates topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Dockerfile Templates
Dockerfile templates live in `resources/com/meesho/`. Each build stage calls `constructTemplate.renderTemplate()` to select and render the appropriate template with service-specific values.
## TL;DR
- Templates use Groovy's `SimpleTemplateEngine` — `${}` substitutions from a `Map binding`.
- `constructTemplate.renderTemplate(binding, templateFile, outputPath)` calls `libraryResource` to load the template, renders it, and writes the output file.
- Template selection is by `dockerBuildVersion` string: `maven-21`, `go-1.22`, `node-20`, etc.
- Output is always written to the workspace and then used by the Docker build step.
- `_construct()` is annotated `@NonCPS` because `SimpleTemplateEngine` is not CPS-serializable.
## Mental model
`constructTemplate` is a thin wrapper around Groovy's standard template engine. The binding map contains all deployment metadata (repo name, version tag, environment, JVM flags, etc.) assembled by the build stage and `enable_backward_compatibility()`. The template file itself is stored as a Jenkins library resource and loaded via `libraryResource`.
## Structure / data flow
```
Dockerfile template selection (in each build stage):
dockerBuildVersion = "go-1.22"
│
▼
constructTemplate.renderTemplate(
binding = { repo_name, tag, env, ... },
templateFile = "go-1.22-Dockerfile", (or "go-Dockerfile" depending on version)
outputPath = "${WORKSPACE}/Dockerfile"
)
│
▼
libraryResource "com/meesho/go-1.22-Dockerfile"
│
▼
SimpleTemplateEngine.createTemplate(text).make(binding)
│
▼
writeFile(Dockerfile) → docker build -f Dockerfile -t <registry>/<repo>:<tag> .
```
Available templates (from `resources/com/meesho/`):
```
maven-Dockerfile maven-21-Dockerfile
go-Dockerfile go-1.22-Dockerfile
node-Dockerfile node-18-Dockerfile node-20-Dockerfile
python-Dockerfile python-3.7-Dockerfile python-3.10-Dockerfile python-3.12-Dockerfile
rust-Dockerfile
php-Dockerfile
argoApp.yaml (ArgoCD Application manifest template)
values.yaml node-values.yaml go-values.yaml python-values.yaml
cron-values.yaml php-values.yaml
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `renderTemplate` | `src/com/meesho/utilities/constructTemplate.groovy:renderTemplate` | Loads library resource + renders + writes file |
| `_construct` | `src/com/meesho/utilities/constructTemplate.groovy:_construct` | `@NonCPS` template rendering via SimpleTemplateEngine |
| `get_value_yaml_file` | `src/com/meesho/stages/deployArgoCD.groovy:get_value_yaml_file` | Selects which values.yaml template to use for Helm |
| `get_default_command` | `src/com/meesho/stages/deployArgoCD.groovy:get_default_command` | Default container command per build version |
## Sharp edges
- **`_construct` is `@NonCPS`** (`src/com/meesho/utilities/constructTemplate.groovy:_construct`): this means it cannot access Jenkins pipeline steps (e.g., `echo`, `sh`) or `env.*` inside the method. All values must be in the `binding` map. Any caller that invokes `_construct` inside a `parallel` block or closure must ensure the closure itself is also `@NonCPS` or does not cross a serialisation boundary. Do not move `_construct` into a CPS context — keep the annotation and call it from a CPS-safe wrapper.
- **`binding` is copied defensively**: `_construct` wraps the incoming map in `new HashMap(binding)` before passing to the engine, preventing mutation of the caller's map.
- **`argoApp.yaml` is also a template**: the ArgoCD Application manifest is rendered the same way as Dockerfiles. This means ArgoCD app metadata (cluster, namespace, Helm chart path) is all driven by the deployment YAML binding.
- **Values templates select by `dockerBuildVersion`**: `get_value_yaml_file` maps `node-*` → `node-values.yaml`, `python-*` → `python-values.yaml`, etc. An unrecognized version returns `null` and the pipeline fails.
## Related concepts
- [Language builds](07-LANGUAGE-BUILDS.md) — which stage calls renderTemplate
- [Deploy ArgoCD](04-DEPLOY-ARGOCD.md) — uses renderTemplate for both ArgoCD app and Helm values
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](07-LANGUAGE-BUILDS.md) · [Index](../index.md) · [Next →](09-INFRA-PODS.md)
docs/wiki/pages/09-INFRA-PODS.md
+81
View File
@@ -0,0 +1,81 @@
<!-- m-wiki: type=top-level slug=infra-pods topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Infra: GCP Jenkins Agent Pods
All GCP builds run inside a Jenkins Kubernetes pod provisioned from YAML specs in `resources/org/meesho/`. The pod YAML is selected by `env.INFRA_ENV` and loaded via `libraryResource` in `eksCICD.gcpInfra()`.
## TL;DR
- Pod specs live in `resources/org/meesho/*.yaml`.
- Filename pattern: `<INFRA_ENV>-pod.yaml` and `<INFRA_ENV>-sidecar-pod.yaml`.
- `INFRA_ENV` values seen in practice: `prd`, `stg`, `dev`, `toolchain`.
- All builds run in the `devops-tools` container defined in the pod spec.
- Sidecar pods (`useSidecar: true` in consumer Jenkinsfile) add extra containers alongside `devops-tools`.
- AWS builds skip this entirely — they run directly on static EKS nodes labeled `EKS`.
## Mental model
Jenkins provisions a fresh Kubernetes pod for each build. The pod spec defines:
- Which container images to run (typically `devops-tools` + optional sidecar).
- Resource requests/limits for the build container.
- Any mounted volumes (e.g., Docker socket for DinD builds).
`gcpInfra()` reads `env.INFRA_ENV` (injected by the Jenkins job configuration), constructs the pod YAML filename, loads it via `libraryResource`, and wraps the entire pipeline in a `podTemplate { node { container('devops-tools') { ... } } }` block.
## Structure / data flow
```
eksCICD.call(repo):
CLOUD_PROVIDER=GCP → gcpInfra(repo)
gcpInfra(repo):
isSidecarNeeded = repo.get('useSidecar', false)
yamlName = isSidecarNeeded ? "${INFRA_ENV}-sidecar-pod.yaml" : "${INFRA_ENV}-pod.yaml"
podyaml = "org/meesho/${yamlName}"
podTemplate(yaml: libraryResource(podyaml)) {
node(POD_LABEL) {
container('devops-tools') {
commonCICDFlow(repo)
}
}
}
Pod YAML inventory (resources/org/meesho/):
prd-pod.yaml prd-sidecar-pod.yaml
stg-pod.yaml stg-sidecar-pod.yaml
dev-pod.yaml dev-sidecar-pod.yaml
toolchain-pod.yaml toolchain-sidecar-pod.yaml
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `gcpInfra` | `vars/eksCICD.groovy:gcpInfra` | Selects pod YAML and wraps pipeline in podTemplate |
| `awsInfra` | `vars/eksCICD.groovy:awsInfra` | AWS path — no pod YAML, just node('EKS') |
| `run` | `src/com/meesho/utilities/nodePoolSelection.groovy:run` | Selects GKE node pool label for deployment (separate from build pod) |
## Sharp edges
- **Build pod vs deployment node pool are different things**: the Jenkins agent pod is where the build runs; `nodePoolSelection.run()` selects which GKE node pool the deployed *application* should land on. These are independent.
- **`INFRA_ENV` is job-level config**: it is not derived from the branch name. It is injected by the Jenkins multibranch pipeline configuration. A mis-configured job can run a `main` branch build with `INFRA_ENV=dev` — which would use the dev pod spec.
- **Toolchain pods are segregated**: `toolchain-pod.yaml` uses images from the `toolchain` namespace in `meesho-central-dev-0622` project — separate from standard build pods.
- **DinD (Docker-in-Docker) socket**: the `devops-tools` container in pod specs mounts a DinD service socket (`env.DOCKER_HOST`) rather than the host Docker socket. The `DOCKER_HOST` env var is set by `constructParam.run()` based on `cicd_environment`. The correct endpoint values are `dind-prd-svc` (prd) and `dind-dev-new-svc.jenkins-new.svc.cluster.local` (stg/ftr). Do not use `tcp://localhost:2375` — it creates conflicts that require unnecessary guard logic. The DinD image must come from the internal GAR registry (`asia-southeast1-docker.pkg.dev/meesho-devops-admin-0622/admin/devops/docker:28-dind`) — not Docker Hub.
- **`build-tools` image version encodes the Go compiler and sonar-scanner CLI**: `resources/org/meesho/prd-pod.yaml` and `stg-pod.yaml` reference a `build-tools` image that bundles Go, `sonar-scanner-cli`, and other toolchain binaries. When a new toolchain binary is needed, bump the image tag in both pod YAMLs — never `curl`/`wget` a tool from the internet inside a pipeline stage.
- **Non-prd node pools are BU-scoped**: `stg`/`ftr`/`dev` pods land on `{BU}-shared` pools, so all services in the same BU share one pool. A memory leak or noisy-neighbour in one `supply` service degrades all other `supply` services on staging.
## Related concepts
- [Node pool selection](infra/node-pool-selection.md) — GKE node pool assignment for deployed applications
- [Architecture](01-ARCHITECTURE.md) — where gcpInfra fits in the pipeline
- [Config policy](06-CONFIG-POLICY.md) — env.DOCKER_HOST set by constructParam
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](08-DOCKERFILE-TEMPLATES.md) · [Index](../index.md) · [Next →](10-NOTIFICATIONS.md)
docs/wiki/pages/10-NOTIFICATIONS.md
+82
View File
@@ -0,0 +1,82 @@
<!-- m-wiki: type=top-level slug=notifications topic=null base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: top-level. 0 sources.
# Notifications
`notify.groovy` runs in the pipeline's `finally` block — it always executes regardless of build outcome. It routes to different backends based on who triggered the build and what environment the build ran in.
## TL;DR
- `ringmaster-bot` or `turbo-turtle` triggered → calls `deployRingmaster.run()` + Slack on `prd`.
- All other users → direct Slack `slackSend` only.
- Toolchain env → HTTP callback to `172.23.72.116:5002/api/v1/deploy/build-callback` (skip Slack/Ringmaster).
- On `prd` push builds, also posts to `deployment-tracker` API and Ringmaster deployment history.
- `notify_channel` comes from `config.yaml`; defaults to `ci-cd-status` if empty.
## Mental model
`notify.run()` is the final stage in `commonCICDFlow`. Since it's in a `finally` block, `env.msg` carries the final build status message (set by each catch block in `commonCICDFlow`). The notification backend selection depends on the triggering user, not on the environment:
- `ringmaster-bot` → Ringmaster API (`/api/v1/key/cicd/cd/update`) with a token-authenticated payload.
- Turbo-Turtle or other allowed users → Turbo-Turtle callback (`/api/v1/ci/jenkins/callback`) via a temporary JSON file to avoid shell quoting issues.
## Structure / data flow
```
notify.run(config):
├─ [toolchain] if env.INFRA_ENV == 'toolchain':
│ POST http://172.23.72.116:5002/api/v1/deploy/build-callback
│ return (skip everything below)
│
├─ build_user = getCause(UserIdCause).getUserId()
│
├─ if build_user in [ringmaster-bot, turbo-turtle]:
│ deployRingmaster.run(...) → POST to Ringmaster or Turbo-Turtle
│ if prd: slackSend(notify_channel, deploy URL)
│ skip_notify = true
│
├─ if !skip_notify:
│ slackSend(notify_channel, job name + build number + tag + msg)
│
└─ if main/master branch:
postTrackingApi(config) ← deployment-tracker API
postTrackingRingmasterApi(config) ← Ringmaster deployment history
deployRingmaster.run(repo_name, deployment_order, tag, ...):
if build_user == "ringmaster-bot":
POST https://ringmaster-api.meeshogcp.in/api/v1/key/cicd/cd/update
else:
POST http://turbo-turtle.meeshogcp.in/api/v1/ci/jenkins/callback
```
## Key code locations
| Symbol | File | What it does |
|--------|------|--------------|
| `run` | `src/com/meesho/stages/notify.groovy:run` | Main notify dispatcher |
| `postTrackingApi` | `src/com/meesho/stages/notify.groovy:postTrackingApi` | Posts to deployment-tracker API |
| `postTrackingRingmasterApi` | `src/com/meesho/stages/notify.groovy:postTrackingRingmasterApi` | Posts to Ringmaster deployment history |
| `run` | `src/com/meesho/stages/deployRingmaster.groovy:run` | Routes to Ringmaster or Turbo-Turtle callback |
| `callApi` | `src/com/meesho/stages/deployRingmaster.groovy:callApi` | Ringmaster API call with `ringmaster-token` credential |
## Sharp edges
- **JSON payload via temp file** (Turbo-Turtle path): the Turbo-Turtle curl call writes the payload to a temp file (`cicd_payload_${BUILD_NUMBER}_${ts}.json`) to avoid shell quoting issues with JSON special characters. The file is always deleted in a `finally` block.
- **`prd` vs `int` API endpoints**: for both Ringmaster and deployment-tracker, `prd` and `int` share the `ringmaster-api.meeshogcp.in` endpoint while `stg`/`ftr` use `ringmaster-api.admin.meeshogcp.in`.
- **`skip_notify` is `env.*` not `config.*`**: it's set by `constructParam.run()` (always `true` for GCP) and can also be set by the consumer `config.yaml`. Both must be false for Slack to fire.
- **`env.msg` is the error channel**: each `catch` block in `commonCICDFlow` sets `env.msg` before the `finally` block calls `notify`. If nothing failed, `env.msg` stays `'Job Passed'`.
## Related concepts
- [Ringmaster integration](deploy/ringmaster-integration.md) — detailed routing logic
- [Architecture](01-ARCHITECTURE.md) — where notify fits in commonCICDFlow
- [Environment mapping](05-ENVIRONMENT-MAPPING.md) — cicd_environment determines API base URL
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Previous](09-INFRA-PODS.md) · [Index](../index.md) · [Next →](10-NOTIFICATIONS.md)
docs/wiki/pages/adr/adr-index.md
+55
View File
@@ -0,0 +1,55 @@
<!-- m-wiki: type=concept slug=adr-index topic=adr base-sha=d6708eca4236 generated-at=2026-05-12T12:00:00+00:00 sources=[docs/adr/README.md, docs/adr/0001-single-shared-library-for-all-services.md, docs/adr/0002-branch-name-as-sole-environment-selector.md, docs/adr/0003-policy-exceptions-in-separate-whitelist-repo.md, docs/adr/0004-fresh-whitelist-clone-per-build.md, docs/adr/0005-config-only-change-detection-skip-build.md, docs/adr/0006-ringmaster-mandatory-build-trigger-gate.md, docs/adr/0007-gitops-via-argocd-4-step-sync-sequence.md, docs/adr/0008-canary-mandatory-for-tier1-services-in-prd.md, docs/adr/0009-jvm-heap-auto-derived-from-pod-memory-request.md] -->
> Generated 2026-05-12 at base-sha d6708eca4236. Type: concept. 10 sources.
# Architecture Decision Records — Index
devops-lib's ADRs capture the "why" behind the library's design — the decisions that would otherwise be tribal knowledge. All 9 ADRs were retroactively documented on 2026-05-12 from codebase analysis and developer interviews. All are **Status: Accepted** and assessed as still appropriate.
## Why ADRs matter here
devops-lib has several decisions whose rationale is non-obvious: why does the whitelist get re-cloned on every build (not cached)? Why is the 4-step ArgoCD sequence non-interchangeable? Why must all builds go through Ringmaster? Without ADRs, engineers modifying the library break load-bearing constraints without realising it.
## Decision inventory
| # | Decision | Category | Key insight |
|---|----------|----------|-------------|
| [ADR-0001](../../../adr/0001-single-shared-library-for-all-services.md) | Single shared library for all services | PATTERN | Policy enforcement must be uniform; per-team Jenkinsfiles produce drift |
| [ADR-0002](../../../adr/0002-branch-name-as-sole-environment-selector.md) | Branch name as sole environment selector | PATTERN | Branch IS the environment contract; per-service env config creates misconfiguration risk |
| [ADR-0003](../../../adr/0003-policy-exceptions-in-separate-whitelist-repo.md) | Policy exceptions in Meesho/whitelists repo | PATTERN | Service teams cannot self-grant bypasses; all exceptions require DevOps review |
| [ADR-0004](../../../adr/0004-fresh-whitelist-clone-per-build.md) | Fresh whitelist clone per build | PATTERN | Policy changes must take effect on the very next build without a library release |
| [ADR-0005](../../../adr/0005-config-only-change-detection-skip-build.md) | Config-only change detection — skip build | PATTERN | No source change → no new binary; reuse the latest image tag from GAR |
| [ADR-0006](../../../adr/0006-ringmaster-mandatory-build-trigger-gate.md) | Ringmaster mandatory trigger gate | PATTERN | Every deployment must be tracked; direct Jenkins triggers bypass Ringmaster's ledger |
| [ADR-0007](../../../adr/0007-gitops-via-argocd-4-step-sync-sequence.md) | GitOps via 4-step ArgoCD sync sequence | INFRA | Steps 2 and 4 are non-interchangeable; swapping them causes silent sync failures |
| [ADR-0008](../../../adr/0008-canary-mandatory-for-tier1-services-in-prd.md) | Canary mandatory for Tier-1 (sp0/up0) in prd | RELIABILITY | Incident-driven: non-canary prd deploys for critical services caused outages |
| [ADR-0009](../../../adr/0009-jvm-heap-auto-derived-from-pod-memory-request.md) | JVM heap auto-derived from pod memory_request | RELIABILITY | OOM incident remediation: auto-calc prevents under-sizing; `jvm_memory_override: true` escapes it |
## Decisions that are load-bearing constraints
These three decisions have "never change without understanding this" consequences:
**ADR-0004 (fresh whitelist clone)** — `getWhitelistedRepos` in `src/com/meesho/utilities/constructParam.groovy:getWhitelistedRepos` re-clones `Meesho/whitelists` on every call by design. Never add a cache — it would prevent immediate policy enforcement.
**ADR-0006 (Ringmaster gate)** — The string `"ringmaster-bot"` at `vars/eksCICD.groovy:call` is the sole routing signal between Ringmaster and Turbo-Turtle callbacks. Never rename it without coordinating with both teams.
**ADR-0007 (4-step ArgoCD sequence)** — Steps 2 (`refresh_app_of_apps`) and 4 (`refresh_and_sync`) in `src/com/meesho/stages/deployArgoCD.groovy:run` are non-interchangeable. For a first-deploy, step 2 must create the Application object before step 4 can sync it.
## When to read which ADR
- **Debugging a build not triggering**: ADR-0006 (Ringmaster gate)
- **Debugging a deploy that failed on ArgoCD sync**: ADR-0007 (4-step sequence)
- **Understanding why policy changes take effect immediately**: ADR-0004 (fresh clone)
- **Service team asking for a sonar bypass**: ADR-0003 (whitelist repo)
- **Canary enforcement failing for a prd deploy**: ADR-0008
- **OOM in a Java service pod**: ADR-0009
## Related concepts
- [Whitelist system](../policy/whitelist-system.md) — runtime whitelist enforcement (ADR-0003, ADR-0004)
- [ArgoCD sync](../deploy/argocd-sync.md) — step-by-step walkthrough (ADR-0007)
- [Ringmaster integration](../deploy/ringmaster-integration.md) — callback flow (ADR-0006)
- [Environment mapping](../05-ENVIRONMENT-MAPPING.md) — branch → environment (ADR-0002)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
docs/wiki/pages/build/build-dispatch.md
+53
View File
@@ -0,0 +1,53 @@
<!-- m-wiki: type=concept slug=build-dispatch topic=build base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# Build Dispatch
`buildObjHelper.run(build_tool)` maps a `build_tool` string from `config.yaml` to a concrete builder class instance using a Groovy `switch/case` with regex patterns.
## Where it applies in this repo
`src/com/meesho/stages/buildObjHelper.groovy:run`
The full dispatch table (in order, first match wins):
| Pattern | Builder class |
|---------|--------------|
| `maven` (exact) | `buildMaven` |
| `docker` (exact) | `buildDocker` |
| `~/^maven-.*/` | `buildMaven` |
| `~/^python-.*/` | `buildPython` |
| `~/^node-.*/` | `buildNode` |
| `~/^rust.*/` | `buildRust` |
| `~/^go.*/` | `buildGo` |
| `gradle` (exact) | `buildGradle` |
| `php` (exact) | `buildPhp` |
| default | `defaultBuild()` |
Each builder is instantiated fresh per build — no shared state between builds.
## Why this design
Groovy `switch/case` evaluates patterns top-to-bottom and returns on the first match. Regex patterns (the `~/…/` syntax) cover version-suffixed variants like `go-1.22`, `node-20`, `python-3.12` without requiring an exhaustive case list. The exact-match cases for `maven` and `docker` appear before the regex catch-all `~/^maven-.*/` to handle the legacy bare-string case.
If no case matches, `defaultBuild()` is called without logging a warning — the pipeline reports success with no artifact produced. A typo in `config.yaml` (e.g. `golang` instead of `go`) produces this silent no-op. If a build succeeds but produces no Docker image, check `build_tool` spelling in `config.yaml` first.
## Related
- [Build stages](../03-BUILD-STAGES.md) — broader build lifecycle
- [Language builds](../07-LANGUAGE-BUILDS.md) — per-language builder details
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/build/config-only-detection.md
+46
View File
@@ -0,0 +1,46 @@
<!-- m-wiki: type=concept slug=config-only-detection topic=build base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# Config-Only Change Detection
When a commit only changes YAML config files (no source code), each language builder skips the compile/test/image-build steps and instead fetches the last-built image tag from GAR (Google Artifact Registry). This avoids rebuilding identical binaries when only configs changed.
## Where it applies in this repo
Implemented independently in each language builder (`src/com/meesho/stages/buildMaven.groovy`, `buildGo.groovy`, `buildNode.groovy`, etc.).
The detection pattern varies slightly by builder but follows the same logic:
1. Run `git diff HEAD~1 -- configs/` (or `git diff ORIG_HEAD -- configs/`).
2. If the diff is non-empty AND no source files changed → set `appConfigChanges = true`, skip compilation.
3. Fetch the latest image tag from the artifact bucket (S3 or GCS) or GAR.
4. Set `env.TAG` to the fetched tag.
5. Proceed directly to `deployArgoCD.run()`.
The check is gated by `skip_s3_check` in `config.yaml`. Setting `skip_s3_check: true` forces a full rebuild even when only configs changed.
## Why this design
Config-only deployments are common at Meesho (dynamic config updates, feature flags). Rebuilding the entire Java or Go binary for a one-line YAML change wastes 3–10 minutes. By reusing the last image tag and skipping straight to ArgoCD deploy, the pipeline completes in ~1 minute for config-only changes.
The detection relies on `git diff` against the previous commit, so it only works when the commit history is linear. Force-pushes or squash merges may produce false negatives (full rebuild triggered unnecessarily).
## Related
- [Language builds](../07-LANGUAGE-BUILDS.md) — where this check is embedded per builder
- [Build dispatch](build-dispatch.md) — the builder instance that contains this check
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/build/docker-tagging.md
+57
View File
@@ -0,0 +1,57 @@
<!-- m-wiki: type=concept slug=docker-tagging topic=build base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# Docker Tagging
`getDockerParams.getTag()` produces the Docker image tag for each build. The tag format encodes the version, commit SHA, and a timestamp to guarantee uniqueness across rebuilds of the same commit.
## Where it applies in this repo
`src/com/meesho/utilities/getDockerParams.groovy:getTag`
**Standard builds:**
```
v{pom-version or package-version}-{7-char-sha}-{epoch-ms}
```
Example: `v1.4.2-a3f8c21-1714920000000`
**Toolchain builds** (when `env.INFRA_ENV == 'toolchain'`):
```
v{version}-{7-char-sha}
```
Example: `v1.4.2-a3f8c21`
The short SHA comes from `git log -1 --format=%h` (7 chars). The full 40-char SHA is also captured in `env.commit_id` as a side effect of `getCommitid()`.
Version is read from:
- `pom.xml` → `xq -r .project.version pom.xml`
- `package.json` → `jq -r .version package.json`
- Default: `1.0`
**Hotfix Maven builds** override the tag format to `v{version}-HOT` (set directly in `buildMaven.run()`, not via `getTag`).
## Why this design
The epoch-ms suffix ensures that two builds from the exact same commit produce different tags. This is intentional: if a build fails mid-way and is retried, the retry must produce a new image (the previous one may be partially pushed or broken). Without the timestamp, `docker push` on a retry would be a no-op if the tag already exists.
Toolchain builds omit the timestamp because toolchain images are content-addressed: the same source commit must always produce the same tag so toolchain consumers can pin to a stable reference without tracking timestamps.
## Related
- [Language builds](../07-LANGUAGE-BUILDS.md) — each builder sets `env.TAG` using getTag
- [Notifications](../10-NOTIFICATIONS.md) — `env.TAG` is included in Ringmaster and Slack payloads
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/build/node-paired-files.md
+49
View File
@@ -0,0 +1,49 @@
<!-- m-wiki: type=concept slug=node-paired-files topic=build base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# Node Build: Paired File Rule
`src/com/meesho/stages/buildNode.groovy` and `resources/com/meesho/node-Dockerfile` are paired files. Changes to Node install logic in one must be mirrored in the other to avoid a split-brain build path.
## Where it applies in this repo
`src/com/meesho/stages/buildNode.groovy:buildDckr`
`resources/com/meesho/node-Dockerfile`
## How they interact
`buildNode.groovy` detects the package manager at runtime (npm vs pnpm vs yarn) and passes the install command to the Dockerfile as the `npm_install_arg` template variable. The Dockerfile's `else` branch handles the fallback when no explicit or detected command is provided (currently `npm ci`).
```
buildNode.buildDckr(config):
├─ Detect package manager → npm_install_arg = "pnpm install" | "npm ci" | ...
├─ constructTemplate.renderTemplate(binding, 'node-Dockerfile')
│ └─ binding.npm_install_arg → substituted into node-Dockerfile
└─ docker build -f Dockerfile ...
```
When `npm_install_arg` is not set in `config.yaml`, `buildNode.groovy` falls back to its own detection logic. The Dockerfile default branch handles the case where detection produces nothing.
## Why this matters
If `buildNode.groovy` changes the fallback install command or adds support for a new package manager, the Dockerfile default branch must be updated in the same PR. Changing only one file leaves them out of sync: the runtime path may succeed while the Docker fallback uses the old command (or vice versa). This creates subtly different images depending on whether `npm_install_arg` is explicitly configured.
## Related
- [Language builds](../07-LANGUAGE-BUILDS.md) — Node builder overview
- [Dockerfile templates](../08-DOCKERFILE-TEMPLATES.md) — how renderTemplate works
## Sources
(no raw/ sources)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/build/scm-variable-scope.md
+54
View File
@@ -0,0 +1,54 @@
<!-- m-wiki: type=concept slug=scm-variable-scope topic=build base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# SCM Variable Scope
The `scm` variable is only available inside consumer Jenkinsfiles, not inside `vars/` or `src/` of the shared library. Referencing it in library code causes a runtime `MissingPropertyException`.
## Where it applies in this repo
`vars/eksCICD.groovy` and all files under `src/com/meesho/`
## Why `scm` is unavailable
The `scm` variable (branch, remote URL, credentials) is injected by the GitHub Branch Source plugin into **consumer Jenkinsfiles** at the time they are loaded by Jenkins. Library code (everything under `vars/` and `src/`) is a separate classloader context — the plugin does not inject `scm` there.
Attempting to access `scm.branches` or `scm.userRemoteConfigs` from library code will throw:
```
MissingPropertyException: No such property: scm for class: groovy.lang.Binding
```
## Correct alternatives
Inside library code, use the Jenkins-injected environment variables instead:
| Need | Use instead of `scm.*` |
|------|------------------------|
| Branch name | `env.BRANCH_NAME` |
| Repository URL | `env.GIT_URL` |
| PR number | `env.CHANGE_ID` |
| PR source branch | `env.CHANGE_BRANCH` |
| PR target branch | `env.CHANGE_TARGET` |
All of these are set by the GitHub Branch Source plugin before library code runs.
## Related
- [Entry points](../02-ENTRYPOINTS.md) — where this constraint applies
- [Environment mapping](../05-ENVIRONMENT-MAPPING.md) — `env.CHANGE_ID` is the PR-build detector
## Sources
(no raw/ sources)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/concepts/observability.md
+55
View File
@@ -0,0 +1,55 @@
<!-- m-wiki: type=concept slug=observability topic=concepts base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:vars/log.groovy] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: concept. 1 source.
# Logging via `log.groovy`
[`vars/log.groovy`](../../../../vars/log.groovy) is the closest thing this codebase has to a logger. It's a 24-line Groovy file that wraps `echo` with ANSI colour codes.
## The three calls
```groovy
log.info(msg) // → echo "${GREEN}INFO: ${msg}${BLACK}"
log.warning(msg) // → echo "${RED}WARNING: ${msg}${BLACK}"
log.error(msg) // → echo "${RED}ERROR: ${msg}${BLACK}"
```
That's the whole API. There is no severity level configuration, no structured fields (no JSON), no destination other than the Jenkins console.
## What `log.info` gives you over bare `echo`
- **ANSI colour** — pipeline output in the Jenkins UI is easier to scan when INFO is green and ERROR is red.
- **Consistent prefix** (`INFO: ` / `WARNING: ` / `ERROR: `) — a `grep ERROR pipeline.log` pattern works across every service.
- **Nothing else.** No timestamps (Jenkins adds those via `timestamps` wrapper), no caller tracing, no correlation ID.
## When `bare echo` is acceptable
Lots of existing code uses bare `echo`. The mix is mostly historical. New code should prefer `log.info` for normal messages and `log.error` for failures, but you'll see bare `echo` in:
- One-off banners (`echo "=========="`).
- Single-line status (`echo "Building module: ${m}"` inside a loop).
- ANSI-coloured ad-hoc messages where the code path needs a one-off colour (some `vars/eksCICD.groovy` and `vars/buildPipeline.groovy` lines do this).
There is no policy that bans bare `echo`. The reconcile run's `BUGS_AND_IMPROVEMENTS_REPORT.md` does list "inconsistent logging" as a P1 item but no rule has been graduated into `CLAUDE.md` NEVER DO yet.
## `env.msg` as failure state
A separate convention: most stages set `env.msg` to a human-readable failure reason before re-throwing, e.g. [`buildNode.groovy:21`](../../../../src/com/meesho/stages/buildNode.groovy):
```groovy
env.msg = 'Error in building node packages...'
log.error(env.msg)
currentBuild.result = env.FAILURE
throw e
```
`notify.groovy` then reads `env.msg` (and the related `env.error_msg_to_db`) when composing the Slack message and the deployment-tracker payload. **Don't rename `env.msg`** — too many callers read it. [`deployRingmaster.groovy`](../../../../src/com/meesho/stages/deployRingmaster.groovy) catch blocks are inconsistent here: some set `env.msg`, others don't — flagged in `BUGS_AND_IMPROVEMENTS_REPORT.md` as standardisation work.
## What this codebase doesn't have
- No structured logging (no JSON, no key-value pairs, no trace IDs).
- No log level filtering — every `log.info` always prints.
- No log forwarding to anything other than the Jenkins console.
- No "audit log" stream separate from the pipeline log.
If you need to instrument a pipeline run for external observability, the current convention is to POST to a downstream system directly (see how `notify.groovy` calls the Deployment Tracker at lines 108-152). There is no shared metric/event emitter.
docs/wiki/pages/concepts/secrets-and-auth.md
+51
View File
@@ -0,0 +1,51 @@
<!-- m-wiki: type=concept slug=secrets-and-auth topic=concepts base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:src/com/meesho/utilities/addSSHKey.groovy, code:src/com/meesho/stages/buildNode.groovy, code:src/com/meesho/stages/buildGo.groovy, code:src/com/meesho/stages/deployArgoCD.groovy, code:src/com/meesho/stages/securityScan.groovy, code:src/com/meesho/stages/notify.groovy] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: concept. 6 sources.
# Secrets, SSH keys, and downstream auth
How the pipeline authenticates to every external system, in one place. Every entry below comes from a real `withCredentials { ... }` block or env var read in the current code.
## Downstreams + auth mechanisms
| Downstream | Credential / mechanism | Code site |
|---|---|---|
| GitHub clone/push (HTTPS) | `env.GITHUB_CRED = 'svc-devops-meesho'` (`gitUsernamePassword`) | [`gitActions.groovy:13`](../../../../src/com/meesho/utilities/gitActions.groovy), [`buildGo.groovy:300`](../../../../src/com/meesho/stages/buildGo.groovy), [`buildNode.groovy:413`](../../../../src/com/meesho/stages/buildNode.groovy) |
| GitHub clone (SSH, private repos) | `credentialsId: 'ssh-private-key'` written to `./id_github_jenkins` (0600) | [`addSSHKey.groovy:4-5`](../../../../src/com/meesho/utilities/addSSHKey.groovy) |
| Vault (GCP secrets) | `env.vaultToken` string | [`buildNode.groovy:555`](../../../../src/com/meesho/stages/buildNode.groovy), [`constructParam.groovy:170,175`](../../../../src/com/meesho/utilities/constructParam.groovy) (`vault-prd.meeshogcp.in`, `vault-dev.meeshogcp.in`) |
| JFrog (Maven deploy) | `-DuseProdRepo=true` / `-DuseTestRepo=true` Maven profile | [`onlyPushtoJfrog.groovy:43-47`](../../../../vars/onlyPushtoJfrog.groovy) |
| GCP Docker registry | `gcloud auth configure-docker` (SDK ambient auth) | [`buildGo.groovy:126`](../../../../src/com/meesho/stages/buildGo.groovy), [`buildNode.groovy:374`](../../../../src/com/meesho/stages/buildNode.groovy) |
| AWS ECR | `aws ecr get-login-password ... \| docker login --password-stdin` | [`buildNode.groovy:371`](../../../../src/com/meesho/stages/buildNode.groovy), [`buildPython.groovy:93`](../../../../src/com/meesho/stages/buildPython.groovy), [`buildMaven.groovy:566`](../../../../src/com/meesho/stages/buildMaven.groovy), [`buildGradle.groovy:520`](../../../../src/com/meesho/stages/buildGradle.groovy), [`buildPhp.groovy:59`](../../../../src/com/meesho/stages/buildPhp.groovy) |
| ArgoCD | `env.argoCreds` (`usernamePassword`); `argocd login ${env.argoURL}:443` | [`deployArgoCD.groovy:490, 522`](../../../../src/com/meesho/stages/deployArgoCD.groovy) |
| npm registry | `.npmrc` from AWS Secrets Manager or Vault → written to workspace | [`buildNode.groovy:63-68, 337`](../../../../src/com/meesho/stages/buildNode.groovy) |
| SonarQube | `env.sonarToken` string; `withSonarQubeEnv { ... }` against `sonarqube-prd` | [`buildMaven.groovy:245-251`](../../../../src/com/meesho/stages/buildMaven.groovy), [`buildNode.groovy:406`](../../../../src/com/meesho/stages/buildNode.groovy) |
| Ringmaster | `credentialsId: 'ringmaster-token'` (`usernamePassword`) | [`deployRingmaster.groovy:115`](../../../../src/com/meesho/stages/deployRingmaster.groovy), [`notify.groovy:117`](../../../../src/com/meesho/stages/notify.groovy) |
## The SSH key write path
[`addSSHKey.groovy:3-7`](../../../../src/com/meesho/utilities/addSSHKey.groovy) writes the credential file inside `withCredentials { ... }`:
```groovy
withCredentials([sshUserPrivateKey(credentialsId: 'ssh-private-key', keyFileVariable: 'FILE')]) {
sh "cat ${FILE} > ./id_github_jenkins; chmod 600 ./id_github_jenkins; ..."
}
```
There is a **race window** between the `cat` write and the `chmod` — a co-resident process could read the file with default umask permissions for that brief interval. There is also **no cleanup** of `./id_github_jenkins` after use. Both are flagged in `BUGS_AND_IMPROVEMENTS_REPORT.md`.
The key file is **NOT cat'd to stdout / logs** — earlier PR-review concerns (PR #634) about that pattern have been remediated; the current `cat ${FILE} > ./id_github_jenkins` is a file write, not a print. See [`review-learnings.md`](../../../../review-learnings.md) for the historical trail.
## NEVER DO
- **Never print or `cat` an SSH private key to stdout / logs.** Always go through `withCredentials` + a 0600 file. (Graduated rule — see [`CLAUDE.md`](../../../../CLAUDE.md) NEVER DO.)
- **Never hard-code bare IPs as curl/HTTP targets.** Use DNS hostnames. Known existing violation: [`securityScan.groovy:11`](../../../../src/com/meesho/stages/securityScan.groovy) — `final String url = '172.31.5.29:63232/scans'`. Flagged for remediation; do not add new violations.
- **Never pass passwords on the command line** where they'll appear in `ps`. ArgoCD's login at [`deployArgoCD.groovy:494, 525`](../../../../src/com/meesho/stages/deployArgoCD.groovy) does pass `--password ${ARGO_PASSWORD}` on argv — also flagged.
## Where the secrets actually live
| System | Where the credential is provisioned |
|---|---|
| Jenkins credential store | `svc-devops-meesho`, `ssh-private-key`, `argoCreds`, `ringmaster-token`, `sonarToken`, `vaultToken` |
| Vault (`vault-prd.meeshogcp.in` / `vault-dev.meeshogcp.in`) | runtime service secrets, `MEESHO_NPMRC_SECRET` |
| AWS Secrets Manager | `MEESHO_NPMRC_SECRET` (alternate fetch path) |
| GCP IAM service accounts | Docker registry, GKE access — via ambient `gcloud auth` |
docs/wiki/pages/concepts/whitelists.md
+31
View File
@@ -0,0 +1,31 @@
<!-- m-wiki: type=concept slug=whitelists topic=concepts base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:src/com/meesho/utilities/constructParam.groovy] -->
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: concept. 1 source.
# The five whitelist gates
[`constructParam.groovy`](../../../../src/com/meesho/utilities/constructParam.groovy) runs five independent whitelist checks against `Meesho/whitelists`. Each one does a **fresh `git clone`** — there is no caching. A build that hits all five clones the whitelist repo five times.
## The five gates
| Gate | What it controls | Function | Read at |
|---|---|---|---|
| `skip-sonar-whitelist` | Blocks `skip_sonar=true` for Maven on prd unless repo is allowlisted | `skipSonarCheckForbidden` | [`constructParam.groovy:40-57`](../../../../src/com/meesho/utilities/constructParam.groovy) |
| `app-config-disabled` | Blocks `appConfig=false` on stg for Maven/Gradle unless allowlisted | `appConfigDisabledForbidden` | [`constructParam.groovy:62-72`](../../../../src/com/meesho/utilities/constructParam.groovy) |
| `multizone-enabled-repos` | Gates the multi-zone deploy path | `isMultizoneEnabled` | [`constructParam.groovy:29-35`](../../../../src/com/meesho/utilities/constructParam.groovy) |
| `allowedNonDevelopPrDeploymentToInt` | Allows non-`develop` PRs to deploy to `int` | `allowedNonDevelopPrDeploymentToIntRepos` | [`constructParam.groovy:77-83`](../../../../src/com/meesho/utilities/constructParam.groovy) |
| `ValidateCacConfig` | Gates CAC validation on PR build | `ValidateCacConfigForRepo` | [`constructParam.groovy:88-95`](../../../../src/com/meesho/utilities/constructParam.groovy) |
## Why fresh-clone every time
This is **intentional**. The whitelist is the live, authoritative source of which repos opt out of which check. By re-cloning on every call, a DevOps change to the whitelist takes effect on the **next** build in the org without needing a devops-lib release. The cost is ~5× clone latency under GitHub rate-limiting; the benefit is zero release coordination.
## Do NOT add caching
The single most tempting refactor in this code is to cache the clone across the five calls in a single build. Don't — the freshness guarantee is the load-bearing property. See [`docs/tribal-knowledge.md`](../../../tribal-knowledge.md) §1. If you must improve clone performance, do it inside the clone itself (shallow clone, single-branch fetch) without touching the per-call invocation pattern.
## Where the whitelist lives
`https://github.com/Meesho/whitelists.git` (cloned via [`gitActions.groovy`](../../../../src/com/meesho/utilities/gitActions.groovy) helpers). The repo contains one YAML per whitelist name, e.g. `skip-sonar-whitelist.yaml`, `multizone-enabled-repos.yaml`. Each is a flat list of repo names.
To add a repo to a whitelist: open a PR on `Meesho/whitelists`, get a DevOps reviewer to approve, merge. The next pipeline run picks up the change automatically.
docs/wiki/pages/deploy/argocd-sync.md
+58
View File
@@ -0,0 +1,58 @@
<!-- m-wiki: type=concept slug=argocd-sync topic=deploy base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# ArgoCD Sync Sequence
The 4-step ArgoCD deploy sequence in `deployArgoCD.groovy` uses Git as the API: each step commits to a config repo, opens a PR, merges it, and then triggers ArgoCD to pick up the change.
## Where it applies in this repo
`src/com/meesho/stages/deployArgoCD.groovy`
**Step 1 — `update_argo_repo`**
- Reads `deployment.yaml` for the deployable.
- Renders `argoApp.yaml` template → `devops-argo-config/applications_v2/<cluster>/<team>-<app>.yaml`.
- Commits + opens PR against `argoBranch` + merges + deletes branch.
- Sets `value_binding1['helm_values_path']` so the ArgoCD app knows where to find Helm values.
**Step 2 — `refresh_app_of_apps`**
- Runs `argocd app sync <argoIncubator>` (the incubator app-of-apps).
- This causes ArgoCD to discover the new/updated Application manifest from Step 1.
- Uses `--http-retry-max 3 --retry-backoff-duration 1m`.
**Step 3 — `update_helm_repo`**
- Reads `deployment.yaml` + `values_properties.yaml` from `devops-helm-charts`.
- Renders `values.yaml` (or `node-values.yaml`, `go-values.yaml`, etc.) template.
- Enforces canary for sp0/up0 prd services: `enabled=true`, `skipAnalysis=false`, `enableManualPromotion=true`.
- Commits + opens PR against `helmBranch` + merges + deletes branch.
**Step 4 — `refresh_and_sync`**
- `argocd app get --hard-refresh <env>-<app_name>` — forces ArgoCD to re-read Helm values from Git.
- `argocd app sync <env>-<app_name>` — triggers rollout.
- Returns exit code 0 on success; any non-zero result fails the build.
## Why this design
Using Git PRs as the deployment mechanism means every config change is auditable in GitHub history. ArgoCD polls its source repos on a configurable interval, but `hard-refresh` forces an immediate re-read instead of waiting for the poll cycle. The app-of-apps pattern allows ArgoCD to manage thousands of Application resources without manual registration.
Steps 2 (`refresh_app_of_apps`) and 4 (`refresh_and_sync`) are **not interchangeable**. For a first-deploy service, step 2 must run before step 4: without step 2, the ArgoCD Application object created in step 1 hasn't been discovered yet, and step 4 will target a non-existent app. The four-step order is load-bearing.
## Related
- [Deploy ArgoCD](../04-DEPLOY-ARGOCD.md) — broader deploy lifecycle including user input and canary enforcement
- [Dockerfile templates](../08-DOCKERFILE-TEMPLATES.md) — renderTemplate used in both argo and helm steps
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/deploy/ringmaster-integration.md
+56
View File
@@ -0,0 +1,56 @@
<!-- m-wiki: type=concept slug=ringmaster-integration topic=deploy base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# Ringmaster / Turbo-Turtle Integration
`deployRingmaster.run()` routes the CI/CD status callback to either Ringmaster or Turbo-Turtle based on which user triggered the build. These are two separate internal systems that track build and deployment state.
## Where it applies in this repo
`src/com/meesho/stages/deployRingmaster.groovy:run`
**Routing decision:**
```
build_user = getCause(UserIdCause).getUserId()
if build_user == "ringmaster-bot":
POST https://ringmaster-api.meeshogcp.in/api/v1/key/cicd/cd/update
(with Authorization: <ringmaster-token> header)
payload: hot_fix, job_name, build_no, image, applications, job_status, etc.
else (turbo-turtle, or other allowed user):
POST http://turbo-turtle.meeshogcp.in/api/v1/ci/jenkins/callback
payload: repo_name, source_branch, pull_request_number, env, job_name,
sub_job_name, build_number, image_tag, build_detailed_error
```
For `prd`/`int` environments, the base URL is the production Ringmaster API; for `stg`/`ftr`, it uses the admin endpoint.
The Turbo-Turtle payload is written to a temp file first to avoid shell escaping issues with JSON special characters, then passed to `curl -d @<file>`. The temp file is always deleted in a `finally` block.
## Why this design
Ringmaster is the primary orchestration plane for production deployments triggered by human operators via its UI. Turbo-Turtle is the automated CI/CD bot that validates and triggers deployments from PRs. Both need to know when a Jenkins build completes so they can update their state machines.
The `ringmaster-bot` user identity is the distinguishing signal: builds triggered from Ringmaster's UI arrive in Jenkins with that user ID, while Turbo-Turtle-triggered builds arrive with the `turbo-turtle` user ID. The string `"ringmaster-bot"` is load-bearing — if Ringmaster ever renames its bot user, callbacks silently fall through to the Turbo-Turtle endpoint. Never change this string without coordinating with the Ringmaster team.
The Turbo-Turtle JSON payload is written to a temp file (`cicd_payload_${BUILD_NUMBER}_${ts}.json`) to avoid shell escaping failures when payload fields contain single quotes, slashes, or error messages with special characters. The temp file is deleted in a `finally` block.
## Related
- [Notifications](../10-NOTIFICATIONS.md) — notify.groovy calls deployRingmaster
- [Architecture](../01-ARCHITECTURE.md) — allowedUsers list that includes both bot users
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/infra/node-pool-selection.md
+54
View File
@@ -0,0 +1,54 @@
<!-- m-wiki: type=concept slug=node-pool-selection topic=infra base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# GKE Node Pool Selection
`nodePoolSelection.run(memory_request, cpu_request, priority_v2)` computes the GKE node pool label for the deployed application (not the Jenkins build agent). The label is written to the Helm values as `nodeSelectorValue`.
## Where it applies in this repo
`src/com/meesho/utilities/nodePoolSelection.groovy:run`
**Selection logic (prd only):**
1. Parse `memory_request` (Mi or Gi) and `cpu_request` (m or whole cores) to numeric values.
2. Compute `ratio = memory_mb / cpu_millicores`.
3. For **non-critical priorities** (cp1-cp3, up1-up3, sp1-sp3): node name is `{mega|sumo}{tetra|duo}lite`.
- `cpu_req >= 2200m` → `sumo`; else `mega`.
- `ratio >= 2.5` → `tetra`; else `duo`.
4. For **critical priorities** (sp0, up0, cp0 — not explicitly listed but implied): node name is `{compact|mega|sumo}{duo|tetra|octa}`.
- `cpu_req >= 2200m` → `sumo`; `1000m-2199m` → `mega`; `<1000m` → `compact`.
- `ratio > 5.5` → `octa`; `>= 2.5` → `tetra`; else `duo`.
5. BU-specific overrides for `supply` and `demand` apply to certain `{nodevalue, ratiovalue}` combinations.
**Non-prd environments:**
- `int` → `preprod-cost-optimized`
- `stg`/`ftr`/`dev` → `{BU}-shared`
The result is set as `value_binding1['nodeSelectorValue']` in `deployArgoCD.update_helm_repo`.
## Why this design
GKE node pools are heterogeneous — some are memory-optimized, some CPU-optimized. Placing services on the wrong pool wastes resources or causes throttling. The automated selection reduces per-team cognitive load: teams declare their resource needs in `deployment.yaml` and the pipeline finds the best matching pool.
The BU-specific overrides exist because certain BUs have limited availability of some pool types in the region — the fallback to `tetra` avoids scheduling failures on unavailable pools.
## Related
- [Deploy ArgoCD](../04-DEPLOY-ARGOCD.md) — nodeSelectorValue is computed in update_helm_repo
- [Infra pods](../09-INFRA-PODS.md) — build agent pod selection (separate from app node pool)
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/policy/cac-validation.md
+49
View File
@@ -0,0 +1,49 @@
<!-- m-wiki: type=concept slug=cac-validation topic=policy base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# CAC (Config-as-Code) Validation
CAC validation runs `resources/com/meesho/validate_configs_v2.py` on PR builds for repos opted in via the `ValidateCacConfig` whitelist. It validates application YAML configs against a schema to catch misconfigurations before they reach staging.
## Where it applies in this repo
`src/com/meesho/utilities/constructParam.groovy:ValidateCacConfigForRepo`
`resources/com/meesho/validate_configs_v2.py`
**Trigger condition:**
- Build is a PR build (`env.CHANGE_ID` is set).
- `ValidateCacConfigForRepo(config.ValidateConfig, repo_name)` returns `true`.
- Either the repo is in the `ValidateCacConfig.yaml` whitelist, OR
- `config.yaml` has `ValidateConfig: true`.
**What gets validated:**
The script reads `configs/<module>/application-*.yml` files from the service repo and checks them against the CAC schema. Validation errors fail the PR build — the commit cannot be merged until the config is corrected.
Used primarily by Go and Maven services. Node, Python, and PHP builders do not call CAC validation.
## Why this design
Application config files (`application-stg.yml`, `application-prd.yml`) define Spring/Gin/etc. runtime config. A typo or wrong data type in these files doesn't fail compilation but causes a runtime crash after deployment. CAC validation catches these at PR time — when the feedback loop is cheapest.
The validation script is bundled as a library resource (`resources/com/meesho/validate_configs_v2.py`) so it travels with the library version rather than requiring a separate checkout.
## Related
- [Whitelist system](whitelist-system.md) — ValidateCacConfig whitelist that gates this
- [Config policy](../06-CONFIG-POLICY.md) — constructParam context
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/policy/multi-tenancy.md
+50
View File
@@ -0,0 +1,50 @@
<!-- m-wiki: type=concept slug=multi-tenancy topic=policy base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# Multi-Tenancy: BU and Team Mapping
Meesho's CI/CD pipeline partitions services by business unit (BU) and team. `buTeamMapping.groovy` translates full BU/team names to short initials used in Helm chart paths, Docker image paths, and ArgoCD namespaces.
## Where it applies in this repo
`src/com/meesho/utilities/buTeamMapping.groovy`
**`get_bu_initials(bu)`** maps full BU names to short abbreviations:
- `supply` → `su`, `demand` → `de`, `central` → `ce`, `dataengg` → `da`, `datascience` → `ds`, `mcache` → `mc`, `infra` → `in`
**`get_team_initials(team)`** maps team slug to a short initial (typically first 2-4 chars of the slug).
These initials are used to construct:
- Helm chart paths: `devops-helm-charts/<helmChartsPath>/<buIni>/<teamIni>/<app_name>/`
- ArgoCD app names: `<env>-<teamIni>-<app_name>` (or similar)
- ArgoCD namespace: `argocd-<bu>-prd` (GCP prd)
- GCPProject: `meesho-<bu>-prd-0622` (from `constructParam.run()`)
**`config.yaml` fields:**
- `bu`: must be one of the valid BU values (`supply`, `demand`, `central`, `dataengg`, `datascience`, `mcache`, `infra`).
- `team`: must match a known team slug in `buTeamMapping`.
- `validateBuTeam.groovy` in `src/com/meesho/stages/` validates the BU/team combination before deployment.
## Why this design
Multi-tenancy isolation is enforced structurally — a `supply` service's Helm values live in a separate directory from `demand`. If a service mis-declares its BU, its Helm charts and ArgoCD apps land in the wrong directory hierarchy, which causes the deploy to fail or overwrite another team's app. The initials mapping abstracts this from individual service owners.
## Related
- [Config policy](../06-CONFIG-POLICY.md) — bu/team are required fields in config.yaml
- [Deploy ArgoCD](../04-DEPLOY-ARGOCD.md) — buini/teamini used in helm path construction
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/policy/whitelist-system.md
+50
View File
@@ -0,0 +1,50 @@
<!-- m-wiki: type=concept slug=whitelist-system topic=policy base-sha=28f54cf7bef9 generated-at=2026-05-12T00:00:00+00:00 sources=[] -->
> Generated 2026-05-12 at base-sha 28f54cf7bef9. Type: concept. 0 sources.
# Whitelist System
devops-lib enforces five policy gates at runtime by cloning `Meesho/whitelists` on GitHub and reading a YAML file. This allows DevOps to grant or revoke exceptions without changing any service code or the library itself.
## Where it applies in this repo
`src/com/meesho/utilities/constructParam.groovy:getWhitelistedRepos`
**Five whitelist files:**
| File in Meesho/whitelists | Gate method | What it controls |
|---|---|---|
| `skip-sonar-whitelist.yaml` | `skipSonarCheckForbidden()` | Allows `skip_sonar: true` in config.yaml for Maven prd builds |
| `app-config-disabled.yaml` | `appConfigDisabledForbidden()` | Allows `appConfigEnabled: false` in stg for Maven/Gradle |
| `multizone-enabled-repos.yaml` | `isMultizoneEnabled()` | Marks deployables that must go via Ringmaster (not Jenkins) |
| `allowedNonDevelopPrDeploymentToInt.yaml` | `allowedNonDevelopPrDeploymentToIntRepos()` | Allows feature-branch PRs to target `main` for int deploy |
| `ValidateCacConfig.yaml` | `ValidateCacConfigForRepo()` | Opts repo into CAC config validation during PR builds |
Each YAML file has a `repos:` list. `getWhitelistedRepos(fileName)` clones the entire `Meesho/whitelists` repo into `whitelist/` in the workspace, reads `whitelist/<fileName>.yaml`, and returns the `repos` list as a `Set`.
`getWhitelistedDeployable(fileName, keyName)` is a variant that reads an arbitrary key from the YAML — used for `multizone_enabled_deployables` which is a list under a non-standard key.
## Why this design
Inline conditionals in stage code would require PRs to `devops-lib` for every exception. The whitelist approach lets DevOps grant exceptions by merging a one-line YAML change to `Meesho/whitelists` — visible in its own audit trail, immediately effective on the next build, and independent of the library release cycle.
The cost is a fresh `git clone` per whitelist check per build. Five checks = five clones. On a slow network or under GitHub rate limiting, this adds measurable latency. This freshness guarantee is by design — never refactor `getWhitelistedRepos` to cache the clone across calls without confirming the freshness requirement is no longer needed.
## Related
- [Config policy](../06-CONFIG-POLICY.md) — all five gates in context
- [CAC validation](cac-validation.md) — triggered by the ValidateCacConfig whitelist
## Sources
(no raw/ sources at bootstrap)
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->
---
[← Wiki index](../../index.md)
<!-- atomic: keep this page ≤600 words. New scope → new concept page that builds on this one. Do not append paragraphs here. -->
docs/wiki/pages/security/security-overview.md
+90
View File
@@ -0,0 +1,90 @@
<!-- m-wiki: type=concept slug=security-overview topic=security base-sha=d6708eca4236 generated-at=2026-05-12T12:00:00+00:00 sources=[docs/SECURITY.md] -->
> Generated 2026-05-12 at base-sha d6708eca4236. Type: concept. 1 source.
# Security Overview
devops-lib is a Jenkins Shared Library — not a web service. It has no HTTP endpoints, no user-facing auth, and no persistent storage. Security properties concern how secrets are handled, what trust boundaries exist, and what rules new code must follow.
## Trust boundaries
### Build trigger gate
Every build is hard-rejected unless triggered by one of:
- `ringmaster-bot` — Ringmaster's automated trigger
- `turbo-turtle` — Turbo-Turtle's CI callback trigger
- A hardcoded `allowedUsers` list of DevOps engineer email addresses
The gate runs at `vars/eksCICD.groovy:call` before any pipeline logic. There is no warning mode — unauthorized triggers abort immediately.
### Supply chain boundary (highest risk)
`devops-lib@main` is loaded via `@Library('devops-lib@main')` by every Meesho microservice on every build. **A malicious or buggy merge to `main` affects all 100+ consumer services' CI/CD pipelines.** Any change to `vars/eksCICD.groovy` or `src/com/meesho/utilities/constructParam.groovy` must be treated as Tier-1 code — these files control every build.
### Whitelist repo boundary
Policy exceptions (sonar skip, multizone, AppConfig, CAC) are fetched from `Meesho/whitelists` at build time via the `cicd-github-app` credential. If the whitelist repo is compromised, an attacker could grant or revoke policy exceptions for any service. See [ADR-0003](../adr/adr-index.md) for why exceptions live in a separate repo.
## Credential handling
All secrets are injected via Jenkins `withCredentials` — never hardcoded, never in `env.*` beyond the immediate operation:
| Credential ID | Used for |
|---|---|
| `cicd-github-app` | Cloning `Meesho/whitelists`, `devops-argo-config`, `devops-helm-charts` |
| `svc-devops-meesho` | GitHub API, JFrog Artifactory |
| `ringmaster-token` | Ringmaster callback API |
| `argocd-{bu}-prd-creds` / `argocd-dev-creds` | ArgoCD CLI login |
| `vault-prd-token` / `vault-dev-token` | Vault secret fetch |
| `sonar-token-prod` / `sonar-token-{bu}-dev` | SonarQube analysis |
**`set +x` guard**: `deployArgoCD.groovy:493` uses `set +x` before any `sh()` that includes a credential argument. Without this, Jenkins echoes the full shell command — including the credential value — to the build log.
**Vault token lifecycle**: `env.VAULT_TOKEN` is set temporarily at `src/com/meesho/stages/buildNode.groovy:548–553` and cleared immediately after the Vault fetch completes. It is not stored beyond the immediate use.
## Outbound calls
All external calls go FROM Jenkins agents TO external services. Protocol classification:
| Service | Protocol | Risk |
|---|---|---|
| ArgoCD | HTTPS + gRPC | Low — `set +x` guards password |
| Ringmaster | HTTPS | Low — auth via `ringmaster-token` |
| SonarQube, Vault, GitHub | HTTPS | Low |
| Turbo-Turtle, Deployment Tracker | **HTTP** (plain) | Low — internal VPC, accepted risk |
| Security scanner (`172.31.5.29:63232`) | **HTTP** (plain) | Low — internal, hardcoded IP |
## Security rules for new code
1. **Credentials**: Use `withCredentials` only. Never assign credential values outside a `withCredentials` block. Never interpolate credentials into log statements.
2. **Shell guard**: Use `set +x` before any `sh()` that includes a credential as an argument.
3. **Policy enforcement**: Never add inline repo-level policy exceptions — all exceptions go through `Meesho/whitelists` (see [ADR-0003](../adr/adr-index.md)).
4. **Supply chain hygiene**: Extra scrutiny for `vars/eksCICD.groovy` and `src/com/meesho/utilities/constructParam.groovy` — changes affect every Meesho microservice build.
5. **DinD image**: Must use the internal GAR-hosted DinD image, not Docker Hub `docker:N-dind`.
## Data classification
devops-lib handles no end-user PII. All data is build metadata (repo name, image tag, environment, team). Credential values are never logged.
## Known security debt
| ID | Gap | Severity |
|---|---|---|
| SEC-DL-001 | `allowedUsers` list has no expiry — stale access risk for departed engineers | Low |
| SEC-DL-002 | No `.github/CODEOWNERS` — supply chain protection depends on repo settings not auditable from source | Medium |
| SEC-DL-003 | Security scanner endpoint hardcoded as `172.31.5.29:63232` — silent failure if IP changes | Low |
| SEC-DL-004 | `env.VAULT_TOKEN` briefly in Jenkins serialized state during fetch | Low |
| SEC-DL-005 | Turbo-Turtle and Deployment Tracker callbacks over plain HTTP | Low |
Full details: [`docs/SECURITY.md`](../../../SECURITY.md)
## Related concepts
- [ADR index](../adr/adr-index.md) — architectural decisions that drive security properties (ADR-0003, ADR-0004, ADR-0006)
- [Whitelist system](../policy/whitelist-system.md) — policy exception enforcement
- [Ringmaster integration](../deploy/ringmaster-integration.md) — trigger gate flow
- [Config and Policy](../06-CONFIG-POLICY.md) — constructParam and credential injection
## Notes
<!-- Anything below is human-owned. wiki-init never reads or modifies content under this heading. -->