157 lines
8.1 KiB
Markdown
157 lines
8.1 KiB
Markdown
<!--
|
|
Auto-generated by /meesho-init.
|
|
Regenerate by deleting this file and re-running the skill.
|
|
Do not duplicate content from docs/architecture.md, docs/tribal-knowledge.md,
|
|
docs/acronyms.md, or review-learnings.md — link to them instead.
|
|
-->
|
|
|
|
# devops-lib — Claude guide
|
|
|
|
Jenkins shared library that backs every Meesho service's CI/CD pipeline.
|
|
Loaded by Jenkins as a Global Library; consumed by service repos through a
|
|
`@Library('devops-lib')` import in their `Jenkinsfile`. There is **no local
|
|
build, test, or run command** — the library only executes inside Jenkins.
|
|
|
|
## What this repo is
|
|
|
|
- Code is **Groovy** (Jenkins CPS-style, with `@NonCPS` islands).
|
|
- `vars/*.groovy` are Jenkins shared-library **globals** — the entry points
|
|
service repos call (e.g. `buildPipeline { ... }`, `eksCICD { ... }`).
|
|
- `src/com/meesho/stages/**.groovy` — stage implementations dispatched from
|
|
the globals (`buildMaven`, `buildGo`, `deployArgoCD`, `notify`, …).
|
|
- `src/com/meesho/utilities/**.groovy` — shared helpers (`constructParam`,
|
|
`gitActions`, `nodePoolSelection`, `constructTemplate`).
|
|
- `resources/com/meesho/**` — per-language `Dockerfile`, `*-deployment.yaml`,
|
|
`*-values.yaml` templates rendered into the service repo at deploy time.
|
|
- `resources/org/meesho/**-pod.yaml` — Jenkins build-agent pod templates
|
|
selected by `env.INFRA_ENV` (`dev` / `stg` / `prd`).
|
|
|
|
## Architecture
|
|
|
|
Long-form architecture and data flow lives in
|
|
[`docs/architecture.md`](docs/architecture.md). Read it before changing any
|
|
file under `src/com/meesho/stages/` or `vars/`.
|
|
|
|
Pre-existing context worth reading:
|
|
|
|
- [`docs/acronyms.md`](docs/acronyms.md) — repo-specific acronyms (BU,
|
|
BUILDKIT, CAC, GCPP, INFRA, …).
|
|
- [`docs/tribal-knowledge.md`](docs/tribal-knowledge.md) — load-bearing
|
|
non-obvious conventions (whitelist git clones, JVM memory auto-calc, canary
|
|
enforcement, `ringmaster-bot` user-id switch, etc.).
|
|
- [`review-learnings.md`](review-learnings.md) — PR-review-derived rules,
|
|
some graduated into the NEVER DO section below.
|
|
- [`BUGS_AND_IMPROVEMENTS_REPORT.md`](BUGS_AND_IMPROVEMENTS_REPORT.md) —
|
|
catalogue of known bugs / debt; do not silently "fix" these without
|
|
reading the linked PR history.
|
|
|
|
## Quick reference
|
|
|
|
| Task | Command |
|
|
|---|---|
|
|
| Sanity-check Groovy syntax | `groovy -e 'load "vars/buildPipeline.groovy"'` (not wired in CI; use only as a local lint) |
|
|
| Validate Helm value templates | runs inside Jenkins via `validate_configs.py` (no standalone CLI) |
|
|
| Run the library | Cannot run locally — push a branch, point a Jenkins job at `@Library('devops-lib@<branch>')`, trigger from a service repo |
|
|
|
|
**No `Makefile`, no `build.gradle`, no `pom.xml`, no `Dockerfile`, no
|
|
`Jenkinsfile`, no `package.json`, no `.github/workflows/` at the repo root.**
|
|
A Jenkins job test-loads this library; there is no project-local equivalent.
|
|
|
|
**No test suite.** `BUGS_AND_IMPROVEMENTS_REPORT.md` flags this as a known
|
|
high-priority gap. Do not fabricate `./gradlew test` or similar — they do not
|
|
exist.
|
|
|
|
## Stack & tools
|
|
|
|
- **Groovy** — Jenkins CPS engine. Beware: any `@NonCPS` method must not be
|
|
called across a serialisation boundary (e.g., inside a `parallel` closure)
|
|
without the closure itself being `@NonCPS`.
|
|
- **Jenkins shared-library layout** — `vars/<name>.groovy` exposes `<name>`
|
|
as a global step; `src/com/meesho/**.groovy` is classpath-loaded.
|
|
- **Helm / ArgoCD** — service deploys go through
|
|
`deployArgoCD.groovy` → `update_helm_repo` → `refresh_app_of_apps` →
|
|
`refresh_and_sync`. Step order is load-bearing
|
|
([tribal-knowledge §10](docs/tribal-knowledge.md)).
|
|
- **Turbo-Turtle / Ringmaster** — `deployRingmaster.groovy` routes callbacks
|
|
based purely on `getCause(UserIdCause).getUserId() == "ringmaster-bot"`.
|
|
Renaming that user id silently breaks the routing.
|
|
- **Validation** — `resources/com/meesho/validate_configs.py` (1207 lines) is
|
|
a monolithic Python validator invoked from Groovy via `sh`.
|
|
- **Whitelists** — `getWhitelistedRepos()` does a fresh `git clone` of
|
|
`Meesho/whitelists` **on every call** (no caching). Five whitelist gates
|
|
each clone independently — this is intentional, do not refactor.
|
|
|
|
## Critical conventions
|
|
|
|
1. **`vars/` is the public API.** Adding a new entry point means adding a new
|
|
file under `vars/`. Renaming or removing a `vars/` global is a breaking
|
|
change for every service Jenkinsfile.
|
|
2. **`build_tool` switch lives in `src/com/meesho/stages/buildObjHelper.groovy`.**
|
|
Supported values today are `maven`, `gradle`, `docker`, `python-*`,
|
|
`node-*`, `go*`, `php`. There is no `sbt`, no `rust`. Unknown values fall
|
|
through to `defaultBuild` silently.
|
|
3. **`@NonCPS` rule.** `constructTemplate._construct()` is annotated
|
|
`@NonCPS` because it uses `groovy.text.SimpleTemplateEngine` (Java,
|
|
non-serialisable). Keep it `@NonCPS` and call it from a CPS-safe wrapper.
|
|
4. **JVM memory auto-calc.** `update_helm_repo` derives `xms == xmx ==
|
|
memory_limit * 0.5`. Do not hard-code `-Xmx` in `JAVA_OPTS`.
|
|
5. **Canary is mandatory for `priority_v2: sp0` / `up0` in prd.** No
|
|
whitelist, no bypass. The build fails fast before the Helm update.
|
|
6. **PR vs branch detection.** Use `env.CHANGE_ID` (set by the GitHub Branch
|
|
Source plugin), never `env.BRANCH_NAME =~ /PR-/`.
|
|
7. **Hard-coded `'Meesho'` org.** Many helpers in `gitActions.groovy` /
|
|
`constructParam.groovy` embed the GitHub org name as a literal — keep this
|
|
stable. Do not parametrise without a coordinated rollout.
|
|
8. **Pod selection.** `env.INFRA_ENV` selects the Jenkins agent pod template
|
|
via `libraryResource("org/meesho/${env.INFRA_ENV}-pod.yaml")`. Don't add
|
|
inline `podTemplate` blocks — they bypass the central agent inventory.
|
|
|
|
## NEVER DO
|
|
|
|
These rules come from PR-review history (`review-learnings.md`) and
|
|
post-incident notes (`docs/tribal-knowledge.md`). Each links to its source.
|
|
|
|
- **Never refactor `getWhitelistedRepos` to cache clones across calls**
|
|
without confirming the freshness guarantee is no longer required
|
|
([tribal-knowledge §1](docs/tribal-knowledge.md)).
|
|
- **Never rename `"ringmaster-bot"`** in `deployRingmaster.groovy` without
|
|
coordinating with the Ringmaster team — it's a load-bearing string
|
|
([tribal-knowledge §2](docs/tribal-knowledge.md)).
|
|
- **Never set `xms` / `xmx` manually in service Helm values** — the pipeline
|
|
computes them from `memory_limit`; manual values collide
|
|
([tribal-knowledge §4](docs/tribal-knowledge.md)).
|
|
- **Never print or `cat` an SSH private key to stdout / logs**
|
|
(review-learnings `P0_NO_PRIVATE_KEY_LEAK`, PR #634). Always
|
|
`withCredentials { ... }` and write to a 0600 file.
|
|
- **Never hard-code bare IPs as curl/HTTP targets in pipeline code.**
|
|
(review-learnings `P0_HARDCODED_IP_IN_PIPELINE`, PR #681; known existing
|
|
exception: `securityScan.groovy:11` flagged for remediation.) Use
|
|
DNS-resolvable hostnames.
|
|
- **Never bypass canary on `sp0`/`up0` prd deploys**
|
|
([tribal-knowledge §5](docs/tribal-knowledge.md)).
|
|
- **Never inline the Turbo-Turtle JSON payload in `curl` differently from
|
|
the current `-d '$newCICD_JSON'` pattern** without re-verifying bash
|
|
escaping ([tribal-knowledge §6](docs/tribal-knowledge.md)).
|
|
- **Never use `--no-verify` on `git commit`** — pre-commit hooks include
|
|
TruffleHog secret scanning and CAC validation. Bypassing is a P0
|
|
policy violation.
|
|
- **Never commit to `main` or `develop` directly** — those branches are
|
|
protected; all changes go via PR.
|
|
|
|
## Where to ask
|
|
|
|
- `#ci-cd-status` Slack channel — pipeline failures, library questions.
|
|
- `@maintainer` Slack handle is required in every pipeline `param` map
|
|
(consumed by `src/com/meesho/stages/notify.groovy:9`).
|
|
|
|
## Pointers
|
|
|
|
- [docs/architecture.md](docs/architecture.md) — full module-by-module map.
|
|
- [docs/acronyms.md](docs/acronyms.md) — repo-specific abbreviations.
|
|
- [docs/tribal-knowledge.md](docs/tribal-knowledge.md) — non-obvious patterns.
|
|
- [review-learnings.md](review-learnings.md) — PR-review-derived rules,
|
|
graduation candidates for this file's NEVER DO section.
|
|
- [review.md](review.md) — review process / rubric.
|
|
- [BUGS_AND_IMPROVEMENTS_REPORT.md](BUGS_AND_IMPROVEMENTS_REPORT.md) — known
|
|
bug catalogue; check before "fixing" anything that looks suspect.
|