85 lines
7.3 KiB
Markdown
85 lines
7.3 KiB
Markdown
<!-- m-wiki: type=top-level slug=cross-cutting topic=null base-sha=5399a5ddc36b generated-at=2026-05-21 sources=[code:src/com/meesho/utilities/constructParam.groovy, code:src/com/meesho/utilities/nodePoolSelection.groovy, code:src/com/meesho/utilities/gitActions.groovy, code:src/com/meesho/stages/deployArgoCD.groovy, code:src/com/meesho/stages/hotFix.groovy, code:vars/log.groovy] -->
|
|
|
|
> Generated 2026-05-21 at base-sha 5399a5ddc36b. Type: top-level. 6 sources.
|
|
|
|
# Cross-cutting patterns
|
|
|
|
## Retries
|
|
|
|
| Caller | Downstream | Retry |
|
|
|---|---|---|
|
|
| [`deployArgoCD.groovy:506,526`](../../../src/com/meesho/stages/deployArgoCD.groovy) | ArgoCD sync | `argocd app sync --http-retry-max 3 --retry-backoff-duration 1m` |
|
|
| [`buildNode.groovy:389`](../../../src/com/meesho/stages/buildNode.groovy), [`buildGo.groovy:172`](../../../src/com/meesho/stages/buildGo.groovy) | Docker registry push | `retryDockerPush` — 5 attempts with 3s sleep between |
|
|
| `buildMaven`, `buildGradle`, `buildPython`, `buildPhp` | various | **No retry** |
|
|
| [`securityScan.groovy:12`](../../../src/com/meesho/stages/securityScan.groovy) | In-house scanner | **No retry** |
|
|
| [`notify.groovy:108-152`](../../../src/com/meesho/stages/notify.groovy) | Deployment Tracker | **No retry** — single failed POST = missed deploy in the dashboard |
|
|
|
|
`BUGS_AND_IMPROVEMENTS_REPORT.md` lists "no retry mechanism" as P1; the reconciled-in nuance is that retry IS present for ArgoCD sync + Docker push, but not for most other downstream calls.
|
|
|
|
## BU multi-tenancy
|
|
|
|
`env.BU` and `config.bu` flow from CAC config (read once by [`constructParam.groovy`](../../../src/com/meesho/utilities/constructParam.groovy)) and drive:
|
|
|
|
| Use | Code site |
|
|
|---|---|
|
|
| ArgoCD namespace | `argocd-${env.BU}-prd` — [`constructParam.groovy:318`](../../../src/com/meesho/utilities/constructParam.groovy) |
|
|
| GCP cluster name | `k8s-${env.BU}-prd-ase1` — [`constructParam.groovy:308`](../../../src/com/meesho/utilities/constructParam.groovy) |
|
|
| GCP project | `meesho-${config.bu}-prd-0622` — [`constructParam.groovy:178`](../../../src/com/meesho/utilities/constructParam.groovy) |
|
|
| Helm chart path | `${env.helmChartsPath}/${config.bu}/...` — [`helmGenerator.groovy:150`](../../../src/com/meesho/stages/helmGenerator.groovy), [`deployArgoCD.groovy:293`](../../../src/com/meesho/stages/deployArgoCD.groovy) |
|
|
| Non-prd node pool | `${env.BU}-shared` (all `dev`/`ftr`/`stg` traffic for the BU collapses to one pool) — [`nodePoolSelection.groovy:133-135`](../../../src/com/meesho/utilities/nodePoolSelection.groovy) |
|
|
|
|
**Implication for non-prd capacity planning:** a single noisy service in `bu=supply` degrades every other `supply` service on staging, because they all share `supply-shared`. See [`docs/tribal-knowledge.md`](../../tribal-knowledge.md) §8.
|
|
|
|
## Feature flags / toggle conditions
|
|
|
|
| Flag | Set by | Effect | Read at |
|
|
|---|---|---|---|
|
|
| `env.hot_fix` | [`hotFix.groovy:11`](../../../src/com/meesho/stages/hotFix.groovy) | Skips Sonar + quality gate; sets canary `skipAnalysis` | [`buildGo.groovy:25-28`](../../../src/com/meesho/stages/buildGo.groovy), [`buildMaven.groovy:37-40`](../../../src/com/meesho/stages/buildMaven.groovy), [`deployArgoCD.groovy:355`](../../../src/com/meesho/stages/deployArgoCD.groovy) |
|
|
| `config.skip_sonar` | Service Jenkinsfile param | Skips quality-gate check (gated by whitelist for Maven prd) | [`buildMaven.groovy:18,240-243`](../../../src/com/meesho/stages/buildMaven.groovy), [`constructParam.groovy:51-56`](../../../src/com/meesho/utilities/constructParam.groovy) |
|
|
| `config.skip_security_scan` | Service Jenkinsfile param | Skips POST to security-scan endpoint | [`securityScan.groovy:7-9`](../../../src/com/meesho/stages/securityScan.groovy) |
|
|
| `config.push_to_jfrog` | Service Jenkinsfile param | Allow non-default-branch JFrog push (default branches: master, main) | [`buildMaven.groovy:19,377`](../../../src/com/meesho/stages/buildMaven.groovy) |
|
|
| `config.push_to_s3` | Service Jenkinsfile param | Allow non-default-branch S3 push (default branches: master, main, gcp-main, gcp-master) | [`buildMaven.groovy:20,462`](../../../src/com/meesho/stages/buildMaven.groovy) |
|
|
| `env.INFRA_ENV == 'toolchain'` | CAC config | Skips Docker push, uses latest-tag logic | [`buildGo.groovy:40-58`](../../../src/com/meesho/stages/buildGo.groovy), [`buildNode.groovy:215-239`](../../../src/com/meesho/stages/buildNode.groovy) |
|
|
| `env.CHANGE_ID` | GitHub Branch Source plugin | PR-build detection; remaps `cicd_environment` (`prd→int` for main/master PRs, `stg→ftr` for develop PRs) | [`constructParam.groovy:107-110`](../../../src/com/meesho/utilities/constructParam.groovy) |
|
|
|
|
**Do not use `env.BRANCH_NAME =~ /PR-/`** as a PR-build detector — it breaks on re-triggered builds and non-GitHub SCMs.
|
|
|
|
## Parallel execution
|
|
|
|
The pipeline is **almost entirely linear**. The one confirmed `parallel { }` block is in multi-module Go builds at [`buildGo.groovy:161-179`](../../../src/com/meesho/stages/buildGo.groovy):
|
|
|
|
```groovy
|
|
for (m in modules) {
|
|
moduleBuilds["build-${moduleName}"] = { ... }
|
|
}
|
|
parallel moduleBuilds
|
|
```
|
|
|
|
Anything called from inside that closure must respect the `@NonCPS` rule (see [`constructTemplate.groovy:13-20`](../../../src/com/meesho/utilities/constructTemplate.groovy) — the template engine wrapper is `@NonCPS` because `SimpleTemplateEngine` is non-serialisable).
|
|
|
|
## Exception handling — inconsistent on purpose-ish
|
|
|
|
`catch (Exception e)` appears ~87 times across stages. Three observed patterns:
|
|
|
|
| Pattern | Example | Behaviour |
|
|
|---|---|---|
|
|
| Catch + log + **rethrow** | [`buildNode.groovy:20-26`](../../../src/com/meesho/stages/buildNode.groovy) | Hard-fail the stage. Most common. |
|
|
| Catch + log + **swallow** (loop body) | [`deployArgoCD.groovy:103-106`](../../../src/com/meesho/stages/deployArgoCD.groovy) | Continue with the next deployment in the loop. Intentional for multi-deploy resilience. |
|
|
| Catch + log + **swallow** (silent fall-through) | [`buildNode.groovy:199-200`](../../../src/com/meesho/stages/buildNode.groovy) | Recovers with a sensible default. Use with caution — readers don't always realise the stage "succeeded" while masking a real error. |
|
|
|
|
When adding new error handling, prefer the rethrow pattern unless the loop-resilience semantics are an explicit requirement.
|
|
|
|
## Validation: `validate_configs.py`
|
|
|
|
[`resources/com/meesho/validate_configs.py`](../../../resources/com/meesho/validate_configs.py) (1207 lines) is the monolithic CAC schema validator, invoked from Groovy via `sh`. Key validation classes:
|
|
|
|
- **Schema** ([line 949](../../../resources/com/meesho/validate_configs.py)) — `application-{dev,int,stg,prd}.yml` against `application-schema.yml`.
|
|
- **Cross-env endpoint check** ([line 1010](../../../resources/com/meesho/validate_configs.py)) — prevents stg configs from referencing prd endpoints.
|
|
- **Secrets detection** ([line 1099](../../../resources/com/meesho/validate_configs.py)) — via `detect_secrets` library.
|
|
- **DB URL patterns** ([lines 41-59](../../../resources/com/meesho/validate_configs.py)) — PostgreSQL/MySQL/MongoDB/Redis/SQLite URI shapes.
|
|
- **Zookeeper endpoint patterns** — env-specific allow-lists.
|
|
|
|
`validate_configs_v2.py` (1261 lines) is the eventual replacement; both are referenced today.
|
|
|
|
See also: [concepts/whitelists](concepts/whitelists.md), [concepts/secrets-and-auth](concepts/secrets-and-auth.md), [concepts/observability](concepts/observability.md).
|