15 KiB
SECURITY.md
Scope: devops-lib is a Jenkins Shared Library, not a deployable web service. It has no HTTP endpoints of its own. This document covers the security properties of the library's CI/CD execution: how secrets are handled, what trust boundaries exist, what security rules new code must follow, and known gaps.
Authentication
devops-lib has no user-facing HTTP endpoints and performs no JWT or session validation. Authentication is enforced at two points:
Build trigger gate
vars/eksCICD.groovy:12–26 validates that every Jenkins build is triggered by an authorized caller:
ringmaster-bot— Ringmaster's automated triggerturbo-turtle— Turbo-Turtle's CI callback triggerallowedUsers— a hardcoded list of DevOps engineer email addresses for emergency access
Unauthorized triggers are hard-rejected before any pipeline logic runs.
Outbound credential injection
All outbound API calls use Jenkins' withCredentials binding — credentials are never hardcoded in source. Jenkins masks bound variables in console output automatically. Key credential IDs:
| Credential ID | Used for | Scope |
|---|---|---|
cicd-github-app |
Cloning Meesho/whitelists, devops-argo-config, devops-helm-charts |
All builds |
svc-devops-meesho |
GitHub API, JFrog Artifactory | Build + deploy |
ringmaster-token |
Ringmaster callback API | Notify stage |
argocd-{bu}-prd-creds / argocd-dev-creds |
ArgoCD CLI login | Deploy stage |
vault-prd-token / vault-dev-token |
Vault secret fetch | Node builds only |
sonar-token-prod / sonar-token-{bu}-dev |
SonarQube analysis | Build stages |
Trust Boundaries
Build trigger trust boundary
| Layer | What happens | Where | Confidence |
|---|---|---|---|
| Ringmaster / Turbo-Turtle | Validates human approval, triggers Jenkins build | Upstream (Ringmaster infra) | docs-referenced |
devops-lib eksCICD |
Validates trigger source against allowedUsers list |
vars/eksCICD.groovy:12 |
code-confirmed |
| Jenkins pipeline | Executes build stages with injected credentials | Jenkins agents (GKE pods) | code-confirmed |
Trust assumption: devops-lib assumes that any build triggered by ringmaster-bot or turbo-turtle has already been approved by the Ringmaster/Turbo-Turtle authorization flow. It does NOT re-validate the approval — it trusts the trigger identity.
Supply chain trust boundary (CRITICAL)
devops-lib@main is loaded via @Library('devops-lib@main') by every Meesho microservice on every build. A malicious or buggy merge to main is an immediate supply chain attack on all 100+ consumer services' CI/CD pipelines.
- Whoever can merge to
devops-lib@maincontrols the full CI/CD path for all Meesho microservices - Branch protection rules for
mainare enforced at the GitHub repository level (not visible in this repo's source) - No
.github/CODEOWNERSfile is present in the repository
Whitelist repo trust boundary
Policy exceptions (sonar skip, multizone, AppConfig, CAC) are fetched from Meesho/whitelists at build time via cicd-github-app credential. If Meesho/whitelists is compromised or the cicd-github-app credential is stolen, an attacker could:
- Add any service to
skip-sonar-whitelistto bypass quality gates - Add a service to
multizone-enabled-reposto block its deployments - Remove a service from
ValidateCacConfigto bypass config validation
Jenkins agent trust boundary
Build stages execute in GKE pods (see resources/org/meesho/prd-pod.yaml). Secrets injected via withCredentials exist in the pod's process environment for the duration of the withCredentials block and are cleared afterward. env.VAULT_TOKEN is temporarily set during Vault secret fetch and immediately cleared:
// buildNode.groovy:548-553
env.VAULT_TOKEN = "${TOKEN}"
sh(script:"${vault_cmd}")
env.VAULT_TOKEN = 'empty' // ← cleared immediately after use
Note: Assigning to env.* persists the value in Jenkins pipeline serialized state for the duration of that block — it is not fully memory-isolated like a withCredentials binding.
Process / runtime boundaries
| Boundary | Inside (trusted) | Outside | Crossing mechanism |
|---|---|---|---|
withCredentials block |
Jenkins credential binding (secret) | Pipeline Groovy scope | Automatic unset on block exit |
| Jenkins agent pod | Build process, injected creds | Other pods, external network | K8s network policy, GKE service account |
set +x shell guard |
ArgoCD password in shell arg | Jenkins console log | set +x before credential use in deployArgoCD.groovy:493 |
| DinD container | Docker daemon | Build container | TCP socket (dind-prd-svc) — not Unix socket (avoids privilege escalation) |
Entry Points
devops-lib has no HTTP entry points. It is invoked as a Jenkins Shared Library.
Build trigger (sole entry point)
| Trigger | Who sends it | Auth check |
|---|---|---|
| Ringmaster-initiated build | ringmaster-bot Jenkins user |
allowedUsers gate in eksCICD.groovy |
| Turbo-Turtle-initiated build | turbo-turtle Jenkins user |
allowedUsers gate in eksCICD.groovy |
| DevOps engineer direct trigger | Email in allowedUsers list |
allowedUsers gate |
| Unauthorized user | Any other Jenkins user | Hard-rejected — pipeline aborts immediately |
Outbound calls (not entry points, but relevant to trust)
All outbound calls are made FROM Jenkins agents TO external services. See docs/downstreams.md for the full inventory.
Security additions to downstreams.md:
| Service | Protocol | Data sent | Risk | Notes |
|---|---|---|---|---|
| ArgoCD | HTTPS + gRPC | App names, image tags | Low | set +x guards password in shell |
| Ringmaster | HTTPS | Build result, image tag, repo name, team | Low | Auth via ringmaster-token credential |
| Turbo-Turtle | HTTP (plain) | Build result, image tag, repo name | Low — accepted risk | Internal VPC only, not reachable externally |
| Deployment Tracker | HTTP (plain) | Repo name, deploy timestamp, tag | Low — accepted risk | Internal VPC only, legacy endpoint |
| Security scanner | HTTP to 172.31.5.29:63232 |
Repo name, branch | Low — accepted risk | Internal scanner, hardcoded IP |
| SonarQube | HTTPS | Source code analysis | Low | Token injected via withCredentials |
| Vault | HTTPS | Vault path (not secrets) | Low | Token cleared immediately after fetch |
| GitHub | HTTPS | Git operations | Low | cicd-github-app credential |
Authorization
Policy enforcement model
devops-lib enforces policy through two mechanisms:
-
Whitelist-controlled gates —
constructParam.groovychecksMeesho/whitelistsat runtime for per-repo exceptions. No service can grant itself a bypass; all exceptions require a PR toMeesho/whitelistsreviewed by DevOps. -
Library-level enforcement —
deployArgoCD.groovyenforces canary for Tier-1 services,eksCICD.groovyenforces the trigger gate. These cannot be overridden by service config.
allowedUsers list
The bypass list at vars/eksCICD.groovy:12 contains hardcoded engineer email addresses. This list has no expiry mechanism — emails remain valid until manually removed.
Data Classification
devops-lib handles no end-user PII. All data is build metadata:
Non-PII (safe to log and pass to external services)
| Data | Where it appears | Notes |
|---|---|---|
repo_name |
All stages, Ringmaster callback | GitHub org slug — not sensitive |
build_tool |
Build stages | Language identifier |
cicd_environment |
All stages | prd / stg / int / ftr |
TAG (image tag) |
Deploy stages, Slack notifications | <branch>-<git-sha> — not sensitive |
notify_channel |
Notify stage | Slack channel name |
team / bu |
Build stages, node pool selection | Org metadata |
deployment_order |
ArgoCD deploy | App names in devops-argo-config |
| Build result / duration | Slack, Ringmaster, Turbo-Turtle | Build observability |
Sensitive (not PII, but must be handled with care)
| Data | Where it exists | Handling |
|---|---|---|
| Jenkins credential values | withCredentials blocks |
Never logged, masked in console |
| Vault secret paths | buildNode.groovy:536-537 |
Path logged (not value); value only in sh subprocess |
ARGO_USERNAME / ARGO_PASSWORD |
deployArgoCD.groovy:490-494 |
set +x guard prevents echo in logs |
GITHUB_TOKEN |
Git clone operations | Via gitUsernamePassword binding — not logged |
Data Lifecycle & Erasure
No end-user data stored. Build artifacts:
Known:
- Jenkins build logs: retained per Jenkins job configuration (managed by Jenkins admins, not devops-lib)
- Docker images in GAR/ECR: no TTL configured in devops-lib — lifecycle managed by GAR cleanup policies outside this library
- Deployment history in Ringmaster: managed by Ringmaster service
Unknown: Retention policy for build logs and deployment records is not configurable from devops-lib.
Data Storage & Encryption
At rest
devops-lib has no persistent storage. It reads from GitHub, Jenkins credentials store, and Vault; it writes to GitHub repos (argo-config, helm-charts) and pushes Docker images to GAR/ECR.
| Store | What is stored | Managed by |
|---|---|---|
| Jenkins credentials store | All CI/CD credentials (tokens, passwords) | Jenkins admins |
| GAR / ECR | Docker images | GCP / AWS infra |
| devops-helm-charts / devops-argo-config | Helm values, ArgoCD manifests | devops-lib writes; git is the store |
In transit
- All GitHub API calls: HTTPS ✓
- ArgoCD CLI: HTTPS + gRPC ✓
- SonarQube, Vault, Ringmaster API: HTTPS ✓
- Turbo-Turtle, Deployment Tracker, security scanner: plain HTTP — accepted risk (internal VPC, not reachable externally)
Secrets management
All secrets are injected at runtime from Jenkins credentials store via withCredentials. No secrets in source code, config files, or environment variables baked into the library. Credentials are identified by their Jenkins credential ID (e.g. ringmaster-token, vault-prd-token) — the actual values are never stored in this repository.
Input Validation
devops-lib takes inputs from two sources:
Service config.yaml (primary input)
Read by getYamlParameter.getParam(). Fields are used directly without schema validation — devops-lib trusts the config.yaml from the consumer service's own repository (cloned via authenticated git). Malformed configs produce runtime errors, not silent misbehaviour.
buTeamMapping.groovy validates bu and team fields against a known mapping and throws on invalid values.
What's NOT validated
repo_namein config.yaml: used in ArgoCD app names, Slack messages, and Vault paths. Not sanitized against shell injection — passed directly intosh()scripts. Risk is mitigated becauserepo_namecomes from the service's own config.yaml in its own GitHub repo (already authenticated).notify_channel: passed directly to Slack API — no format validation. A malformed channel name produces a Slack API error, not a security issue.deployment_orderapp names: passed toargocd app sync— no format validation. An invalid app name produces an ArgoCD error.
Security Headers & CORS
Not applicable — devops-lib has no HTTP server and serves no responses.
Security Rules for New Code
These rules apply to anyone adding code to devops-lib:
Credentials: All secrets must be injected via withCredentials — never assign a credential value to a variable outside a withCredentials block, never interpolate credentials into log statements, and never store them in env.* variables beyond the immediate operation that needs them.
Shell commands with credentials: Use set +x immediately before any sh() that includes a credential variable as an argument (as done in deployArgoCD.groovy:493). Without set +x, Jenkins echoes the full shell command including the credential value to the build log.
Policy enforcement: Never add inline conditionals for repo-level policy exceptions. All exceptions must go through Meesho/whitelists — see ADR-0003.
Supply chain hygiene: Any change to vars/eksCICD.groovy or src/com/meesho/utilities/constructParam.groovy affects every Meesho microservice build. These files require extra scrutiny — treat them as Tier-1 code.
DinD image: The Docker-in-Docker image must come from the internal GAR registry (asia-southeast1-docker.pkg.dev/meesho-devops-admin-0622/admin/devops/docker:28-dind). Never use docker:N-dind from Docker Hub — it is not security-scanned and will be blocked by network policy. (See docs/tribal-knowledge.md TK#14.)
The following security rules are already enforced in CLAUDE.md NEVER DO and are not repeated here:
- Never hard-code AWS/GCP account IDs, vault tokens, or credentials — all secrets are passed via Jenkins credentials (
withCredentials) or injected throughconstructParam.run(). Credentials IDs are defined inconstructParam.groovy. - Never change the string
"ringmaster-bot"without coordinating with the Ringmaster team — it is the sole signal that routes callbacks to Ringmaster vs Turbo-Turtle.
Security Debt Tracker
| ID | Gap | Severity | Source | Ticket |
|---|---|---|---|---|
| SEC-DL-001 | allowedUsers list in eksCICD.groovy contains engineer email addresses with no expiry — stale access risk if engineers leave the org |
Low | Code-discovered | — |
| SEC-DL-002 | No .github/CODEOWNERS visible in repo — supply chain protection for main branch depends entirely on GitHub repo settings not auditable from source |
Medium | Code-discovered | — |
| SEC-DL-003 | Security scanner endpoint hardcoded as 172.31.5.29:63232 (plain HTTP, no auth) — if IP changes, scanner silently stops running |
Low | Code-discovered | — |
| SEC-DL-004 | env.VAULT_TOKEN temporarily assigned in buildNode.groovy:550 — token exists in Jenkins pipeline serialized state during Vault fetch block (cleared immediately after) |
Low | Code-discovered | — |
| SEC-DL-005 | Turbo-Turtle and Deployment Tracker callbacks over plain HTTP | Low | Code-discovered | — |
Past Security Incidents
No known security incidents documented in code, wiki, or Jira at time of generation.