Files
devops-lib-gcp/docs/SECURITY.md
T
2026-08-26 02:02:24 +05:30

15 KiB
Raw Blame History

SECURITY.md

Scope: devops-lib is a Jenkins Shared Library, not a deployable web service. It has no HTTP endpoints of its own. This document covers the security properties of the library's CI/CD execution: how secrets are handled, what trust boundaries exist, what security rules new code must follow, and known gaps.


Authentication

devops-lib has no user-facing HTTP endpoints and performs no JWT or session validation. Authentication is enforced at two points:

Build trigger gate

vars/eksCICD.groovy:1226 validates that every Jenkins build is triggered by an authorized caller:

  • ringmaster-bot — Ringmaster's automated trigger
  • turbo-turtle — Turbo-Turtle's CI callback trigger
  • allowedUsers — a hardcoded list of DevOps engineer email addresses for emergency access

Unauthorized triggers are hard-rejected before any pipeline logic runs.

Outbound credential injection

All outbound API calls use Jenkins' withCredentials binding — credentials are never hardcoded in source. Jenkins masks bound variables in console output automatically. Key credential IDs:

Credential ID Used for Scope
cicd-github-app Cloning Meesho/whitelists, devops-argo-config, devops-helm-charts All builds
svc-devops-meesho GitHub API, JFrog Artifactory Build + deploy
ringmaster-token Ringmaster callback API Notify stage
argocd-{bu}-prd-creds / argocd-dev-creds ArgoCD CLI login Deploy stage
vault-prd-token / vault-dev-token Vault secret fetch Node builds only
sonar-token-prod / sonar-token-{bu}-dev SonarQube analysis Build stages

Trust Boundaries

Build trigger trust boundary

Layer What happens Where Confidence
Ringmaster / Turbo-Turtle Validates human approval, triggers Jenkins build Upstream (Ringmaster infra) docs-referenced
devops-lib eksCICD Validates trigger source against allowedUsers list vars/eksCICD.groovy:12 code-confirmed
Jenkins pipeline Executes build stages with injected credentials Jenkins agents (GKE pods) code-confirmed

Trust assumption: devops-lib assumes that any build triggered by ringmaster-bot or turbo-turtle has already been approved by the Ringmaster/Turbo-Turtle authorization flow. It does NOT re-validate the approval — it trusts the trigger identity.

Supply chain trust boundary (CRITICAL)

devops-lib@main is loaded via @Library('devops-lib@main') by every Meesho microservice on every build. A malicious or buggy merge to main is an immediate supply chain attack on all 100+ consumer services' CI/CD pipelines.

  • Whoever can merge to devops-lib@main controls the full CI/CD path for all Meesho microservices
  • Branch protection rules for main are enforced at the GitHub repository level (not visible in this repo's source)
  • No .github/CODEOWNERS file is present in the repository

Whitelist repo trust boundary

Policy exceptions (sonar skip, multizone, AppConfig, CAC) are fetched from Meesho/whitelists at build time via cicd-github-app credential. If Meesho/whitelists is compromised or the cicd-github-app credential is stolen, an attacker could:

  • Add any service to skip-sonar-whitelist to bypass quality gates
  • Add a service to multizone-enabled-repos to block its deployments
  • Remove a service from ValidateCacConfig to bypass config validation

Jenkins agent trust boundary

Build stages execute in GKE pods (see resources/org/meesho/prd-pod.yaml). Secrets injected via withCredentials exist in the pod's process environment for the duration of the withCredentials block and are cleared afterward. env.VAULT_TOKEN is temporarily set during Vault secret fetch and immediately cleared:

// buildNode.groovy:548-553
env.VAULT_TOKEN = "${TOKEN}"
sh(script:"${vault_cmd}")
env.VAULT_TOKEN = 'empty'   // ← cleared immediately after use

Note: Assigning to env.* persists the value in Jenkins pipeline serialized state for the duration of that block — it is not fully memory-isolated like a withCredentials binding.

Process / runtime boundaries

Boundary Inside (trusted) Outside Crossing mechanism
withCredentials block Jenkins credential binding (secret) Pipeline Groovy scope Automatic unset on block exit
Jenkins agent pod Build process, injected creds Other pods, external network K8s network policy, GKE service account
set +x shell guard ArgoCD password in shell arg Jenkins console log set +x before credential use in deployArgoCD.groovy:493
DinD container Docker daemon Build container TCP socket (dind-prd-svc) — not Unix socket (avoids privilege escalation)

Entry Points

devops-lib has no HTTP entry points. It is invoked as a Jenkins Shared Library.

Build trigger (sole entry point)

Trigger Who sends it Auth check
Ringmaster-initiated build ringmaster-bot Jenkins user allowedUsers gate in eksCICD.groovy
Turbo-Turtle-initiated build turbo-turtle Jenkins user allowedUsers gate in eksCICD.groovy
DevOps engineer direct trigger Email in allowedUsers list allowedUsers gate
Unauthorized user Any other Jenkins user Hard-rejected — pipeline aborts immediately

Outbound calls (not entry points, but relevant to trust)

All outbound calls are made FROM Jenkins agents TO external services. See docs/downstreams.md for the full inventory.

Security additions to downstreams.md:

Service Protocol Data sent Risk Notes
ArgoCD HTTPS + gRPC App names, image tags Low set +x guards password in shell
Ringmaster HTTPS Build result, image tag, repo name, team Low Auth via ringmaster-token credential
Turbo-Turtle HTTP (plain) Build result, image tag, repo name Low — accepted risk Internal VPC only, not reachable externally
Deployment Tracker HTTP (plain) Repo name, deploy timestamp, tag Low — accepted risk Internal VPC only, legacy endpoint
Security scanner HTTP to 172.31.5.29:63232 Repo name, branch Low — accepted risk Internal scanner, hardcoded IP
SonarQube HTTPS Source code analysis Low Token injected via withCredentials
Vault HTTPS Vault path (not secrets) Low Token cleared immediately after fetch
GitHub HTTPS Git operations Low cicd-github-app credential

Authorization

Policy enforcement model

devops-lib enforces policy through two mechanisms:

  1. Whitelist-controlled gatesconstructParam.groovy checks Meesho/whitelists at runtime for per-repo exceptions. No service can grant itself a bypass; all exceptions require a PR to Meesho/whitelists reviewed by DevOps.

  2. Library-level enforcementdeployArgoCD.groovy enforces canary for Tier-1 services, eksCICD.groovy enforces the trigger gate. These cannot be overridden by service config.

allowedUsers list

The bypass list at vars/eksCICD.groovy:12 contains hardcoded engineer email addresses. This list has no expiry mechanism — emails remain valid until manually removed.


Data Classification

devops-lib handles no end-user PII. All data is build metadata:

Non-PII (safe to log and pass to external services)

Data Where it appears Notes
repo_name All stages, Ringmaster callback GitHub org slug — not sensitive
build_tool Build stages Language identifier
cicd_environment All stages prd / stg / int / ftr
TAG (image tag) Deploy stages, Slack notifications <branch>-<git-sha> — not sensitive
notify_channel Notify stage Slack channel name
team / bu Build stages, node pool selection Org metadata
deployment_order ArgoCD deploy App names in devops-argo-config
Build result / duration Slack, Ringmaster, Turbo-Turtle Build observability

Sensitive (not PII, but must be handled with care)

Data Where it exists Handling
Jenkins credential values withCredentials blocks Never logged, masked in console
Vault secret paths buildNode.groovy:536-537 Path logged (not value); value only in sh subprocess
ARGO_USERNAME / ARGO_PASSWORD deployArgoCD.groovy:490-494 set +x guard prevents echo in logs
GITHUB_TOKEN Git clone operations Via gitUsernamePassword binding — not logged

Data Lifecycle & Erasure

No end-user data stored. Build artifacts:

Known:

  • Jenkins build logs: retained per Jenkins job configuration (managed by Jenkins admins, not devops-lib)
  • Docker images in GAR/ECR: no TTL configured in devops-lib — lifecycle managed by GAR cleanup policies outside this library
  • Deployment history in Ringmaster: managed by Ringmaster service

Unknown: Retention policy for build logs and deployment records is not configurable from devops-lib.


Data Storage & Encryption

At rest

devops-lib has no persistent storage. It reads from GitHub, Jenkins credentials store, and Vault; it writes to GitHub repos (argo-config, helm-charts) and pushes Docker images to GAR/ECR.

Store What is stored Managed by
Jenkins credentials store All CI/CD credentials (tokens, passwords) Jenkins admins
GAR / ECR Docker images GCP / AWS infra
devops-helm-charts / devops-argo-config Helm values, ArgoCD manifests devops-lib writes; git is the store

In transit

  • All GitHub API calls: HTTPS ✓
  • ArgoCD CLI: HTTPS + gRPC ✓
  • SonarQube, Vault, Ringmaster API: HTTPS ✓
  • Turbo-Turtle, Deployment Tracker, security scanner: plain HTTP — accepted risk (internal VPC, not reachable externally)

Secrets management

All secrets are injected at runtime from Jenkins credentials store via withCredentials. No secrets in source code, config files, or environment variables baked into the library. Credentials are identified by their Jenkins credential ID (e.g. ringmaster-token, vault-prd-token) — the actual values are never stored in this repository.


Input Validation

devops-lib takes inputs from two sources:

Service config.yaml (primary input)

Read by getYamlParameter.getParam(). Fields are used directly without schema validation — devops-lib trusts the config.yaml from the consumer service's own repository (cloned via authenticated git). Malformed configs produce runtime errors, not silent misbehaviour.

buTeamMapping.groovy validates bu and team fields against a known mapping and throws on invalid values.

What's NOT validated

  • repo_name in config.yaml: used in ArgoCD app names, Slack messages, and Vault paths. Not sanitized against shell injection — passed directly into sh() scripts. Risk is mitigated because repo_name comes from the service's own config.yaml in its own GitHub repo (already authenticated).
  • notify_channel: passed directly to Slack API — no format validation. A malformed channel name produces a Slack API error, not a security issue.
  • deployment_order app names: passed to argocd app sync — no format validation. An invalid app name produces an ArgoCD error.

Security Headers & CORS

Not applicable — devops-lib has no HTTP server and serves no responses.


Security Rules for New Code

These rules apply to anyone adding code to devops-lib:

Credentials: All secrets must be injected via withCredentials — never assign a credential value to a variable outside a withCredentials block, never interpolate credentials into log statements, and never store them in env.* variables beyond the immediate operation that needs them.

Shell commands with credentials: Use set +x immediately before any sh() that includes a credential variable as an argument (as done in deployArgoCD.groovy:493). Without set +x, Jenkins echoes the full shell command including the credential value to the build log.

Policy enforcement: Never add inline conditionals for repo-level policy exceptions. All exceptions must go through Meesho/whitelists — see ADR-0003.

Supply chain hygiene: Any change to vars/eksCICD.groovy or src/com/meesho/utilities/constructParam.groovy affects every Meesho microservice build. These files require extra scrutiny — treat them as Tier-1 code.

DinD image: The Docker-in-Docker image must come from the internal GAR registry (asia-southeast1-docker.pkg.dev/meesho-devops-admin-0622/admin/devops/docker:28-dind). Never use docker:N-dind from Docker Hub — it is not security-scanned and will be blocked by network policy. (See docs/tribal-knowledge.md TK#14.)

The following security rules are already enforced in CLAUDE.md NEVER DO and are not repeated here:

  • Never hard-code AWS/GCP account IDs, vault tokens, or credentials — all secrets are passed via Jenkins credentials (withCredentials) or injected through constructParam.run(). Credentials IDs are defined in constructParam.groovy.
  • Never change the string "ringmaster-bot" without coordinating with the Ringmaster team — it is the sole signal that routes callbacks to Ringmaster vs Turbo-Turtle.

Security Debt Tracker

ID Gap Severity Source Ticket
SEC-DL-001 allowedUsers list in eksCICD.groovy contains engineer email addresses with no expiry — stale access risk if engineers leave the org Low Code-discovered
SEC-DL-002 No .github/CODEOWNERS visible in repo — supply chain protection for main branch depends entirely on GitHub repo settings not auditable from source Medium Code-discovered
SEC-DL-003 Security scanner endpoint hardcoded as 172.31.5.29:63232 (plain HTTP, no auth) — if IP changes, scanner silently stops running Low Code-discovered
SEC-DL-004 env.VAULT_TOKEN temporarily assigned in buildNode.groovy:550 — token exists in Jenkins pipeline serialized state during Vault fetch block (cleared immediately after) Low Code-discovered
SEC-DL-005 Turbo-Turtle and Deployment Tracker callbacks over plain HTTP Low Code-discovered

Past Security Incidents

No known security incidents documented in code, wiki, or Jira at time of generation.