Infra: GCP Jenkins Agent Pods

All GCP builds run inside a Jenkins Kubernetes pod provisioned from YAML specs in resources/org/meesho/. The pod YAML is selected by env.INFRA_ENV and loaded via libraryResource in eksCICD.gcpInfra().

TL;DR

Pod specs live in resources/org/meesho/*.yaml.
Filename pattern: <INFRA_ENV>-pod.yaml and <INFRA_ENV>-sidecar-pod.yaml.
INFRA_ENV values seen in practice: prd, stg, dev, toolchain.
All builds run in the devops-tools container defined in the pod spec.
Sidecar pods (useSidecar: true in consumer Jenkinsfile) add extra containers alongside devops-tools.
AWS builds skip this entirely — they run directly on static EKS nodes labeled EKS.

Mental model

Jenkins provisions a fresh Kubernetes pod for each build. The pod spec defines:

Which container images to run (typically devops-tools + optional sidecar).
Resource requests/limits for the build container.
Any mounted volumes (e.g., Docker socket for DinD builds).

gcpInfra() reads env.INFRA_ENV (injected by the Jenkins job configuration), constructs the pod YAML filename, loads it via libraryResource, and wraps the entire pipeline in a podTemplate { node { container('devops-tools') { ... } } } block.

Structure / data flow

eksCICD.call(repo):
  CLOUD_PROVIDER=GCP → gcpInfra(repo)

gcpInfra(repo):
  isSidecarNeeded = repo.get('useSidecar', false)
  yamlName = isSidecarNeeded ? "${INFRA_ENV}-sidecar-pod.yaml" : "${INFRA_ENV}-pod.yaml"
  podyaml = "org/meesho/${yamlName}"
  podTemplate(yaml: libraryResource(podyaml)) {
    node(POD_LABEL) {
      container('devops-tools') {
        commonCICDFlow(repo)
      }
    }
  }

Pod YAML inventory (resources/org/meesho/):
  prd-pod.yaml            prd-sidecar-pod.yaml
  stg-pod.yaml            stg-sidecar-pod.yaml
  dev-pod.yaml            dev-sidecar-pod.yaml
  toolchain-pod.yaml      toolchain-sidecar-pod.yaml

Key code locations

Symbol	File	What it does
`gcpInfra`	`vars/eksCICD.groovy:gcpInfra`	Selects pod YAML and wraps pipeline in podTemplate
`awsInfra`	`vars/eksCICD.groovy:awsInfra`	AWS path — no pod YAML, just node('EKS')
`run`	`src/com/meesho/utilities/nodePoolSelection.groovy:run`	Selects GKE node pool label for deployment (separate from build pod)

Sharp edges

Build pod vs deployment node pool are different things: the Jenkins agent pod is where the build runs; nodePoolSelection.run() selects which GKE node pool the deployed application should land on. These are independent.
INFRA_ENV is job-level config: it is not derived from the branch name. It is injected by the Jenkins multibranch pipeline configuration. A mis-configured job can run a main branch build with INFRA_ENV=dev — which would use the dev pod spec.
Toolchain pods are segregated: toolchain-pod.yaml uses images from the toolchain namespace in meesho-central-dev-0622 project — separate from standard build pods.
DinD (Docker-in-Docker) socket: the devops-tools container in pod specs mounts a DinD service socket (env.DOCKER_HOST) rather than the host Docker socket. The DOCKER_HOST env var is set by constructParam.run() based on cicd_environment. The correct endpoint values are dind-prd-svc (prd) and dind-dev-new-svc.jenkins-new.svc.cluster.local (stg/ftr). Do not use tcp://localhost:2375 — it creates conflicts that require unnecessary guard logic. The DinD image must come from the internal GAR registry (asia-southeast1-docker.pkg.dev/meesho-devops-admin-0622/admin/devops/docker:28-dind) — not Docker Hub.
build-tools image version encodes the Go compiler and sonar-scanner CLI: resources/org/meesho/prd-pod.yaml and stg-pod.yaml reference a build-tools image that bundles Go, sonar-scanner-cli, and other toolchain binaries. When a new toolchain binary is needed, bump the image tag in both pod YAMLs — never curl/wget a tool from the internet inside a pipeline stage.
Non-prd node pools are BU-scoped: stg/ftr/dev pods land on {BU}-shared pools, so all services in the same BU share one pool. A memory leak or noisy-neighbour in one supply service degrades all other supply services on staging.

Node pool selection — GKE node pool assignment for deployed applications
Architecture — where gcpInfra fits in the pipeline
Config policy — env.DOCKER_HOST set by constructParam

Notes

← Previous · Index · Next →

4.9 KiB Raw Blame History