Remove unused legacy files, rewrite README for the actual system
~65 files carried over unmodified from the original org-wide library (GKE/EKS-specific pipeline entry points, real Maven/Gradle/Node/Python/ PHP/Rust build stages entangled with AWS ECR/S3, GCP GAR/GCS, JFrog, Sonar, a hardcoded internal security-scan endpoint, GitHub-specific multi-branch scripting, BU/team org validation, and their supporting pod templates/Helm value templates) — none directly usable without a full rewrite regardless (same effort as what's already built for the 5 supported languages), so removed rather than kept as unused reference. Also drops the Gradle wrapper (build.gradle, gradlew) — its test suite was already removed in an earlier cleanup this session, so nothing was left to build/test with it. README.md rewrote entirely — it was still describing the old system (eksCICD entry point, GKE/EKS, JFrog, S3, buTeamMapping) and even referenced buildObjHelper.groovy, a file this same commit removes. Only the live path remains: 7 stage files, 1 utility, 3 vars files, the 5 language fallback Dockerfiles + Jenkinsfile template, and the dind pod template + build-tools Dockerfile.
This commit is contained in:
Mukul Sharma
@@ -1,67 +1,72 @@
|
||||
# devops-lib
|
||||
|
||||
---
|
||||
Jenkins Shared Library for this homelab's CI/CD pipeline. Adapted from a
|
||||
much larger, company-wide library — everything not needed for a
|
||||
single-node homelab (GKE/EKS, JFrog, S3, Sonar, org-specific BU/team
|
||||
validation, and ~65 other files covering languages/deploy-targets this
|
||||
setup never uses) has been removed rather than carried along unused; see
|
||||
git history if any of that is ever worth reviving.
|
||||
|
||||
## Parameters
|
||||
Most of the functionality depends on the parameters provided by the users in form of groovy map of key and value pairs. The supported parameters are as below:
|
||||
## Using it in a service repo
|
||||
|
||||
### Required parameters
|
||||
**repo_name**: The key repo_name is required for checking out the code in a subdirectory. The value is the repository name that you want to checkout
|
||||
|
||||
**build_tool**: This parameter is required to identify which build_tool to use in the pipeline. The supported values are *maven*, *gradle*, *docker*, *python*, *node*, *go*, *php* (and their prefixed variants such as *maven-3.3-jdk-17*, *python-3*, *node-16*, *go1.21*)
|
||||
|
||||
**maintainer** : This parameter is required to send the notification in the slack channel *#ci-cd-status*. Please provide your slack username here
|
||||
|
||||
### Optional parameter
|
||||
|
||||
`devops-lib` is Homelab's Jenkins Shared Library that provides a unified CI/CD pipeline for all microservices across the organisation. Consumer repos load it via `@Library('devops-lib@main')` and call a single `eksCICD(repo)` entry point — the library handles language-specific building (Maven, Go, Gradle, Node.js, Python, PHP), code quality gates (Sonar), Docker image publishing to GAR/ECR, Helm chart updates, and ArgoCD-based deployment to GKE/EKS clusters. Build status and deployment metadata are reported back to Ringmaster and Slack.
|
||||
|
||||
**Stack:** Groovy (Jenkins Shared Library) · ArgoCD · Helm · GCP (GKE, GAR, GCS, Vault, Sonar) · AWS (EKS, ECR, S3)
|
||||
|
||||
## Dependencies
|
||||
|
||||
**push_to_jfrog**: By default master, main, gcp-main, and gcp-master branches push artifacts to jfrog/s3 repository, set this parameter to true to push artifacts from non-master branches
|
||||
|
||||
---
|
||||
|
||||
## config.yaml schema (consumer services)
|
||||
|
||||
Every service that uses this library must provide a `config.yaml`:
|
||||
|
||||
| Key | Required | Description |
|
||||
|-----|----------|-------------|
|
||||
| `repo_name` | yes | GitHub repo slug — must match exactly |
|
||||
| `build_tool` | yes | `maven`, `go`, `gradle`, `node-*`, `python-*`, `php`, `docker` |
|
||||
| `dockerBuildVersion` | yes | Drives Dockerfile template: `maven-21`, `go-1.22`, `node-20`, etc. |
|
||||
| `team` | yes | Team slug — validated against `buTeamMapping` |
|
||||
| `bu` | yes | Business unit: `supply`, `demand`, `central`, `dataengg`, `datascience`, `mcache`, `infra` |
|
||||
| `maintainer` | yes | GitHub handle for Slack notifications |
|
||||
| `deployment_order` | yes | List of ArgoCD application names to deploy |
|
||||
| `notify_channel` | no | Slack channel (default: `ci-cd-status`) |
|
||||
| `skip_sonar` | no | Whitelist-gated; see `constructParam.groovy` |
|
||||
| `deployArgo` | no | Set `false` to skip ArgoCD sync |
|
||||
| `appConfigEnabled` | no | Required `true` for `stg`; whitelist-gated |
|
||||
| `skip_test` | no | Skip unit tests (Maven) |
|
||||
| `push_to_jfrog` | no | Publish JAR to JFrog Artifactory |
|
||||
| `push_to_s3` | no | Push artifact to S3 |
|
||||
| `build_packages` | no | System development packages required while compiling (currently consumed by Rust builds; for example `libpq-dev`) |
|
||||
| `runtime_packages` | no | System runtime libraries required by the compiled binary (currently consumed by Rust builds; for example `libpq5`) |
|
||||
|
||||
## Adding this library to a new service
|
||||
The entire Jenkinsfile is 2 lines:
|
||||
|
||||
```groovy
|
||||
// Jenkinsfile
|
||||
@Library('devops-lib@main') _
|
||||
|
||||
eksCICD([
|
||||
repo_name: 'my-service'
|
||||
])
|
||||
@Library('devops-lib') _
|
||||
homelabPipeline(repo_name: 'my-service')
|
||||
```
|
||||
|
||||
Place `config.yaml` at the repo root with the required fields above.
|
||||
`repo_name` is the only required key. Everything else has a sensible
|
||||
default — override any of them by passing extra keys to `homelabPipeline`,
|
||||
or by committing a `config.yaml` to the service repo's own root (merged
|
||||
in after checkout; repo-committed values win over the Jenkinsfile call).
|
||||
|
||||
## Adding a new build stage
|
||||
| Key | Default | Notes |
|
||||
|---|---|---|
|
||||
| `service_name` | `repo_name` | Second path segment under `devops-helm-charts/values/` |
|
||||
| `argo_app_name` | `repo_name` | Must match the ArgoCD Application's `metadata.name` |
|
||||
| `harbor_project` | `homelab` | Must be an existing, public Harbor project |
|
||||
| `helm_repo_url` | `devops-helm-charts` on this Gitea | — |
|
||||
| `image_tag_yq_path` | `.deployment.image.tag` | **Override this if the app's chart isn't `1.0.0`** — e.g. `sts-2.0.0` uses `.podtemplate.image.tag` instead. Getting this wrong doesn't fail loudly: `yq -i` creates the path if missing rather than erroring, silently leaving the real field un-bumped. |
|
||||
| `dockerBuildVersion` | none | Only read when the repo has **no Dockerfile of its own** — picks a fallback template (see below). No default; either ship a Dockerfile or set this. |
|
||||
|
||||
1. Create `src/com/homelab/stages/build<Lang>.groovy` implementing `def run(Map config)`.
|
||||
2. Add a `case` in `src/com/homelab/stages/buildObjHelper.groovy`.
|
||||
3. Add a Dockerfile template in `resources/com/homelab/<lang>-Dockerfile` if needed.
|
||||
## Pipeline stages
|
||||
|
||||
`checkOut → loadConfig → runHooks(pre_build) → buildDocker →
|
||||
runHooks(post_build) → updateHelmTag → syncArgoApp → notify`, all inside
|
||||
a `podTemplate` (`resources/org/homelab/dind-pod.yaml`) via
|
||||
`node(POD_LABEL) { ... }`.
|
||||
|
||||
- **`loadConfig`** — if the repo has a `config.yaml` at its root, its
|
||||
keys are merged into the pipeline config (repo values win).
|
||||
- **`runHooks`** — reads `config.yaml`'s `hooks.pre_build`/`hooks.post_build`
|
||||
lists, each `{name, script, interpreter, requirements, blocking,
|
||||
timeout_seconds}`. Blocking by default; `blocking: false` demotes a
|
||||
failure to advisory (log + continue). Script paths must be
|
||||
repo-relative (no `..`, no absolute paths).
|
||||
- **`buildDocker`** — uses the repo's own `Dockerfile` if present;
|
||||
otherwise renders one from `resources/com/homelab/<lang>-Dockerfile`
|
||||
based on `dockerBuildVersion` (e.g. `go-1.22`, `node-20`,
|
||||
`python-3.12`, `java-21`, `php-8.3`). All fallback templates pull base
|
||||
images from Harbor's `base-images` project (mirrored via the separate
|
||||
`devops-base-images` repo), not Docker Hub directly.
|
||||
- **`updateHelmTag`** — clones `devops-helm-charts`, bumps the image tag
|
||||
via `yq` at `image_tag_yq_path`, commits, pushes to `main`.
|
||||
- **`syncArgoApp`** — calls the ArgoCD REST API to sync `argo_app_name`.
|
||||
|
||||
## Adding a new language's fallback template
|
||||
|
||||
1. Add the base image to `devops-base-images/images.txt`, re-mirror it
|
||||
into Harbor.
|
||||
2. Add `resources/com/homelab/<lang>-Dockerfile`, parametrized by
|
||||
`${version}` (rendered via `constructTemplate.groovy`'s
|
||||
`SimpleTemplateEngine` wrapper).
|
||||
3. Add a case for it in `buildDocker.groovy`'s `templates` map.
|
||||
|
||||
## Build-tools image
|
||||
|
||||
`resources/org/homelab/build-tools.Dockerfile` bakes git/yq/bash/
|
||||
python3+pip/venv/curl into the `docker-cli` container's image, so
|
||||
nothing gets installed on demand on every single build. Built and pushed
|
||||
manually (not through any Jenkins job) — see that file's own header
|
||||
comment.
|
||||
|
||||
Reference in New Issue
Block a user