buildDocker.groovy hardcoded harbor.35.238.248.203.nip.io as the push
target, so the registry could not move without editing this shared library
and every consumer moving in the same commit. It now reads
config.harbor_registry, whose default lives in homelabPipeline.groovy
beside harbor_project and every other key.
The stage errors rather than defaulting when the value is missing. Carrying
a second copy of the literal would leave two defaults free to disagree, and
an unset value would otherwise build an image named "null/<project>/<repo>"
— which docker accepts as a hostname and then fails to resolve, pointing
nowhere near the cause.
The dind pod no longer mounts the registry CA. That mount existed because
the registry was a nip.io name, which no public CA will issue for, so
cert-manager signed Harbor from a private CA; the node pool was told to
trust it for pulls, but a push comes from dockerd inside the build pod,
which has its own trust store. harbor.infra.deployshed.com carries a Let's
Encrypt certificate that both already trust, so the mount, its volume and
the whole arrangement go away rather than being repointed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LEsTefWWifp4ikvhHF5s6N
The GCP counterpart of devops-lib. Registered in Jenkins under the same
name, so consuming repos need no change: the two-line Jenkinsfile is
identical on both clusters, and which library it resolves to is a property
of the Jenkins running it.
Substantive changes, all consequences of GKE being a real cloud:
- Harbor speaks TLS here, so dind no longer passes --insecure-registry.
It mounts the private CA at
/etc/docker/certs.d/harbor.35.238.248.203.nip.io/ca.crt instead, from the
registry-ca ConfigMap. This is not redundant with the node pool's trust:
that covers pulls, performed by containerd on the node, while the push
comes from dockerd in the build pod with its own trust store. Without it,
pushes fail TLS verification while pulls of the same image succeed —
which reads like a broken registry rather than a missing trust anchor.
- The registry hostname changes in the push target and all five fallback
Dockerfiles. It still must be spelled identically everywhere, because
Docker matches credentials and trust by exact hostname.
- helm_repo_url moves to cluster DNS. That clone runs in a build pod, so
sending it out through the ingress and back would make the pipeline
depend on Contour for pod-to-pod traffic. syncArgoApp already addressed
ArgoCD this way and needed no change.
- build-tools.Dockerfile is removed: devops-base-images-gcp owns it now,
next to the mirrored base images, and the pod references the result by
tag at base-images/build-tools:1. The base-images project is public, so
the pod can pull it before it has any credentials.
Verified: no homelab addresses remain; the pod template parses with the CA
mount, the new image and no insecure-registry flag; and every fallback
template's base image, with the default version buildDocker would pick, is
present in the mirror manifest — an unmirrored tag now fails the build
rather than silently falling back to Docker Hub.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LEsTefWWifp4ikvhHF5s6N
The previous fix (47a60b7) escaped the functional \$PHPIZE_DEPS
reference but added an explanatory comment that itself used several
bare/example $ characters (${PHPIZE_DEPS}, $PHPIZE_DEPS, $IDENTIFIER,
${IDENTIFIER}, and a bare $ before a comma). SimpleTemplateEngine
parses the ENTIRE file as template source, comments included, so any
of those threw either MissingPropertyException or a harder
'illegal string body character after dollar sign' parse error
depending on what followed the $.
Rewrote the comment to explain the rule in prose without using the $
glyph as an illustrative example at all, so future edits to the
comment can't reintroduce this. Verified via grep across all five
fallback Dockerfiles that the only remaining $ occurrences are the
intentional ${version} placeholders plus the one properly-escaped
\$PHPIZE_DEPS.
Build failed: groovy.lang.MissingPropertyException: No such property:
PHPIZE_DEPS. This whole file is rendered through
constructTemplate.groovy's SimpleTemplateEngine before it ever becomes
a real Dockerfile — bare $IDENTIFIER is Groovy GString interpolation
syntax, same as ${IDENTIFIER}, so it tried resolving PHPIZE_DEPS
against the render binding (which only has `version`) instead of
leaving it as literal shell syntax for docker-php-ext-install to
expand at RUN time. \$ is the correct escape for a literal dollar sign
in SimpleTemplateEngine output.
~65 files carried over unmodified from the original org-wide library
(GKE/EKS-specific pipeline entry points, real Maven/Gradle/Node/Python/
PHP/Rust build stages entangled with AWS ECR/S3, GCP GAR/GCS, JFrog,
Sonar, a hardcoded internal security-scan endpoint, GitHub-specific
multi-branch scripting, BU/team org validation, and their supporting
pod templates/Helm value templates) — none directly usable without a
full rewrite regardless (same effort as what's already built for the
5 supported languages), so removed rather than kept as unused
reference. Also drops the Gradle wrapper (build.gradle, gradlew) — its
test suite was already removed in an earlier cleanup this session, so
nothing was left to build/test with it.
README.md rewrote entirely — it was still describing the old system
(eksCICD entry point, GKE/EKS, JFrog, S3, buTeamMapping) and even
referenced buildObjHelper.groovy, a file this same commit removes.
Only the live path remains: 7 stage files, 1 utility, 3 vars files,
the 5 language fallback Dockerfiles + Jenkinsfile template, and the
dind pod template + build-tools Dockerfile.
Every language's Dockerfile fallback template now pulls from
harbor.192.168.1.7.nip.io/base-images/... (mirrored from Docker Hub via
the new devops-base-images repo) instead of pulling live from Docker
Hub on every build. Eliminates that external dependency at build time,
and lets the mirrored tags be deliberately the leanest official
variant rather than whatever a public tag happens to default to.
- go: unchanged tags (golang:1.22-alpine, alpine:3.20 — already
minimal), just re-hosted.
- node/python/java: moved from their Debian-slim defaults to the
-alpine equivalent (node:20-alpine, python:3.12-alpine,
maven:3-eclipse-temurin-21-alpine, eclipse-temurin:21-jre-alpine).
- php: bigger change — dropped php:*-apache (Debian, full Apache
httpd) entirely for php:*-cli-alpine + PHP's own built-in dev server
(`php -S`), moving to port 8080 like every other language instead of
PHP's special-cased 80. Not production-grade PHP serving (PHP's own
docs call the built-in server not designed for that), but genuinely
minimal and fine for a homelab/demo app — would need php-fpm+nginx
for anything serving real traffic.
Only versions actually mirrored into Harbor resolve now — a
dockerBuildVersion whose tag isn't in devops-base-images/images.txt
needs that added and re-mirrored first, unlike pulling straight from
Docker Hub where any tag "just worked". Chose keeping a shell (Alpine)
over full distroless — homelab kubectl-exec debuggability weighed more
than the last bit of attack-surface reduction.
Replaces per-build on-demand installs (apk add bash/python3/curl,
curl-downloading yq to /tmp) in runHooks.groovy, syncArgoApp.groovy,
and updateHelmTag.groovy with a single custom image
(build-tools.Dockerfile) that has all of it baked in once, at
image-build time — not repeated on every single pipeline run.
updateHelmTag.groovy also now runs inside container('docker-cli')
(previously unwrapped, defaulting to the auto-injected jnlp agent
container, which is why it needed the curl-downloaded yq fallback in
the first place — that container has git but not yq).
dind-pod.yaml's docker-cli container now points at
harbor.192.168.1.7.nip.io/homelab/build-tools:1 instead of the stock
docker:27-cli — this image needs building and pushing once before any
build using this pod template will work; see build-tools.Dockerfile's
header comment.
demo-go-app's Deployment failed to pull: "dial tcp: lookup
harbor-core.harbor.svc.cluster.local on 127.0.0.53:53: server
misbehaving". docker push worked from the Jenkins build pod because
it has pod-network DNS (CoreDNS); pulling for a real Deployment
happens via containerd on the node itself, using the node's host-level
resolver, which has no route to *.svc.cluster.local at all. Switches
buildDocker.groovy's push target and dind-pod.yaml's
--insecure-registry flag to the Contour ingress hostname instead,
which resolves via normal public DNS (nip.io) from both pods and the
host.
Confirmed via a direct token request to Harbor's own token endpoint
(same username/password from harbor-robot-dockerconfig) that the
robot account genuinely has push access to homelab/demo-go-app —
Harbor returned a valid token with actions:[pull,push]. So the actual
`docker push` failure ("unauthorized... action: push") wasn't a
permissions problem at all.
Root cause: kubernetes.io/dockerconfigjson secrets are required to
store their data under the fixed key `.dockerconfigjson`. Mounting
the secret without remapping that key meant the file that actually
landed at /root/.docker was named `.dockerconfigjson`, not
`config.json` — the only filename docker's CLI reads for stored
credentials. Docker found nothing there and pushed unauthenticated,
which Harbor correctly rejected. Adds an items: remap so the mounted
file is named config.json.
docker push was hanging: "Client.Timeout exceeded while awaiting
headers" doing a TLS handshake against harbor-core.harbor.svc.cluster.local,
which only ever speaks plain HTTP (TLS disabled cluster-wide by
design). Docker defaults to HTTPS for any bare registry hostname
regardless of network path — that default has nothing to do with
whether traffic routes through Contour/Ingress, contrary to what an
earlier pending-items note assumed. Adds --insecure-registry to the
dind container's dockerd startup args.
Build #5 got through checkout, loadConfig, and the pre_build hook,
then hit two real bugs at the actual docker build step:
1. `docker build` failed with "mkdir /root/.docker/buildx: read-only
file system" — dind-pod.yaml mounts the Harbor push-auth secret
read-only at /root/.docker, but modern docker defaults to
BuildKit/buildx, which wants to write its own state there. Forces
the classic builder via DOCKER_BUILDKIT=0 instead.
2. The subsequent error-handling itself then threw a
NullPointerException — every stage file (including untouched
legacy ones from the real devops-lib) reads env.FAILURE when
setting currentBuild.result, but nothing in this repo ever defined
it, so it was null. Defined once in homelabPipeline.groovy rather
than touching 40+ individual occurrences across every stage file.
Renames src/com/meesho -> src/com/homelab, resources/com/meesho ->
resources/com/homelab, resources/org/meesho -> resources/org/homelab
(via git mv, preserving history), and sweeps every remaining
occurrence of "meesho" (any casing) out of package declarations,
imports, libraryResource() paths, and comments across the whole repo.
Also drops the per-user allowlist in vars/eksCICD.groovy, which
hardcoded real former-colleagues' emails and doesn't apply to a
single-person homelab — that branch is now permanently skipped rather
than deleted outright, to avoid hand-editing the escape-sequence-heavy
echo blocks it guards (eksCICD.groovy itself is unused legacy code,
not called by homelabPipeline.groovy).
Does not touch the ~114 files that were already missing from the
working tree but still tracked in the prior commit — that's unrelated
pre-existing state, left as-is.
