Fold the deployshed ingresses into their charts, keep nip.io for Harbor
Argo CD and Harbor each had a standalone Ingress serving the real domain while their charts still served nip.io. Both charts now own their real hostname directly, so the standalone objects are removed — extra-manifests syncs the directory with prune, so deleting the files deletes the objects. harbor-ingress-legacy-nipio.yaml is the inverse of the file it replaces. Harbor's chart now serves harbor.infra.deployshed.com with a Let's Encrypt certificate, and this object serves the old nip.io name with the private CA the chart used to use. It has to exist: apps deployed before the move recorded harbor.35.238.248.203.nip.io image references in toolshed's database and in already-rendered pod specs, and nothing in git can rewrite those — they change only when each app is rebuilt. Until then a pod that restarts, reschedules or scales up pulls the old reference, and if the name stops resolving that pull fails looking like a broken registry rather than a retired hostname. It retires together with the registry-ca manifests and Terraform's node-pool trust. harbor-robot-dockerconfig lists both hostnames. Docker matches stored credentials by exact host and nothing else, so a single-value swap breaks whichever side lands second: change the credential first and the pipeline pushes unauthenticated, change the pipeline first and the same happens in reverse. Two entries make the order irrelevant — it is one robot account on one Harbor reached by two names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LEsTefWWifp4ikvhHF5s6N
This commit is contained in:
Mukul Sharma
co-authored by
Claude Opus 5
Claude Opus 5
parent
5a008ffa08
commit
1e8cc7a028
@@ -1,45 +0,0 @@
|
||||
# ArgoCD's own hostname on the real domain, with TLS.
|
||||
#
|
||||
# A standalone Ingress rather than the chart's, because the argo-cd chart
|
||||
# cannot express this. Its server.ingress.tls is a BOOLEAN covering
|
||||
# `hostname` plus every entry in `extraHosts`, all from one fixed secret,
|
||||
# and there is no extraTls to scope it more narrowly. Turning it on while
|
||||
# the nip.io hostname is still served would request a single certificate
|
||||
# spanning both names — and Let's Encrypt cannot issue for nip.io, so the
|
||||
# request fails and NEITHER name ends up with TLS.
|
||||
#
|
||||
# So the chart keeps serving argocd.35.238.248.203.nip.io over plain HTTP
|
||||
# through its own Ingress, and this object serves the real domain over TLS.
|
||||
# Both route to the same backend. When nip.io is retired, this file should
|
||||
# go away and the chart's own tls: true should be turned on instead — one
|
||||
# hostname, one certificate, no second object to remember.
|
||||
#
|
||||
# Backed by port 80, not 443: this deployment runs argocd-server with
|
||||
# server.insecure: true, so it speaks plain HTTP and TLS terminates at
|
||||
# Envoy. Pointing at 443 would have Envoy attempt TLS to a backend that
|
||||
# does not serve it.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: argocd-ingress-deployshed
|
||||
namespace: argocd
|
||||
annotations:
|
||||
# ingress-shim issues and renews the certificate named in tls below.
|
||||
cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
spec:
|
||||
ingressClassName: contour
|
||||
tls:
|
||||
- hosts:
|
||||
- argocd.infra.deployshed.com
|
||||
secretName: argocd-deployshed-tls
|
||||
rules:
|
||||
- host: argocd.infra.deployshed.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: argocd-admin-prd-server
|
||||
port:
|
||||
number: 80
|
||||
@@ -1,85 +0,0 @@
|
||||
# Harbor's hostname on the real domain, with a public certificate.
|
||||
#
|
||||
# A standalone Ingress because Harbor's chart has no multi-host mechanism at
|
||||
# all — no extraHosts like argo-cd, no secondaryingress like jenkins. Its
|
||||
# expose.ingress.hosts.core is a single scalar, so the only way to serve a
|
||||
# second name is a second object. Checked against the chart, not assumed.
|
||||
#
|
||||
# The chart's own Ingress is untouched and keeps serving
|
||||
# harbor.35.238.248.203.nip.io with its private-CA certificate. Both names
|
||||
# route to the same backends.
|
||||
#
|
||||
# THE PATH SPLIT IS LOAD-BEARING. harbor-core is the API, auth and registry
|
||||
# backend; harbor-portal is only the web UI. Mirrored from this cluster's own
|
||||
# generated Ingress (read from the live object, not copied blind): /api/,
|
||||
# /service/, /v2/ and /c/ go to core, everything else to the portal. Sending
|
||||
# /v2/ to the portal instead breaks every image pull, and it fails looking
|
||||
# like an authentication problem rather than a routing one.
|
||||
#
|
||||
# Contour matches the longest prefix, so the trailing / rule cannot shadow
|
||||
# the four above it regardless of order — they are written first anyway, so
|
||||
# the intent is obvious to whoever reads this next.
|
||||
#
|
||||
# WHAT THIS DOES NOT DO: Harbor still advertises the nip.io address.
|
||||
# externalURL is a single value handed to docker clients in Harbor's own API
|
||||
# responses, so the UI may redirect there and a docker login against this
|
||||
# hostname is issued a token endpoint pointing at the old one. Both names
|
||||
# resolve, so it works — but this name is not truly primary until
|
||||
# externalURL moves, and that cannot happen until every image reference in
|
||||
# the cluster moves with it. That is the nip.io removal, done deliberately
|
||||
# and on its own.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: harbor-ingress-deployshed
|
||||
namespace: harbor
|
||||
annotations:
|
||||
# A real certificate, unlike the chart's Ingress beside it: that one
|
||||
# serves a nip.io name, which Let's Encrypt cannot issue for, and is
|
||||
# signed by the private CA the node pool was told to trust instead.
|
||||
cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
spec:
|
||||
ingressClassName: contour
|
||||
tls:
|
||||
- hosts:
|
||||
- harbor.infra.deployshed.com
|
||||
secretName: harbor-deployshed-tls
|
||||
rules:
|
||||
- host: harbor.infra.deployshed.com
|
||||
http:
|
||||
paths:
|
||||
- path: /api/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /service/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /v2/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /c/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-portal
|
||||
port:
|
||||
number: 80
|
||||
@@ -0,0 +1,92 @@
|
||||
# Harbor's OLD hostname, kept alive on the private CA.
|
||||
#
|
||||
# This is the inverse of what this file used to be. Harbor's own chart now
|
||||
# serves harbor.infra.deployshed.com with a Let's Encrypt certificate, and
|
||||
# this standalone object serves the legacy nip.io name that the chart used
|
||||
# to own.
|
||||
#
|
||||
# WHY IT STILL EXISTS, and when it can go. Apps deployed before the move
|
||||
# recorded their image reference — harbor.35.238.248.203.nip.io/... — in
|
||||
# toolshed's database (apps.current_image) and in already-rendered pod
|
||||
# specs. Nothing in git can rewrite those; they change only when each app is
|
||||
# rebuilt and redeployed. Until then a running pod that restarts, reschedules
|
||||
# or scales up pulls the old reference, and if this name stops resolving that
|
||||
# pull fails. The failure would look like a broken registry rather than a
|
||||
# retired hostname.
|
||||
#
|
||||
# Delete this file, registry-ca-clusterissuer.yaml and registry-ca-configmap.yaml
|
||||
# together, once no image reference anywhere names the old host. Check with:
|
||||
#
|
||||
# kubectl get pods -A -o jsonpath='{range .items[*]}{.spec.containers[*].image}{"\n"}{end}' | grep nip.io
|
||||
# (and toolshed's own: select current_image from apps)
|
||||
#
|
||||
# The node pool's private_registry_access_config in Terraform retires at the
|
||||
# same time, but that one recycles nodes, so it belongs in a maintenance
|
||||
# window rather than in a cleanup commit.
|
||||
#
|
||||
# THE PATH SPLIT IS LOAD-BEARING. harbor-core is the API, auth and registry
|
||||
# backend; harbor-portal is only the web UI. /api/, /service/, /v2/ and /c/
|
||||
# go to core, everything else to the portal. Sending /v2/ to the portal
|
||||
# instead breaks every image pull, and it fails looking like an
|
||||
# authentication problem rather than a routing one.
|
||||
#
|
||||
# Contour matches the longest prefix, so the trailing / rule cannot shadow
|
||||
# the four above it regardless of order — they are written first anyway, so
|
||||
# the intent is obvious to whoever reads this next.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: harbor-ingress-legacy-nipio
|
||||
namespace: harbor
|
||||
annotations:
|
||||
# The private CA, not Let's Encrypt, which cannot issue for a nip.io
|
||||
# name: it is not on the public suffix list, and every *.nip.io
|
||||
# certificate shares one rate limit. The GKE node pool was told to trust
|
||||
# this CA for exactly this hostname when it was created, which is what
|
||||
# makes the old image references still pullable.
|
||||
cert-manager.io/cluster-issuer: registry-ca-issuer
|
||||
spec:
|
||||
ingressClassName: contour
|
||||
tls:
|
||||
- hosts:
|
||||
- harbor.35.238.248.203.nip.io
|
||||
secretName: harbor-tls
|
||||
rules:
|
||||
- host: harbor.35.238.248.203.nip.io
|
||||
http:
|
||||
paths:
|
||||
- path: /api/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /service/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /v2/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /c/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-portal
|
||||
port:
|
||||
number: 80
|
||||
@@ -29,13 +29,26 @@ spec:
|
||||
data:
|
||||
# Keyed by exact registry hostname — docker matches credentials
|
||||
# against the host portion of the image reference, so this must be
|
||||
# the same hostname the pipeline pushes to, the node pool was told
|
||||
# to trust, and Harbor's own externalURL uses. On GKE that is the
|
||||
# Contour ingress host, never harbor-core.harbor.svc.cluster.local:
|
||||
# cluster DNS resolves from a pod but not from the node's containerd
|
||||
# doing the actual image pull.
|
||||
# the same hostname the pipeline pushes to and Harbor's own
|
||||
# externalURL uses. On GKE that is the Contour ingress host, never
|
||||
# harbor-core.harbor.svc.cluster.local: cluster DNS resolves from a
|
||||
# pod but not from the node's containerd doing the actual image pull.
|
||||
#
|
||||
# BOTH hostnames are listed, deliberately, while the deployment moves
|
||||
# off nip.io. Because the match is by exact host and nothing else, a
|
||||
# single-value swap breaks whichever side lands second: change this
|
||||
# first and the pipeline (still pushing to nip.io) pushes
|
||||
# unauthenticated, which Harbor rejects as "unauthorized"; change the
|
||||
# pipeline first and the same happens in reverse. Two entries make
|
||||
# the order irrelevant — the credential is identical either way,
|
||||
# since it is one robot account on one Harbor reached by two names.
|
||||
#
|
||||
# The nip.io entry comes out once no image reference anywhere names
|
||||
# it: the pipeline's push target, the fallback Dockerfiles, the
|
||||
# mirrored base images, and every app already deployed from an image
|
||||
# tagged under the old name.
|
||||
.dockerconfigjson: |
|
||||
{"auths":{"harbor.35.238.248.203.nip.io":{"username":"{{ .username }}","password":"{{ .password }}","auth":"{{ printf "%s:%s" .username .password | b64enc }}"}}}
|
||||
{"auths":{"harbor.infra.deployshed.com":{"username":"{{ .username }}","password":"{{ .password }}","auth":"{{ printf "%s:%s" .username .password | b64enc }}"},"harbor.35.238.248.203.nip.io":{"username":"{{ .username }}","password":"{{ .password }}","auth":"{{ printf "%s:%s" .username .password | b64enc }}"}}}
|
||||
data:
|
||||
- secretKey: username
|
||||
remoteRef:
|
||||
|
||||
@@ -51,7 +51,7 @@ appSpec:
|
||||
valuesDir: gitea
|
||||
- name: contour
|
||||
# The cluster's only inbound path, pinned to the reserved IP that every
|
||||
# nip.io hostname is built from. Installed by ArgoCD rather than by
|
||||
# deployshed.com record resolves to. Installed by ArgoCD rather than by
|
||||
# hand, which is only possible because nothing in the bootstrap needed
|
||||
# an ingress: ArgoCD reaches Gitea over cluster DNS.
|
||||
nameOverride: contour
|
||||
|
||||
Reference in New Issue
Block a user