Serve every tool on deployshed.com instead of nip.io

Harbor, Gitea, Argo CD, Jenkins, Vault, Grafana and vmui now answer on
their deployshed.com names alone. Each was already serving both while the
move was proved out; this removes the nip.io half.

The dual-hostname workarounds go with it. Jenkins' secondaryingress existed
only because its chart's primary ingress takes one hostName and a
certificate could not span both names — the real domain moves onto the
primary with jenkins-tls, which it already holds. Argo CD gets extraTls
rather than ingress.tls, because the boolean hardcodes secretName
argocd-server-tls and would request a second certificate for a name that
already has a valid one in argocd-deployshed-tls.

Harbor also changes in two ways beyond the hostname:

  - externalURL moves to https://harbor.infra.deployshed.com. Harbor hands
    this to docker clients in its own API responses and builds the push
    commands shown in its UI from it, so a stale value is what makes a
    correctly-configured registry still advertise the old address.

  - updateStrategy is now Recreate. Its jobservice and registry volumes are
    standard-rwo (ReadWriteOnce), and a RollingUpdate starts the new pod
    before the old one releases the disk, so the replacement hangs forever
    on Multi-Attach. The cluster was sitting in exactly that state, old pods
    serving while new ones stayed in ContainerCreating. The chart's own
    comment on this value recommends Recreate when RWM is unavailable. The
    cost is a brief outage during upgrades, which beats a rollout that
    cannot complete.

The private registry CA is not removed yet. Apps deployed before this move
recorded nip.io image references that only change when each is rebuilt, so
the old hostname stays served by a standalone Ingress until then.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LEsTefWWifp4ikvhHF5s6N

helm-overrides/gke-toolshed-prd-usc1/argocd-admin-prd/custom-values.yaml

@@ -4,9 +4,18 @@ argo-cd:
   # Installed once by hand with `helm install argocd-admin-prd` (namespace
   # "argocd"), then manages itself through the argocd Application in
   # devops-infra-argo-config-gcp, whose nameOverride matches that release.
-  global:
-    image:
-      tag: "v2.13.8"
+  # Deliberately no image tag pin, unlike the homelab: the chart's own
+  # appVersion (v3.5.2) governs, so the image cannot drift from the chart.
+  # A pin that outlives its chart is close to the failure this upgrade
+  # fixes — software older than the cluster it manages.
+  #
+  # Upgraded from chart 7.7.23 / Argo CD v2.13.8. Three v3 behaviour changes
+  # apply to this deployment, none of which needs a values change today:
+  #   - logs RBAC is now enforced, so an account that reads pod logs needs
+  #     an explicit `logs, get` policy. jenkins-ci below only syncs.
+  #   - update/delete no longer inherit to an application's sub-resources.
+  #   - resource tracking moves from labels to annotations, so the first
+  #     sync after the upgrade re-stamps every managed resource.
 
   # SSO still deferred, same as the homelab.
   dex:
@@ -65,12 +74,30 @@ argo-cd:
       maxReplicas: 3
       targetCPUUtilizationPercentage: 70
       targetMemoryUtilizationPercentage: null
-    extraArgs:
-      - --insecure
+    # --insecure is NOT set here as an extra arg: configs.params below
+    # carries server.insecure, which is the supported way to express it and
+    # is what the chart renders into argocd-cmd-params-cm. Setting both
+    # works but leaves two places to disagree.
     ingress:
       enabled: true
       ingressClassName: contour
-      hostname: "argocd.35.238.248.203.nip.io"
+      hostname: "argocd.infra.deployshed.com"
+      # TLS via extraTls rather than the `tls: true` boolean, deliberately.
+      #
+      # The boolean hardcodes `secretName: argocd-server-tls` (see the
+      # chart's argocd-server/ingress.yaml). This deployment already holds a
+      # valid, issued certificate for this exact hostname in
+      # argocd-deployshed-tls, created by the standalone Ingress that served
+      # the real domain while nip.io was still on `hostname`. Flipping the
+      # boolean would ignore that and request a second certificate for the
+      # same name — a needless issuance and a gap while it is obtained.
+      #
+      # extraTls takes an explicit secretName, so the existing certificate is
+      # adopted as-is and the standalone Ingress can simply be deleted.
+      extraTls:
+        - hosts:
+            - argocd.infra.deployshed.com
+          secretName: argocd-deployshed-tls
     resources:
       requests:
         cpu: 50m
@@ -79,14 +106,35 @@ argo-cd:
         cpu: 200m
         memory: 256Mi
 
+  # applicationSet.enabled no longer exists in this chart, and there is no
+  # replacement: unlike dex and notifications below, the ApplicationSet
+  # controller's Deployment has no conditional at all. replicas: 0 is the
+  # only lever — the Deployment exists but runs nothing. Carrying the old
+  # `enabled: false` forward would have quietly started the controller,
+  # since Helm ignores unknown keys.
+  #
+  # Nothing here uses the ApplicationSet CRD; Applications are rendered by
+  # generic-argo-apps-chart instead.
   applicationSet:
-    enabled: false
+    replicas: 0
   notifications:
     enabled: false
 
   configs:
+    # Contour terminates TLS in front of Argo CD; leaving Argo CD's own TLS
+    # on as well produces a redirect loop. This renders into
+    # argocd-cmd-params-cm, which the server actually reads.
+    #
+    # This file previously expressed it as server.extraArgs: [--insecure],
+    # inherited from the homelab. Both work, but only one should exist, and
+    # the rendered ConfigMap is the thing to check when it looks wrong.
+    params:
+      server.insecure: true
     cm:
-      url: "http://argocd.35.238.248.203.nip.io"
+      # https now that the only hostname served carries a real certificate.
+      # This is what ArgoCD builds its own links from, so leaving it http
+      # would hand out plain-HTTP URLs for a TLS-only deployment.
+      url: "https://argocd.infra.deployshed.com"
       timeout.reconciliation: 3m
       timeout.reconciliation.jitter: 60s
       # No Ingress health override, unlike the homelab: there Contour sat

helm-overrides/gke-toolshed-prd-usc1/contour/custom-values.yaml

@@ -0,0 +1,70 @@
+contour:
+  # GKE counterpart of helm-overrides/k8s-admin-prd-ase1/contour. Same
+  # official projectcontour chart (0.7.0, see helm-templates/contour), but
+  # exposed the opposite way.
+  #
+  # The homelab binds Envoy to node ports 80/443 with hostPort, because
+  # VMware bridging over Wi-Fi never made a LoadBalancer IP reachable
+  # (claude.md issue #6). None of that applies here: this is a real cloud
+  # load balancer on the reserved address, and it is the ONE inbound path
+  # into the cluster now that the nodes have no public IPs of their own.
+
+  contour:
+    replicaCount: 1
+    # Ingress objects across this cluster say `ingressClassName: contour`,
+    # so the class must be created under exactly that name. The chart's
+    # default is an empty string, which derives a name from the release.
+    ingressClass:
+      name: contour
+      create: true
+      default: true
+    resources:
+      requests:
+        cpu: 50m
+        memory: 64Mi
+      limits:
+        memory: 192Mi
+
+  envoy:
+    # DaemonSet (the chart default): one Envoy per node, which pairs with
+    # externalTrafficPolicy: Local below — every node the load balancer can
+    # send to is running a proxy that can serve the request locally.
+    kind: daemonset
+
+    service:
+      type: LoadBalancer
+
+      # The reserved address from Terraform (module.network's
+      # google_compute_address). Every hostname in this deployment — the
+      # deployshed.com records, including the two wildcards — resolves here,
+      # so this pin is what makes DNS work at all: an unpinned Service takes
+      # a fresh ephemeral IP and every hostname points at nothing.
+      #
+      # More load-bearing now, not less, than when hostnames were
+      # <name>.35.238.248.203.nip.io. Those encoded the address, so a changed
+      # IP produced names that were merely wrong. Real DNS records point here
+      # until somebody edits them in Cloudflare, so a changed IP is an
+      # outage across every hostname at once.
+      #
+      # spec.loadBalancerIP is deprecated upstream (Kubernetes 1.24), and
+      # GKE's replacement is the annotation
+      # networking.gke.io/load-balancer-ip-addresses. That annotation is NOT
+      # a drop-in: it takes the address resource's NAME rather than the
+      # address, and on an external Service it also requires
+      # spec.loadBalancerClass: networking.gke.io/l4-regional-external,
+      # which changes which controller programs the load balancer. GKE still
+      # honours this field, so the deprecated-but-working one is the smaller
+      # change; revisit if a GKE upgrade ever stops honouring it.
+      loadBalancerIP: "35.238.248.203"
+
+      # Chart default, kept deliberately: preserves the real client IP
+      # instead of replacing it with a node's address. Valid here precisely
+      # because Envoy is a DaemonSet.
+      externalTrafficPolicy: Local
+
+    resources:
+      requests:
+        cpu: 50m
+        memory: 96Mi
+      limits:
+        memory: 256Mi

helm-overrides/gke-toolshed-prd-usc1/external-secrets/custom-values.yaml

@@ -0,0 +1,42 @@
+external-secrets:
+  # Same as the homelab's, which carries nothing cluster-specific: the
+  # controller is configured entirely by the ClusterSecretStore and
+  # ExternalSecret objects in devops-infra-argo-config-gcp, not by values.
+  #
+  # This is the piece every credential in the cluster hangs off — Harbor,
+  # Jenkins, Grafana and the pipeline all read their secrets from Vault
+  # through it, so it comes up before any of them.
+  #
+  # Two things must exist in Vault before the first ExternalSecret can sync,
+  # and neither is declarative: the KV v2 engine at secret/, and the
+  # Kubernetes auth method with a role bound to this controller's service
+  # account. Until then ExternalSecrets stay in a retry loop rather than
+  # failing outright.
+  #
+  # installCRDs defaults to true — kept, as on a fresh cluster there are no
+  # existing SecretStore/ExternalSecret CRs whose schema it could clobber.
+  #
+  # All three components default to unbounded resources; trimmed here for
+  # the same reason as everything else in this repo.
+  resources:
+    requests:
+      cpu: 25m
+      memory: 32Mi
+    limits:
+      memory: 128Mi
+
+  webhook:
+    resources:
+      requests:
+        cpu: 25m
+        memory: 32Mi
+      limits:
+        memory: 64Mi
+
+  certController:
+    resources:
+      requests:
+        cpu: 25m
+        memory: 32Mi
+      limits:
+        memory: 64Mi

helm-overrides/gke-toolshed-prd-usc1/gitea/custom-values.yaml

@@ -43,17 +43,33 @@ gitea:
       actions:
         ENABLED: true
       server:
-        ROOT_URL: http://gitea.35.238.248.203.nip.io/
+        ROOT_URL: https://gitea.infra.deployshed.com/
       service:
         # The homelab sat on a LAN; this Gitea is on a public IP. Open
         # registration would let anyone on the internet create an account.
         DISABLE_REGISTRATION: true
       security:
         # The homelab allowed "*" because every host was on a private LAN.
-        # Here "*" would also allow webhooks to the node metadata server,
-        # so this is narrowed to private ranges — which still covers every
+        # Here "*" would also allow webhooks to the node metadata server, so
+        # this stays narrowed to private ranges — which covers every
         # in-cluster Service (Jenkins included) reached over cluster DNS.
-        ALLOWED_HOST_LIST: private
+        #
+        # The one public entry is toolshed's own dashboard host, and it is a
+        # STOPGAP. toolshed builds each app's webhook target from the
+        # hostname the dashboard was browsed on (internal/api/apps.go's
+        # queueRepo), with no override, so an app registered through the
+        # public URL gets a public webhook target and Gitea refuses to call
+        # it: "webhook can only call allowed HTTP servers".
+        #
+        # The cost is real but bounded: this permits exactly one hostname,
+        # which happens to be our own load balancer, so the callback
+        # hairpins out and back in rather than staying pod-to-pod. It does
+        # not re-expose the metadata server, which is why "*" was rejected.
+        #
+        # The proper fix is a configurable webhook base URL in toolshed
+        # pointing at toolshed-api.toolshed.svc.cluster.local:8080, after
+        # which this entry should be removed.
+        ALLOWED_HOST_LIST: private,console.deployshed.com
     admin:
       username: gitadmin
       # Created by hand with kubectl at bootstrap, because Vault and ESO
@@ -64,12 +80,27 @@ gitea:
       email: "admin@local.lab"
 
   # Contour does not exist yet at bootstrap — the Ingress just sits unused
-  # until ArgoCD installs it. One host only: no Tailscale on GKE.
+  # until ArgoCD installs it.
+  #
+  # One host. The nip.io name was served alongside this one while the
+  # deployment moved onto its own domain, and came out once everything
+  # referencing it had been repointed: ROOT_URL above, the webhook allow-list
+  # above that, and any git remote anyone had configured.
   ingress:
     enabled: true
     className: contour
+    annotations:
+      # Issues the certificate named in tls below. This could only ever cover
+      # the real domain: Let's Encrypt cannot issue for nip.io, so while both
+      # names were served, asking for one certificate spanning them returned
+      # nothing for either.
+      cert-manager.io/cluster-issuer: letsencrypt-prod
     hosts:
-      - host: gitea.35.238.248.203.nip.io
+      - host: gitea.infra.deployshed.com
         paths:
           - path: /
             pathType: Prefix
+    tls:
+      - secretName: gitea-tls
+        hosts:
+          - gitea.infra.deployshed.com

helm-overrides/gke-toolshed-prd-usc1/grafana/custom-values.yaml

@@ -0,0 +1,283 @@
+grafana:
+  # From Vault via ExternalSecret (devops-infra-argo-config/secretstores/
+  # grafana-admin-credentials.yaml), same pattern as gitea/harbor/jenkins
+  # admin credentials elsewhere in this project — never a plaintext
+  # adminPassword in this file.
+  admin:
+    existingSecret: grafana-admin-credentials
+    userKey: username
+    passwordKey: password
+
+  persistence:
+    # 1Gi, not the chart's 10Gi default: this holds dashboards, folders and
+    # Grafana's own sqlite state, not metric data — VictoriaMetrics keeps
+    # that. Unlike the homelab's local-path, standard-rwo can be expanded
+    # later if that ever proves tight.
+    enabled: true
+    storageClassName: standard-rwo
+    size: 1Gi
+
+  resources:
+    requests:
+      cpu: 50m
+      memory: 128Mi
+    limits:
+      memory: 384Mi
+
+  # Provisioned at boot, not clicked through in the UI — the same reason
+  # every other credential/config in this project is committed rather
+  # than set by hand: it survives a pod restart and a fresh install gets
+  # it automatically. VictoriaMetrics speaks Prometheus's own query API,
+  # so `type: prometheus` here is correct even though the URL is VM's —
+  # see this repo's victoria-metrics-single chart for why.
+  datasources:
+    datasources.yaml:
+      apiVersion: 1
+      datasources:
+        - name: VictoriaMetrics
+          # Fixed uid, not left to auto-generate — the provisioned
+          # dashboard below references this datasource by uid, and an
+          # auto-generated one would only exist after Grafana's first
+          # boot, too late for a dashboard provisioned in the same boot.
+          uid: victoriametrics
+          type: prometheus
+          access: proxy
+          url: http://victoria-metrics-single-server.monitoring.svc.cluster.local:8428
+          isDefault: true
+
+  # Dashboard provisioning. Provisioned rather than built by hand in the
+  # UI for the same reason the datasource above is: it survives a pod
+  # restart (this deployment has no persistent Grafana database beyond
+  # the 1Gi PVC, and even with one, a fresh install should not start
+  # with an empty dashboard list) and a `git diff` shows what changed.
+  dashboardProviders:
+    dashboardproviders.yaml:
+      apiVersion: 1
+      providers:
+        - name: default
+          orgId: 1
+          folder: ""
+          type: file
+          disableDeletion: false
+          editable: true
+          options:
+            path: /var/lib/grafana/dashboards/default
+
+  # "Homelab Overview" — total Envoy/Contour RPS, per-namespace CPU and
+  # memory (the $namespace template variable filters every relevant
+  # panel), cluster-wide utilization against actual node capacity, and a
+  # total-resources row (cores/memory/pods/disk). Envoy and node-exporter
+  # metrics both required their own vmagent scrape job — see
+  # helm-overrides/.../vmagent/custom-values.yaml for why neither was
+  # reachable through the chart's own defaults in this cluster.
+  #
+  # Every panel except "Disk free" was run against the live deployment
+  # (vmui, over Tailscale) before being written in here — RPS, per-
+  # namespace CPU/memory, machine_cpu_cores/machine_memory_bytes all
+  # returned real data. "Disk free" depends on the node-exporter scrape
+  # job added alongside this same change, which had not been live yet to
+  # verify against — worth checking once this actually deploys, same as
+  # everything else in this repo that gets a `helm template` check but
+  # cannot get a live one before the first sync.
+  dashboards:
+    default:
+      homelab:
+        json: |
+          {
+            "title": "Homelab Overview",
+            "uid": "homelab-overview",
+            "schemaVersion": 39,
+            "editable": true,
+            "timezone": "browser",
+            "time": { "from": "now-1h", "to": "now" },
+            "refresh": "30s",
+            "templating": {
+              "list": [
+                {
+                  "name": "namespace",
+                  "type": "query",
+                  "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                  "query": "label_values(container_memory_working_set_bytes{container!=\"\", container!=\"POD\"}, namespace)",
+                  "refresh": 2,
+                  "multi": true,
+                  "includeAll": true,
+                  "current": { "selected": true, "text": "All", "value": "$__all" }
+                }
+              ]
+            },
+            "panels": [
+              { "type": "row", "title": "Ingress (Envoy / Contour)", "gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 }, "id": 100 },
+
+              {
+                "type": "stat", "title": "Total RPS", "id": 1,
+                "gridPos": { "h": 6, "w": 6, "x": 0, "y": 1 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "sum(rate(envoy_http_downstream_rq_total{namespace=\"projectcontour\"}[5m]))", "legendFormat": "rps" }],
+                "fieldConfig": { "defaults": { "unit": "reqps", "decimals": 2 }, "overrides": [] },
+                "options": { "reduceOptions": { "calcs": ["lastNotNull"] }, "graphMode": "area" }
+              },
+              {
+                "type": "stat", "title": "Active downstream connections", "id": 2,
+                "gridPos": { "h": 6, "w": 6, "x": 6, "y": 1 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "sum(envoy_http_downstream_cx_active{namespace=\"projectcontour\"})", "legendFormat": "connections" }],
+                "fieldConfig": { "defaults": { "unit": "short" }, "overrides": [] }
+              },
+              {
+                "type": "timeseries", "title": "Requests by response class", "id": 3,
+                "gridPos": { "h": 6, "w": 12, "x": 12, "y": 1 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{
+                  "expr": "sum by (envoy_response_code_class) (rate(envoy_http_downstream_rq_xx{namespace=\"projectcontour\"}[5m]))",
+                  "legendFormat": "{{envoy_response_code_class}}xx"
+                }],
+                "fieldConfig": { "defaults": { "unit": "reqps" }, "overrides": [] },
+                "options": { "legend": { "displayMode": "list", "placement": "bottom" } }
+              },
+
+              { "type": "row", "title": "Service level (by namespace)", "gridPos": { "h": 1, "w": 24, "x": 0, "y": 7 }, "id": 101 },
+
+              {
+                "type": "timeseries", "title": "CPU usage by namespace", "id": 10,
+                "gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{
+                  "expr": "sum by (namespace) (rate(container_cpu_usage_seconds_total{namespace=~\"$namespace\", container!=\"\", container!=\"POD\"}[5m]))",
+                  "legendFormat": "{{namespace}}"
+                }],
+                "fieldConfig": { "defaults": { "unit": "short", "custom": { "fillOpacity": 10, "stacking": { "mode": "normal" } } }, "overrides": [] },
+                "options": { "legend": { "displayMode": "table", "placement": "right", "calcs": ["mean", "max"] } }
+              },
+              {
+                "type": "timeseries", "title": "Memory usage by namespace", "id": 11,
+                "gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{
+                  "expr": "sum by (namespace) (container_memory_working_set_bytes{namespace=~\"$namespace\", container!=\"\", container!=\"POD\"})",
+                  "legendFormat": "{{namespace}}"
+                }],
+                "fieldConfig": { "defaults": { "unit": "bytes", "custom": { "fillOpacity": 10, "stacking": { "mode": "normal" } } }, "overrides": [] },
+                "options": { "legend": { "displayMode": "table", "placement": "right", "calcs": ["mean", "max"] } }
+              },
+              {
+                "type": "table", "title": "Current usage per namespace", "id": 12,
+                "gridPos": { "h": 8, "w": 24, "x": 0, "y": 16 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [
+                  { "expr": "sum by (namespace) (rate(container_cpu_usage_seconds_total{namespace=~\"$namespace\", container!=\"\", container!=\"POD\"}[5m]))", "format": "table", "instant": true, "refId": "A" },
+                  { "expr": "sum by (namespace) (container_memory_working_set_bytes{namespace=~\"$namespace\", container!=\"\", container!=\"POD\"})", "format": "table", "instant": true, "refId": "B" },
+                  { "expr": "count by (namespace) (count by (namespace, pod) (container_memory_working_set_bytes{namespace=~\"$namespace\", container!=\"\", container!=\"POD\"}))", "format": "table", "instant": true, "refId": "C" }
+                ],
+                "transformations": [
+                  { "id": "merge", "options": {} },
+                  { "id": "organize", "options": {
+                      "excludeByName": { "Time": true, "Time 1": true, "Time 2": true, "Time 3": true },
+                      "renameByName": { "Value #A": "CPU (cores)", "Value #B": "Memory", "Value #C": "Pods" }
+                  } }
+                ],
+                "fieldConfig": { "defaults": {}, "overrides": [
+                  { "matcher": { "id": "byName", "options": "Memory" }, "properties": [{ "id": "unit", "value": "bytes" }] },
+                  { "matcher": { "id": "byName", "options": "CPU (cores)" }, "properties": [{ "id": "unit", "value": "short" }, { "id": "decimals", "value": 3 }] }
+                ] }
+              },
+
+              { "type": "row", "title": "Cluster utilization", "gridPos": { "h": 1, "w": 24, "x": 0, "y": 24 }, "id": 102 },
+
+              {
+                "type": "gauge", "title": "CPU utilization", "id": 20,
+                "gridPos": { "h": 7, "w": 6, "x": 0, "y": 25 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "100 * sum(rate(container_cpu_usage_seconds_total{container!=\"\", container!=\"POD\"}[5m])) / sum(machine_cpu_cores)" }],
+                "fieldConfig": { "defaults": { "unit": "percent", "min": 0, "max": 100,
+                  "thresholds": { "mode": "absolute", "steps": [
+                    { "color": "green", "value": null }, { "color": "yellow", "value": 70 }, { "color": "red", "value": 90 }
+                  ] } }, "overrides": [] }
+              },
+              {
+                "type": "gauge", "title": "Memory utilization", "id": 21,
+                "gridPos": { "h": 7, "w": 6, "x": 6, "y": 25 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "100 * sum(container_memory_working_set_bytes{container!=\"\", container!=\"POD\"}) / sum(machine_memory_bytes)" }],
+                "fieldConfig": { "defaults": { "unit": "percent", "min": 0, "max": 100,
+                  "thresholds": { "mode": "absolute", "steps": [
+                    { "color": "green", "value": null }, { "color": "yellow", "value": 70 }, { "color": "red", "value": 90 }
+                  ] } }, "overrides": [] }
+              },
+              {
+                "type": "timeseries", "title": "Cluster CPU utilization over time", "id": 22,
+                "gridPos": { "h": 7, "w": 12, "x": 12, "y": 25 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [
+                  { "expr": "100 * sum(rate(container_cpu_usage_seconds_total{container!=\"\", container!=\"POD\"}[5m])) / sum(machine_cpu_cores)", "legendFormat": "CPU %" },
+                  { "expr": "100 * sum(container_memory_working_set_bytes{container!=\"\", container!=\"POD\"}) / sum(machine_memory_bytes)", "legendFormat": "Memory %" }
+                ],
+                "fieldConfig": { "defaults": { "unit": "percent", "min": 0 }, "overrides": [] }
+              },
+
+              { "type": "row", "title": "Total resources", "gridPos": { "h": 1, "w": 24, "x": 0, "y": 32 }, "id": 103 },
+
+              {
+                "type": "stat", "title": "Node CPU capacity", "id": 30,
+                "gridPos": { "h": 5, "w": 4, "x": 0, "y": 33 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "max(machine_cpu_cores)" }],
+                "fieldConfig": { "defaults": { "unit": "short", "displayName": "cores" }, "overrides": [] }
+              },
+              {
+                "type": "stat", "title": "Node memory capacity", "id": 31,
+                "gridPos": { "h": 5, "w": 4, "x": 4, "y": 33 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "max(machine_memory_bytes)" }],
+                "fieldConfig": { "defaults": { "unit": "bytes" }, "overrides": [] }
+              },
+              {
+                "type": "stat", "title": "CPU used (cluster)", "id": 32,
+                "gridPos": { "h": 5, "w": 4, "x": 8, "y": 33 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "sum(rate(container_cpu_usage_seconds_total{container!=\"\", container!=\"POD\"}[5m]))" }],
+                "fieldConfig": { "defaults": { "unit": "short", "displayName": "cores", "decimals": 2 }, "overrides": [] }
+              },
+              {
+                "type": "stat", "title": "Memory used (cluster)", "id": 33,
+                "gridPos": { "h": 5, "w": 4, "x": 12, "y": 33 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "sum(container_memory_working_set_bytes{container!=\"\", container!=\"POD\"})" }],
+                "fieldConfig": { "defaults": { "unit": "bytes" }, "overrides": [] }
+              },
+              {
+                "type": "stat", "title": "Running pods", "id": 34,
+                "gridPos": { "h": 5, "w": 4, "x": 16, "y": 33 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "count(count by (namespace, pod) (container_memory_working_set_bytes{container!=\"\", container!=\"POD\"}))" }],
+                "fieldConfig": { "defaults": { "unit": "short" }, "overrides": [] }
+              },
+              {
+                "type": "stat", "title": "Disk free (root)", "id": 35,
+                "gridPos": { "h": 5, "w": 4, "x": 20, "y": 33 },
+                "datasource": { "type": "prometheus", "uid": "victoriametrics" },
+                "targets": [{ "expr": "node_filesystem_avail_bytes{mountpoint=\"/\"}" }],
+                "fieldConfig": { "defaults": { "unit": "bytes",
+                  "thresholds": { "mode": "absolute", "steps": [
+                    { "color": "red", "value": null }, { "color": "yellow", "value": 5000000000 }, { "color": "green", "value": 15000000000 }
+                  ] } }, "overrides": [] }
+              }
+            ]
+          }
+
+  ingress:
+    enabled: true
+    ingressClassName: contour
+    annotations:
+      # The real domain is now the only host, so this covers everything
+      # served. While nip.io was alongside it, only the real domain could
+      # appear in tls below — Let's Encrypt cannot issue for nip.io, and one
+      # certificate spanning both would have failed outright rather than
+      # covering the half it could serve.
+      cert-manager.io/cluster-issuer: letsencrypt-prod
+    path: /
+    hosts:
+      - grafana.infra.deployshed.com
+    tls:
+      - secretName: grafana-tls
+        hosts:
+          - grafana.infra.deployshed.com

helm-overrides/gke-toolshed-prd-usc1/harbor/custom-values.yaml

@@ -0,0 +1,160 @@
+harbor:
+  # GKE counterpart of helm-overrides/k8s-admin-prd-ase1/harbor, same chart
+  # (1.19.1), with one fundamental difference: this Harbor is served over
+  # real TLS, and it has to be.
+  #
+  # The homelab runs Harbor on plain HTTP and works around containerd's
+  # refusal to pull from an insecure registry by hand-editing
+  # /etc/containerd/certs.d/<host>/hosts.toml on the node. GKE nodes are
+  # managed and replaced, so that edit cannot survive.
+  #
+  # That used to mean a private CA: the registry was a nip.io name, Let's
+  # Encrypt cannot issue for one (not on the public suffix list, and every
+  # *.nip.io certificate shares a single rate limit), so Terraform generated
+  # a CA, the node pool was told at creation to trust it for exactly that
+  # hostname, and cert-manager signed Harbor from it. It worked for pulls,
+  # which is what mattered, but every browser warned on the Harbor UI and
+  # every docker client that was not a node needed the CA mounted by hand.
+  #
+  # Owning a domain removes all of it. harbor.infra.deployshed.com takes an
+  # ordinary Let's Encrypt certificate that everything already trusts — the
+  # nodes, dockerd in a build pod, a laptop, a browser. What retires with it:
+  # registry-ca-clusterissuer.yaml, registry-ca-configmap.yaml, the CA mount
+  # in every dind pod, and eventually the node pool's own
+  # private_registry_access_config block in Terraform.
+  #
+  # The nip.io name is still served, by a standalone Ingress alongside this
+  # one, and still signed by the private CA. It has to be: apps deployed
+  # before the move recorded their image as harbor.35.238.248.203.nip.io/...
+  # in toolshed's database, and that reference only changes when each app is
+  # rebuilt. Retiring the old name before then breaks their next image pull.
+  expose:
+    type: ingress
+    tls:
+      enabled: true
+      # secret, not the chart's "auto": auto generates its own self-signed
+      # certificate, which nothing has any reason to trust.
+      certSource: secret
+      secret:
+        secretName: harbor-deployshed-tls
+    ingress:
+      hosts:
+        core: "harbor.infra.deployshed.com"
+      className: contour
+      annotations:
+        # cert-manager's ingress-shim watches for this and creates the
+        # Certificate itself, writing the result into the secret named
+        # above. Nothing here ever touches a Certificate resource directly.
+        cert-manager.io/cluster-issuer: letsencrypt-prod
+
+  # The update strategy for the two deployments with persistent volumes,
+  # jobservice and registry. Recreate, not the chart's default RollingUpdate,
+  # because their volumes are standard-rwo — ReadWriteOnce — and a rolling
+  # update starts the replacement pod before the old one releases the disk.
+  # The new pod then waits on "Multi-Attach error for volume ... already used
+  # by pod" forever, and the rollout never converges: this cluster sat in
+  # exactly that state, old pods serving while new ones hung in
+  # ContainerCreating. The chart's own comment on this value says to set it
+  # this way when RWM volumes are unavailable.
+  #
+  # The cost is honest: Harbor is briefly down during an upgrade, because the
+  # old pod must stop before the new one starts. That beats an update that
+  # cannot complete at all.
+  updateStrategy:
+    type: Recreate
+
+  # https, matching the ingress above. Harbor hands this URL to docker
+  # clients in its own API responses, so a mismatch here breaks pushes in
+  # ways that look like registry errors rather than configuration — and it is
+  # what the "docker login / docker push" commands shown in Harbor's own UI
+  # are built from, which is where a stale value is noticed first.
+  externalURL: "https://harbor.infra.deployshed.com"
+
+  # From Vault through External Secrets — see
+  # secretstores/harbor-admin-credentials.yaml and Vault path
+  # secret/harbor/admin.
+  existingSecretAdminPassword: harbor-admin-credentials
+  existingSecretAdminPasswordKey: HARBOR_ADMIN_PASSWORD
+
+  # The one genuinely optional component. Harbor's database and redis are
+  # its own required internal state, not add-ons.
+  trivy:
+    enabled: false
+
+  persistence:
+    enabled: true
+    # Keeps the PVCs if the release is ever uninstalled: these hold the
+    # actual images.
+    resourcePolicy: "keep"
+    persistentVolumeClaim:
+      registry:
+        storageClass: standard-rwo
+        size: 5Gi
+      jobservice:
+        jobLog:
+          storageClass: standard-rwo
+          size: 1Gi
+      database:
+        storageClass: standard-rwo
+        size: 1Gi
+      redis:
+        storageClass: standard-rwo
+        size: 1Gi
+
+  portal:
+    resources:
+      requests:
+        cpu: 50m
+        memory: 128Mi
+      limits:
+        memory: 256Mi
+
+  core:
+    resources:
+      requests:
+        cpu: 100m
+        memory: 256Mi
+      limits:
+        memory: 512Mi
+
+  jobservice:
+    resources:
+      requests:
+        cpu: 50m
+        memory: 128Mi
+      limits:
+        memory: 256Mi
+
+  registry:
+    registry:
+      resources:
+        requests:
+          cpu: 100m
+          memory: 128Mi
+        limits:
+          memory: 256Mi
+    controller:
+      resources:
+        requests:
+          cpu: 50m
+          memory: 64Mi
+        limits:
+          memory: 128Mi
+
+  database:
+    internal:
+      resources:
+        requests:
+          cpu: 100m
+          memory: 256Mi
+        limits:
+          memory: 512Mi
+
+  redis:
+    internal:
+      resources:
+        requests:
+          cpu: 50m
+          memory: 64Mi
+        limits:
+          memory: 128Mi

helm-overrides/gke-toolshed-prd-usc1/jenkins/custom-values.yaml

@@ -0,0 +1,94 @@
+jenkins:
+  # GKE counterpart of helm-overrides/k8s-admin-prd-ase1/jenkins, same chart
+  # (5.8.58). Dynamic Kubernetes build agents, so idle cost is the
+  # controller alone.
+  #
+  # Every plugin pin below is carried over deliberately, not copied
+  # blindly — each one fixes a failure that is not obvious from its symptom.
+  # See the homelab file for the full history.
+
+  controller:
+    image:
+      # The chart's own default plugin list requires core >= 2.504.3, while
+      # its default image tag is 2.504.2. An upstream inconsistency in the
+      # chart, not our configuration: bumping core is the fix, since the
+      # plugins involved (kubernetes above all) are what dynamic agents
+      # depend on.
+      tag: "2.504.3-jdk21"
+
+    # Helm replaces lists wholesale rather than merging, so this is the
+    # chart's full default list with one version corrected, plus two
+    # additions — not a hand-picked subset.
+    installPlugins:
+      # Pinned as a pair. kubernetes needs kubernetes-client-api >=
+      # 7.3.1-256.v788a_0b_787114; left unpinned it resolves to an older
+      # version at image-build time and every agent launch dies with
+      # NoSuchMethodError while constructing the client. The pods start
+      # fine, so it presents as builds hanging forever at "Still waiting to
+      # schedule task" rather than as a plugin problem.
+      - kubernetes:4437.v3a_18554d3f32
+      - kubernetes-client-api:7.3.1-256.v788a_0b_787114
+      - workflow-aggregator:608.v67378e9d3db_1
+      - git:5.7.0
+      # kubernetes/git/credentials need this version, though the chart's
+      # own default list pins an older one. Same class of upstream
+      # inconsistency as the image tag.
+      - configuration-as-code:2006.v001a_2ca_6b_574
+      # Not in the chart's default list at all — provides readYaml, which
+      # the shared library's loadConfig stage uses to parse each repo's
+      # config.yaml.
+      - pipeline-utility-steps:3.810.va_7672d206740
+
+    resources:
+      requests:
+        cpu: 100m
+        memory: 512Mi
+      limits:
+        cpu: 500m
+        memory: 1Gi
+
+    admin:
+      # From Vault through External Secrets, never a plaintext password
+      # here. Requires secretstores/jenkins-admin-credentials.yaml to have
+      # synced, which in turn requires the Vault path secret/jenkins/admin.
+      existingSecret: jenkins-admin-credentials
+      userKey: jenkins-admin-user
+      passwordKey: jenkins-admin-password
+
+    # One hostname, on the primary ingress, with its certificate.
+    #
+    # This chart's primary ingress supports exactly one hostName — no
+    # extraHosts like argo-cd — so while nip.io was also served, the real
+    # domain lived in a whole second Ingress object (secondaryingress). That
+    # was the only way to keep the certificate clean, since a certificate
+    # covering both names is impossible: Let's Encrypt cannot issue for
+    # nip.io. With nip.io gone there is one name, so the second object is
+    # gone with it and the certificate moves onto the primary.
+    #
+    # controller.ingress.tls is a LIST here, not a boolean, taking an
+    # explicit secretName — so jenkins-tls, already issued for this exact
+    # hostname by the secondary ingress, is adopted rather than reissued.
+    ingress:
+      enabled: true
+      hostName: "jenkins.infra.deployshed.com"
+      ingressClassName: contour
+      annotations:
+        cert-manager.io/cluster-issuer: letsencrypt-prod
+      tls:
+        - secretName: jenkins-tls
+          hosts:
+            - jenkins.infra.deployshed.com
+
+  agent:
+    resources:
+      requests:
+        cpu: 250m
+        memory: 256Mi
+      limits:
+        cpu: 500m
+        memory: 512Mi
+
+  persistence:
+    enabled: true
+    storageClass: standard-rwo
+    size: 5Gi

helm-overrides/gke-toolshed-prd-usc1/node-exporter/custom-values.yaml

@@ -0,0 +1,17 @@
+prometheus-node-exporter:
+  # Unchanged from the homelab's — nothing here is cluster-specific. One
+  # DaemonSet pod per node on hostNetwork; three pods here rather than one.
+  #
+  # Host-level metrics are independent of which TSDB stores them, which is
+  # why this is its own release rather than a subchart of anything.
+  resources:
+    requests:
+      cpu: 20m
+      memory: 32Mi
+    limits:
+      memory: 64Mi
+
+  # The chart's Service carries prometheus.io/scrape by default. vmagent
+  # also targets these pods directly by container port, because that
+  # annotation-based path found nothing in the homelab — see the vmagent
+  # values for the detail.

helm-overrides/gke-toolshed-prd-usc1/postgresql/custom-values.yaml

@@ -0,0 +1,45 @@
+# PostgreSQL for toolshed's control plane, on GKE.
+#
+# Its own namespace rather than toolshed's, so it is addressed over cluster
+# DNS like any other platform component and outlives its first consumer:
+#
+#   postgresql.postgres.svc.cluster.local:5432
+#
+# Hand-written chart, not Bitnami's: that registry has been unstable, and
+# PostgreSQL ships no official chart.
+#
+# Credentials come from Vault through External Secrets — see
+# secretstores/toolshed-postgres-credentials.yaml. The Secret must exist
+# before this pod starts; without it the pod sits in
+# CreateContainerConfigError, which does not explain itself.
+
+fullnameOverride: postgresql
+
+image:
+  repository: postgres
+  tag: "16-alpine"
+  pullPolicy: IfNotPresent
+
+existingSecret: postgresql-credentials
+database: toolshed
+
+persistence:
+  enabled: true
+  storageClass: standard-rwo
+  size: 5Gi
+
+config:
+  # Kept at the homelab's deliberately small values even though this
+  # cluster has room to spare: for a handful of control-plane tables it
+  # makes no measurable difference, and matching the homelab keeps one
+  # fewer variable between the two deployments. Raise it if a real query
+  # workload ever shows up here.
+  sharedBuffers: 32MB
+  maxConnections: "50"
+
+resources:
+  requests:
+    cpu: 50m
+    memory: 64Mi
+  limits:
+    memory: 256Mi

helm-overrides/gke-toolshed-prd-usc1/redis/custom-values.yaml

@@ -0,0 +1,45 @@
+# Redis backing toolshed's managed cache add-on, on GKE.
+#
+# Its own namespace, same reasoning as postgresql:
+#
+#   redis.redis.svc.cluster.local:6379
+#
+# Hand-written chart, not Bitnami's, for the same reason as postgresql.
+#
+# Read the chart's own values.yaml before changing anything about
+# authentication. The absence of requirepass is deliberate and
+# security-relevant, not an oversight: access is defined by an ACL file
+# seeded from Vault through External Secrets
+# (secretstores/toolshed-redis-credentials.yaml). The Secret must exist
+# before this pod starts, or the init container cannot seed that file.
+
+fullnameOverride: redis
+
+image:
+  repository: redis
+  tag: "7-alpine"
+  pullPolicy: IfNotPresent
+
+existingSecret: redis-credentials
+
+persistence:
+  enabled: true
+  storageClass: standard-rwo
+  # Holds the ACL file and nothing else worth keeping, since snapshotting
+  # is off. 1Gi is already far more than needed.
+  size: 1Gi
+
+config:
+  # Kept at the homelab's value. It was chosen there to fit an 8GB node
+  # under pressure, and while this cluster has room, a bigger cache buys
+  # nothing for a handful of small internal tools.
+  maxmemory: 48mb
+  maxmemoryPolicy: allkeys-lru
+  save: ""
+
+resources:
+  requests:
+    cpu: 25m
+    memory: 32Mi
+  limits:
+    memory: 96Mi

helm-overrides/gke-toolshed-prd-usc1/vault/custom-values.yaml

@@ -0,0 +1,99 @@
+vault:
+  # GKE counterpart of helm-overrides/k8s-admin-prd-ase1/vault, same chart
+  # (0.34.1) and same shape — production mode, file storage, standalone, no
+  # HA — with one substantial difference: this Vault unseals itself from
+  # Cloud KMS instead of by hand.
+  #
+  # The homelab unseals with 3 of 5 Shamir keys after every restart. That
+  # was fine on an always-on VM. Here the nodes are spot and can be
+  # reclaimed at any hour, and a sealed Vault means every secret in the
+  # cluster is unavailable until a human notices. The trade, stated plainly:
+  # unsealing now depends on GCP IAM rather than on people holding key
+  # shares.
+  #
+  # This is a fresh install, not an adoption. Init is still a one-off manual
+  # step (`vault operator init`), and with a KMS seal it returns RECOVERY
+  # keys plus a root token — recovery keys cannot unseal a running Vault,
+  # they exist to recover or rekey it. Keep them and the root token in a
+  # password manager; they belong in neither Git nor Vault itself.
+
+  injector:
+    enabled: false
+
+  server:
+    # The Workload Identity binding Terraform created names exactly
+    # vault/vault — this namespace and this service account name. The
+    # annotation is the other half of that pair. Miss either and Vault
+    # starts, fails to reach KMS, and stays sealed with a permission error
+    # that does not mention Workload Identity at all.
+    serviceAccount:
+      create: true
+      name: vault
+      annotations:
+        iam.gke.io/gcp-service-account: toolshed-vault@toolshed-testing-508208.iam.gserviceaccount.com
+
+    dataStorage:
+      enabled: true
+      # 10Gi rather than the homelab's 5Gi: that number exists only because
+      # local-path cannot expand a bound volume. standard-rwo can expand, so
+      # this is the chart default, not a constraint.
+      size: 10Gi
+      storageClass: standard-rwo
+
+    ha:
+      enabled: false
+
+    resources:
+      requests:
+        cpu: 100m
+        memory: 128Mi
+      limits:
+        memory: 256Mi
+
+    standalone:
+      enabled: true
+      # `ui = true` here AND ui.enabled below are both required — the chart
+      # has two separate toggles for the same thing, and setting only one
+      # leaves the UI 404ing (claude.md issue #10).
+      #
+      # The seal stanza's values come from terraform output vault_seal, so
+      # Terraform and this file cannot disagree about which key Vault
+      # unseals with. No credentials appear here: the pod authenticates to
+      # KMS as its Workload Identity, so there is no key file to mount,
+      # rotate, or leak.
+      config: |
+        ui = true
+        listener "tcp" {
+          address = "[::]:8200"
+          cluster_address = "[::]:8201"
+          tls_disable = "true"   # lab only - enable TLS for anything beyond local testing
+        }
+        storage "file" {
+          path = "/vault/data"
+        }
+        seal "gcpckms" {
+          project    = "toolshed-testing-508208"
+          region     = "us-central1"
+          key_ring   = "toolshed-vault"
+          crypto_key = "unseal"
+        }
+
+    # Single host — no Tailscale here. Plain HTTP through Contour, matching
+    # tls_disable above.
+    ingress:
+      enabled: true
+      ingressClassName: contour
+      annotations:
+        # Certificate for the real domain, which is now the only one served.
+        # nip.io could never have had one.
+        cert-manager.io/cluster-issuer: letsencrypt-prod
+      hosts:
+        - host: "vault.infra.deployshed.com"
+          paths: []
+      tls:
+        - secretName: vault-tls
+          hosts:
+            - vault.infra.deployshed.com
+
+  ui:
+    enabled: true

helm-overrides/gke-toolshed-prd-usc1/victoria-metrics-single/custom-values.yaml

@@ -0,0 +1,69 @@
+victoria-metrics-single:
+  server:
+    # 3 days rather than the chart's month: this cluster exists to prove a
+    # pipeline, and every extra day is disk against the project's 250GB SSD
+    # quota, which every standard-rwo volume in the cluster shares. Three
+    # days still spans a weekend, which is the window that matters for
+    # working out what happened to something overnight.
+    #
+    # Down from 7d. Note what this does and does not do: it reclaims disk as
+    # old partitions are dropped, and shrinks the index a little, but it
+    # does NOT reduce the memory this needs to run. That tracks active time
+    # series and cache size, neither of which depends on how long data is
+    # kept — which is why the limit above had to be raised rather than this
+    # lowered when it started OOM-killing on restart. The lever for memory
+    # is scrape cardinality: several targets here carry more than forty
+    # labels per series (see the maxLabelsPerTimeseries warnings in its
+    # log), and dropping labels there would cut series count directly.
+    retentionPeriod: "3d"
+
+    persistentVolume:
+      storageClassName: standard-rwo
+      # VictoriaMetrics' compression is why it replaced Prometheus here;
+      # this cluster's metric volume at 7 days fits well inside 3Gi. Unlike
+      # the homelab's local-path, this class can be expanded later.
+      size: 3Gi
+
+    # Raised from 128Mi/512Mi, which this no longer fits inside.
+    #
+    # The pod ran for weeks at the old limit and then OOM-killed in a loop
+    # the first time it was restarted — exit 137 roughly fifty seconds after
+    # a clean start, every time. Nothing had changed about its configuration;
+    # the dataset had simply grown into 3.14 billion rows, and the memory a
+    # restart needs to resume ingestion no longer fit. A long-lived pod can
+    # sit well past the limit it would need to start again, so the failure
+    # only appears the next time something restarts it.
+    #
+    # Memory here tracks active time series rather than disk, which is why
+    # shortening retentionPeriod above would not have helped: the scrape
+    # targets are the same either way, and several of them carry 40+ labels
+    # (see the maxLabelsPerTimeseries warnings in its log).
+    #
+    # Affordable: memory requests across the three nodes sit at 62%, 18% and
+    # 47%, so there is room. CPU is the constrained resource on this cluster,
+    # and this costs none.
+    resources:
+      requests:
+        cpu: 50m
+        memory: 256Mi
+      limits:
+        memory: 1Gi
+
+    # vmui, VictoriaMetrics' built-in query UI, on the same pod and port —
+    # ad-hoc PromQL only, no saved dashboards; Grafana is the real UI. Free
+    # to expose, since it is not a separate component.
+    ingress:
+      enabled: true
+      ingressClassName: contour
+      annotations:
+        # Certificate for the real domain, now the only one served. nip.io
+        # could never have had one.
+        cert-manager.io/cluster-issuer: letsencrypt-prod
+      hosts:
+        - name: vm.infra.deployshed.com
+          path: ["/"]
+          port: http
+      tls:
+        - secretName: vm-tls
+          hosts:
+            - vm.infra.deployshed.com

helm-overrides/gke-toolshed-prd-usc1/vmagent/custom-values.yaml

@@ -0,0 +1,70 @@
+victoria-metrics-agent:
+  # Unchanged from the homelab's, deliberately: everything here is
+  # addressed by cluster-internal Service DNS and namespace, none of which
+  # differs on GKE.
+  #
+  # The write path is what wires the two components together; the chart
+  # names do not imply it on their own.
+  remoteWrite:
+    - url: http://victoria-metrics-single-server.monitoring.svc.cluster.local:8428/api/v1/write
+
+  # config.scrape_configs stays at the chart's default, which already
+  # covers kubelet's cAdvisor endpoint and the prometheus.io/scrape
+  # annotation convention. extraScrapeConfigs is concatenated onto it
+  # rather than replacing it.
+  #
+  # Both jobs below exist because annotation-based discovery did not reach
+  # these targets in the homelab. Contour's Envoy carries no scrape
+  # annotation at all, and node-exporter's annotation sits on its Service,
+  # where the endpointslice discovery path found nothing. Targeting each by
+  # its fixed container port sidesteps both problems and is no less
+  # correct. Worth re-checking on this cluster rather than assuming the
+  # same gaps: if the defaults do find them here, these jobs are harmless
+  # duplicates, not errors.
+  extraScrapeConfigs:
+    - job_name: contour-envoy
+      kubernetes_sd_configs:
+        - role: pod
+          namespaces:
+            names: ["projectcontour"]
+      relabel_configs:
+        # 8002 is the official Contour chart's fixed metrics port for
+        # Envoy. /stats/prometheus is Envoy's own admin endpoint, not
+        # anything Contour-specific.
+        - action: keep
+          source_labels: [__meta_kubernetes_pod_container_port_number]
+          regex: "8002"
+        - target_label: __metrics_path__
+          replacement: /stats/prometheus
+        - action: labelmap
+          regex: __meta_kubernetes_pod_label_(.+)
+        - source_labels: [__meta_kubernetes_pod_name]
+          target_label: pod
+        - source_labels: [__meta_kubernetes_namespace]
+          target_label: namespace
+        - source_labels: [__meta_kubernetes_pod_node_name]
+          target_label: node
+    - job_name: node-exporter
+      kubernetes_sd_configs:
+        - role: pod
+          namespaces:
+            names: ["monitoring"]
+      relabel_configs:
+        - action: keep
+          source_labels: [__meta_kubernetes_pod_container_port_number]
+          regex: "9100"
+        - action: labelmap
+          regex: __meta_kubernetes_pod_label_(.+)
+        - source_labels: [__meta_kubernetes_pod_name]
+          target_label: pod
+        - source_labels: [__meta_kubernetes_namespace]
+          target_label: namespace
+        - source_labels: [__meta_kubernetes_pod_node_name]
+          target_label: node
+
+  resources:
+    requests:
+      cpu: 25m
+      memory: 64Mi
+    limits:
+      memory: 192Mi

helm-templates/argo-cd/Chart.lock

@@ -1,6 +1,6 @@
 dependencies:
 - name: argo-cd
   repository: https://argoproj.github.io/argo-helm
-  version: 7.7.23
-digest: sha256:5d5bb5374bd6a589b5a070a2e48bb1dd211e3871c31af9700297575c84e6f70e
-generated: "2026-08-31T06:40:31.347755+05:30"
+  version: 10.8.4
+digest: sha256:bf31d2cce073a775bf05ec90b74ae45625df5fb6774323a9139d4f8a1be2440a
+generated: "2026-09-12T15:22:56.688354+05:30"

helm-templates/argo-cd/Chart.yaml

@@ -3,5 +3,14 @@ name: argocd
 version: 1.0.0
 dependencies:
   - name: argo-cd
-    version: 7.7.23
+    # Argo CD v3.5.2. Bumped from 7.7.23 (v2.13.8), which was older than
+    # this cluster: it diffs against a Kubernetes schema compiled into its
+    # own binary, and GKE 1.35 has fields that schema predates
+    # (status.terminatingReplicas, beta since 1.33), which aborted the diff
+    # on every server-side-apply app. ServerSideDiff works around that; this
+    # removes the cause.
+    #
+    # Not a drop-in: see the 2.x -> 3.0 notes in this cluster's
+    # argocd-admin-prd/custom-values.yaml for what changed in behaviour.
+    version: 10.8.4
     repository: https://argoproj.github.io/argo-helm