GKE: register the remaining tools, add the registry CA issuer
Adds nine Applications — external-secrets, harbor, jenkins, postgresql, redis, victoria-metrics-single, vmagent, node-exporter, grafana — bringing the cluster to the homelab's full set. Every nameOverride is pinned, which is load-bearing rather than cosmetic in at least three cases: external-secrets (its service account name is what vault-backend's serviceAccountRef and Vault's auth role bind to), victoria-metrics-single (its Service renders as <release>-server, the address vmagent and Grafana both hardcode) and harbor (the pipeline refers to its object names). extra-manifests gains the ClusterIssuer that makes cert-manager useful here: a CA issuer over the registry-ca Secret, which is created by hand from terraform output because the CA's private key exists only in Terraform state and nothing can reconcile it from git. Two homelab-only things removed rather than left to confuse: the Tailscale Harbor Ingress, and the homelab cluster's own entry point and values file, which have no meaning in a GCP-only repo. app-of-secretstores and app-of-extra-manifests now point at this cluster's Gitea over cluster DNS, and harbor-robot-dockerconfig is keyed to this cluster's registry hostname — docker matches stored credentials by exact host, so the homelab's would have silently pushed unauthenticated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LEsTefWWifp4ikvhHF5s6N
This commit is contained in:
Mukul Sharma
co-authored by
Claude Opus 5
Claude Opus 5
parent
a3429adf2c
commit
9c67afd5e0
@@ -16,7 +16,7 @@ metadata:
|
||||
spec:
|
||||
project: default
|
||||
source:
|
||||
repoURL: 'http://gitea.192.168.1.7.nip.io/mukul/devops-infra-argo-config.git'
|
||||
repoURL: 'http://gitea-http.gitea.svc.cluster.local:3000/gitadmin/devops-infra-argo-config-gcp.git'
|
||||
targetRevision: main
|
||||
path: extra-manifests
|
||||
directory:
|
||||
|
||||
@@ -18,7 +18,7 @@ metadata:
|
||||
spec:
|
||||
project: default
|
||||
source:
|
||||
repoURL: 'http://gitea.192.168.1.7.nip.io/mukul/devops-infra-argo-config.git'
|
||||
repoURL: 'http://gitea-http.gitea.svc.cluster.local:3000/gitadmin/devops-infra-argo-config-gcp.git'
|
||||
targetRevision: main
|
||||
path: secretstores
|
||||
directory:
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
# Harbor's chart has no multi-host mechanism at all (checked against the
|
||||
# actual template — no extraHosts like argo-cd, no secondaryingress like
|
||||
# jenkins), so this is a standalone second Ingress for the Tailscale
|
||||
# hostname. Correction from an earlier version of this file: it routed
|
||||
# everything to harbor-core alone. That's wrong — harbor-core is the
|
||||
# API/auth backend; the actual web UI (the portal you log into) is served
|
||||
# by harbor-portal. Mirrors the chart's own generated Ingress path split
|
||||
# exactly (confirmed against its real rendered spec, not assumed this
|
||||
# time): /api/, /service/, /v2/, /c/ go to harbor-core, everything else
|
||||
# goes to harbor-portal.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: harbor-ingress-tailscale
|
||||
namespace: harbor
|
||||
spec:
|
||||
ingressClassName: contour
|
||||
rules:
|
||||
- host: "harbor.100.90.248.118.nip.io"
|
||||
http:
|
||||
paths:
|
||||
- path: /api/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /service/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /v2/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /c/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-core
|
||||
port:
|
||||
number: 80
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: harbor-portal
|
||||
port:
|
||||
number: 80
|
||||
@@ -0,0 +1,31 @@
|
||||
# The issuer that signs Harbor's certificate, from the private CA the GKE
|
||||
# node pool was told to trust.
|
||||
#
|
||||
# Terraform generates that CA and puts its PUBLIC key in Secret Manager,
|
||||
# which is what every node fetches to verify image pulls. cert-manager needs
|
||||
# both halves in order to sign, so the key pair is created as the
|
||||
# `registry-ca` Secret in the cert-manager namespace by hand, from
|
||||
# `terraform output` — the private key exists nowhere else, so nothing can
|
||||
# reconcile it from git:
|
||||
#
|
||||
# TF=~/Documents/localgit/toolshed-gke-infra/envs/prod/10-infra
|
||||
# kubectl -n cert-manager create secret tls registry-ca \
|
||||
# --cert=<(terraform -chdir=$TF output -raw registry_ca_cert_pem) \
|
||||
# --key=<(terraform -chdir=$TF output -raw registry_ca_key_pem)
|
||||
#
|
||||
# A ClusterIssuer, not an Issuer, so Harbor's namespace can request a
|
||||
# certificate without the CA's private key being copied there. Note that a
|
||||
# ClusterIssuer always reads its secret from cert-manager's own namespace,
|
||||
# which is what makes that possible.
|
||||
#
|
||||
# This is the whole reason cert-manager is installed at all: nothing else in
|
||||
# this cluster uses TLS. Let's Encrypt cannot help here, because nip.io is
|
||||
# not on the public suffix list and every *.nip.io certificate shares one
|
||||
# rate limit.
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: ClusterIssuer
|
||||
metadata:
|
||||
name: registry-ca-issuer
|
||||
spec:
|
||||
ca:
|
||||
secretName: registry-ca
|
||||
@@ -1,19 +0,0 @@
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: incubator-infra-k8s-admin-prd-ase1
|
||||
namespace: argocd
|
||||
finalizers:
|
||||
- resources-finalizer.argocd.argoproj.io
|
||||
spec:
|
||||
project: default
|
||||
source:
|
||||
repoURL: 'http://gitea.192.168.1.7.nip.io/mukul/devops-infra-argo-config.git'
|
||||
targetRevision: main
|
||||
path: generic-argo-apps-chart
|
||||
helm:
|
||||
valueFiles:
|
||||
- ../values/incubator-infra-k8s-admin-prd-ase1-values.yaml
|
||||
destination:
|
||||
name: in-cluster
|
||||
namespace: argocd
|
||||
@@ -28,13 +28,14 @@ spec:
|
||||
type: kubernetes.io/dockerconfigjson
|
||||
data:
|
||||
# Keyed by exact registry hostname — docker matches credentials
|
||||
# against the host portion of the image reference. Was
|
||||
# harbor-core.harbor.svc.cluster.local; switched to the Contour
|
||||
# ingress hostname alongside buildDocker.groovy and dind-pod.yaml
|
||||
# so push/pull share one consistent, resolvable-from-anywhere
|
||||
# reference.
|
||||
# against the host portion of the image reference, so this must be
|
||||
# the same hostname the pipeline pushes to, the node pool was told
|
||||
# to trust, and Harbor's own externalURL uses. On GKE that is the
|
||||
# Contour ingress host, never harbor-core.harbor.svc.cluster.local:
|
||||
# cluster DNS resolves from a pod but not from the node's containerd
|
||||
# doing the actual image pull.
|
||||
.dockerconfigjson: |
|
||||
{"auths":{"harbor.192.168.1.7.nip.io":{"username":"{{ .username }}","password":"{{ .password }}","auth":"{{ printf "%s:%s" .username .password | b64enc }}"}}}
|
||||
{"auths":{"harbor.35.238.248.203.nip.io":{"username":"{{ .username }}","password":"{{ .password }}","auth":"{{ printf "%s:%s" .username .password | b64enc }}"}}}
|
||||
data:
|
||||
- secretKey: username
|
||||
remoteRef:
|
||||
|
||||
@@ -70,3 +70,73 @@ appSpec:
|
||||
# cert-manager's CRDs embed large OpenAPI schemas; SSA avoids the
|
||||
# 256KiB last-applied-configuration limit, same as external-secrets.
|
||||
serverSideApply: true
|
||||
- name: external-secrets
|
||||
# Every credential in the cluster arrives through this. nameOverride is
|
||||
# not cosmetic here: without it the release becomes
|
||||
# "external-secrets-<cluster>-<env>", which renames the controller's
|
||||
# service account — and both secretstores/vault-backend.yaml's
|
||||
# serviceAccountRef and Vault's own Kubernetes auth role are bound to
|
||||
# the plain name.
|
||||
nameOverride: external-secrets
|
||||
namespace: external-secrets
|
||||
chartDir: external-secrets
|
||||
valuesDir: external-secrets
|
||||
# ClusterSecretStore's CRD embeds a large provider schema and exceeds
|
||||
# the 256KiB annotation limit on a client-side apply.
|
||||
serverSideApply: true
|
||||
- name: harbor
|
||||
# Object names are all prefixed with the release name, and the pipeline
|
||||
# refers to them, so it is pinned.
|
||||
nameOverride: harbor
|
||||
namespace: harbor
|
||||
chartDir: harbor
|
||||
valuesDir: harbor
|
||||
- name: jenkins
|
||||
nameOverride: jenkins
|
||||
namespace: jenkins
|
||||
chartDir: jenkins
|
||||
valuesDir: jenkins
|
||||
- name: postgresql
|
||||
# Backs toolshed's control plane. Own namespace so it is addressed over
|
||||
# cluster DNS like any other platform component and outlives whatever
|
||||
# consumes it: postgresql.postgres.svc.cluster.local:5432
|
||||
nameOverride: postgresql
|
||||
namespace: postgres
|
||||
chartDir: postgresql
|
||||
valuesDir: postgresql
|
||||
- name: redis
|
||||
# Backs toolshed's managed cache add-on, same namespace reasoning as
|
||||
# postgresql: redis.redis.svc.cluster.local:6379
|
||||
nameOverride: redis
|
||||
namespace: redis
|
||||
chartDir: redis
|
||||
valuesDir: redis
|
||||
- name: victoria-metrics-single
|
||||
# Pinned name, because the chart renders its Service as
|
||||
# <release>-server and both vmagent's remote write and Grafana's
|
||||
# datasource address it at
|
||||
# victoria-metrics-single-server.monitoring.svc.cluster.local:8428
|
||||
nameOverride: victoria-metrics-single
|
||||
namespace: monitoring
|
||||
chartDir: victoria-metrics-single
|
||||
valuesDir: victoria-metrics-single
|
||||
- name: vmagent
|
||||
# The scraper. Needs the Service above to exist, so bring it up after.
|
||||
nameOverride: vmagent
|
||||
namespace: monitoring
|
||||
chartDir: vmagent
|
||||
valuesDir: vmagent
|
||||
- name: node-exporter
|
||||
nameOverride: node-exporter
|
||||
namespace: monitoring
|
||||
chartDir: node-exporter
|
||||
valuesDir: node-exporter
|
||||
- name: grafana
|
||||
# Requires secretstores/grafana-admin-credentials.yaml to have synced
|
||||
# first: without that Secret the pod generates a random admin password
|
||||
# nobody has. It does not crash, it just cannot be logged into until
|
||||
# the Secret exists and the pod restarts.
|
||||
nameOverride: grafana
|
||||
namespace: monitoring
|
||||
chartDir: grafana
|
||||
valuesDir: grafana
|
||||
|
||||
@@ -1,217 +0,0 @@
|
||||
clusterSpec:
|
||||
# "k8s-admin-prd-ase1" only resolved in the fleet because that name was
|
||||
# registered as an external cluster in the hub ArgoCD's cluster list.
|
||||
# There's no hub here — one ArgoCD, running on the cluster it manages —
|
||||
# so this has to be the built-in local-cluster alias instead.
|
||||
destination:
|
||||
server: ""
|
||||
name: "in-cluster"
|
||||
|
||||
argocdSpec:
|
||||
# Was argocd-admin (a separate hub namespace in the fleet's two-tier
|
||||
# setup). Single ArgoCD instance here, so Application objects live in
|
||||
# the same namespace as ArgoCD itself — see claude.md.
|
||||
namespace: argocd
|
||||
|
||||
teamSpec:
|
||||
devops:
|
||||
source:
|
||||
repoURL: http://gitea.192.168.1.7.nip.io/mukul/devops-infra-helm-charts.git
|
||||
targetRevision: main
|
||||
path: helm-templates
|
||||
valueFiles: ../../helm-overrides/k8s-admin-prd-ase1
|
||||
labels:
|
||||
bu: infra
|
||||
team: devops
|
||||
env: prd
|
||||
cluster: k8s-admin-prd-ase1
|
||||
|
||||
appSpec:
|
||||
- name: argocd
|
||||
nameOverride: argocd-admin-prd
|
||||
namespace: argocd
|
||||
chartDir: argo-cd
|
||||
valuesDir: argocd-admin-prd
|
||||
- name: gitea
|
||||
# Adopting the already-running standalone install (helm release
|
||||
# "gitea" in namespace "gitea", from deploy_gitea.sh) rather than
|
||||
# deploying a second one — nameOverride pins the rendered
|
||||
# Application's name (and therefore the Helm release name Argo
|
||||
# renders with) to match those existing object names exactly.
|
||||
nameOverride: gitea
|
||||
namespace: gitea
|
||||
chartDir: gitea
|
||||
valuesDir: gitea
|
||||
# NOT using the Application-wide `replace: true` here anymore — it
|
||||
# forces a full PUT of every resource this Application renders, and a
|
||||
# bound PVC's spec is immutable (volumeName/storageClassName get
|
||||
# filled in by the provisioner after binding; a PUT that omits them
|
||||
# looks like clearing them, which the API correctly refuses). The
|
||||
# Deployment-only fix now lives as a per-resource sync-option
|
||||
# annotation in helm-overrides/k8s-admin-prd-ase1/gitea/custom-values.yaml
|
||||
# (deployment.annotations), which only Replaces the Deployment.
|
||||
- name: vault
|
||||
# Adopting the running production-mode Vault (helm release "vault" in
|
||||
# namespace "vault", chart 0.34.1 — see helm-templates/vault/Chart.yaml).
|
||||
# It's already initialized and unsealed; this Application only manages
|
||||
# Vault's Deployment/config, never its data or seal state. Review the
|
||||
# first diff carefully before syncing — this is the highest-consequence
|
||||
# adoption in this repo so far.
|
||||
nameOverride: vault
|
||||
namespace: vault
|
||||
chartDir: vault
|
||||
valuesDir: vault
|
||||
- name: contour
|
||||
# Adopting the running ingress (helm release "contour" in namespace
|
||||
# "projectcontour", chart 0.7.0 — the OFFICIAL projectcontour chart,
|
||||
# not the Bitnami one that used to be wired up as helm-templates/contour
|
||||
# — see the note in that Chart.yaml and claude.md issue #4). This is
|
||||
# the ingress path for every other Application in this repo — review
|
||||
# the diff before syncing, same caution as vault.
|
||||
nameOverride: contour
|
||||
namespace: projectcontour
|
||||
chartDir: contour
|
||||
valuesDir: contour
|
||||
- name: external-secrets
|
||||
# Correction from an earlier version of this file: "no nameOverride
|
||||
# needed" was wrong. Without one, the Application (and therefore the
|
||||
# Helm release name the chart templates with) becomes
|
||||
# "external-secrets-admin-prd" — so the controller's ServiceAccount
|
||||
# actually ends up named external-secrets-admin-prd, not
|
||||
# external-secrets. secretstores/vault-backend.yaml's
|
||||
# serviceAccountRef assumes the plain name, and Vault's role was bound
|
||||
# to bound_service_account_names=external-secrets — both need this
|
||||
# pinned name to match.
|
||||
nameOverride: external-secrets
|
||||
namespace: external-secrets
|
||||
chartDir: external-secrets
|
||||
valuesDir: external-secrets
|
||||
# ClusterSecretStore's CRD (large embedded OpenAPI schema) exceeds the
|
||||
# 256KiB last-applied-configuration annotation limit on a normal
|
||||
# client-side apply. SSA sidesteps it entirely — see the note in
|
||||
# generic-argo-apps-chart's template.
|
||||
serverSideApply: true
|
||||
- name: jenkins
|
||||
# Fresh install, but pinning nameOverride anyway — learned from
|
||||
# external-secrets that skipping it produces
|
||||
# "jenkins-admin-prd"-suffixed resource names, which
|
||||
# jenkins-admin-credentials (the ExternalSecret, namespace "jenkins")
|
||||
# doesn't need to care about, but keeps naming predictable and
|
||||
# consistent with every other app here regardless.
|
||||
nameOverride: jenkins
|
||||
namespace: jenkins
|
||||
chartDir: jenkins
|
||||
valuesDir: jenkins
|
||||
- name: harbor
|
||||
# Fresh install (helm list -n harbor came back empty despite claude.md
|
||||
# saying otherwise). nameOverride pinned for the same predictability
|
||||
# reason as jenkins — rendered object names all end up prefixed with
|
||||
# this (harbor-core, harbor-registry, etc.), which is also what
|
||||
# Jenkins needs to reference for internal image pushes
|
||||
# (harbor-core.harbor.svc.cluster.local).
|
||||
nameOverride: harbor
|
||||
namespace: harbor
|
||||
chartDir: harbor
|
||||
valuesDir: harbor
|
||||
- name: postgresql
|
||||
# Backs toolshed's control plane. Own namespace rather than living
|
||||
# inside toolshed, so it is addressed over cluster DNS like any other
|
||||
# platform component and outlives whatever consumes it:
|
||||
# postgresql.postgres.svc.cluster.local:5432
|
||||
#
|
||||
# nameOverride pinned for the same reason as everything else here —
|
||||
# without it the rendered Application (and therefore the Helm release
|
||||
# name, and therefore every object name) becomes
|
||||
# "postgresql-admin-prd-prd".
|
||||
#
|
||||
# Hand-written chart, not Bitnami's: that registry has been actively
|
||||
# unstable (infra issue #4) and PostgreSQL ships no official chart.
|
||||
# Requires secretstores/toolshed-postgres-credentials.yaml to have
|
||||
# synced first — the pod cannot start without the Secret.
|
||||
nameOverride: postgresql
|
||||
namespace: postgres
|
||||
chartDir: postgresql
|
||||
valuesDir: postgresql
|
||||
- name: redis
|
||||
# Backs toolshed's managed cache add-on — toolshed provisions a per-app
|
||||
# ACL user, scoped to its own key prefix, on request. Own namespace for
|
||||
# the same reason postgresql has one: addressed over cluster DNS like
|
||||
# any other platform component, outliving whatever consumes it:
|
||||
# redis.redis.svc.cluster.local:6379
|
||||
#
|
||||
# Hand-written chart, not Bitnami's, for the same reason as postgresql
|
||||
# (infra issue #4) — Redis ships no official chart either.
|
||||
#
|
||||
# Authentication is defined by an ACL file with no requirepass, which
|
||||
# is a security property rather than a preference: see the chart's own
|
||||
# values.yaml, where getting it wrong leaves the server open to
|
||||
# unauthenticated access after its first restart.
|
||||
#
|
||||
# Requires secretstores/toolshed-redis-credentials.yaml to have synced
|
||||
# first — the init container cannot seed the ACL file without it.
|
||||
nameOverride: redis
|
||||
namespace: redis
|
||||
chartDir: redis
|
||||
valuesDir: redis
|
||||
- name: victoria-metrics-single
|
||||
# Replaces the Prometheus server this entry briefly was (see git
|
||||
# history on this file) — same job, lower RAM/disk footprint for the
|
||||
# same metric volume, and it speaks Prometheus's own query API
|
||||
# (/api/v1/query) so nothing downstream (toolshed's metrics
|
||||
# connection, docs/PRODUCT-ARCHITECTURE.md step 5) needed to change,
|
||||
# only the URL it points at.
|
||||
#
|
||||
# Vendored official chart (victoriametrics/helm-charts), same
|
||||
# vendor-the-official-chart pattern as Contour/ArgoCD/Vault/Gitea/
|
||||
# Harbor/Jenkins. This exact directory name already existed in this
|
||||
# repo before — a leftover GKE-targeted vendored copy from the
|
||||
# original Meesho monorepo import — and was removed rather than
|
||||
# adapted; see that chart's own Chart.yaml comment.
|
||||
#
|
||||
# nameOverride pinned to exactly "victoria-metrics-single" for the
|
||||
# same reason as postgresql/gitea/prometheus above: the chart's
|
||||
# server Service renders as "<release-name>-server", so this is what
|
||||
# makes it resolvable at a predictable hostname
|
||||
# (victoria-metrics-single-server.monitoring.svc.cluster.local:8428)
|
||||
# rather than "victoria-metrics-single-admin-prd-prd-server".
|
||||
nameOverride: victoria-metrics-single
|
||||
namespace: monitoring
|
||||
chartDir: victoria-metrics-single
|
||||
valuesDir: victoria-metrics-single
|
||||
- name: vmagent
|
||||
# The scraper — pulls from the same targets the Prometheus server
|
||||
# used to scrape directly (kubelet's cAdvisor endpoint, and anything
|
||||
# carrying a prometheus.io/scrape annotation, e.g. node-exporter
|
||||
# below) and remote_writes into victoria-metrics-single. Needs that
|
||||
# component's Service name, so bring it up after, not before.
|
||||
nameOverride: vmagent
|
||||
namespace: monitoring
|
||||
chartDir: vmagent
|
||||
valuesDir: vmagent
|
||||
- name: node-exporter
|
||||
# Host-level metrics (disk/memory/load) — independent of which TSDB
|
||||
# stores them, so vendored standalone rather than as a subchart of
|
||||
# anything. Was a subchart of the (now removed) Prometheus server
|
||||
# entry; moved out to its own release when that server was replaced,
|
||||
# since victoria-metrics-single has no equivalent bundled subchart.
|
||||
nameOverride: node-exporter
|
||||
namespace: monitoring
|
||||
chartDir: node-exporter
|
||||
valuesDir: node-exporter
|
||||
- name: grafana
|
||||
# Dashboards over VictoriaMetrics — see that chart for why "type:
|
||||
# prometheus" is correct for a VictoriaMetrics URL. This directory
|
||||
# already held a fully-vendored old Grafana chart (v6.58.7) from the
|
||||
# original Meesho monorepo import with generic production config
|
||||
# (fullnameOverride: grafana-infra-prd) — removed and re-vendored
|
||||
# fresh as a thin wrapper, same treatment as victoria-metrics-single.
|
||||
#
|
||||
# Requires secretstores/grafana-admin-credentials.yaml to have synced
|
||||
# first — the pod falls back to a randomly-generated admin password
|
||||
# nobody has if that Secret does not exist yet when it boots (not a
|
||||
# crash, just an inaccessible login until the Secret exists and the
|
||||
# pod restarts).
|
||||
nameOverride: grafana
|
||||
namespace: monitoring
|
||||
chartDir: grafana
|
||||
valuesDir: grafana
|
||||
Reference in New Issue
Block a user