devops-infra-helm-charts-gcp/helm-templates/argo-cd/README.md

argo-cd — bootstrap runbook

Thin wrapper chart (see Chart.yaml — dependency on argo/argo-cd). This README covers a full reset and clean bootstrap of ArgoCD itself on the homelab VM (cluster k8s-admin-prd-ase1, namespace argocd), end to end. ArgoCD is the one thing in this repo that can't bootstrap itself via GitOps — it has to exist before it can manage anything, including itself — so every step here is a manual, imperative command run once.

Homelab only. Every k8s-admin-prd-ase1 path below lives in the homelab devops-infra-helm-charts / devops-infra-argo-config repos, not in this GCP copy. On GKE the same steps apply with helm-overrides/gke-toolshed-prd-usc1/argocd-admin-prd/custom-values.yaml and devops-infra-argo-config-gcp's values/incubator-infra-gke-toolshed-prd-usc1-values.yaml. toolshed-gke-infra/envs/prod/20-bootstrap is superseded and not applied.

0. Full uninstall (only if resetting)

helm uninstall argocd-admin-prd -n argocd
kubectl delete namespace argocd

helm uninstall removes everything the release owns, including the Application/AppProject/ApplicationSet CRDs (this chart installs them as plain templates, not as a separate hook that survives uninstall). Deleting the namespace on top cleans up anything left behind (PVCs for the bundled Redis, stray ConfigMaps/Secrets created outside the Helm release, etc.). Confirm it's actually clean before moving on:

kubectl get all -n argocd            # should error "not found" once the ns is gone
kubectl get crd | grep argoproj.io   # should return nothing

1. Pull the real chart

This wrapper's Chart.yaml only declares a dependency — the actual argo-cd chart isn't vendored, so pull it first:

helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm dependency update helm-templates/argo-cd   # run from the repo root

2. Install

kubectl create namespace argocd --dry-run=client -o yaml | kubectl apply -f -

helm upgrade --install argocd-admin-prd helm-templates/argo-cd \
  --namespace argocd \
  -f helm-overrides/k8s-admin-prd-ase1/argocd-admin-prd/custom-values.yaml

kubectl -n argocd rollout status deploy -l app.kubernetes.io/component=server --timeout=180s

argocd-admin-prd is the release name — it has to match the nameOverride in devops-infra-argo-config's values/incubator-infra-k8s-admin-prd-ase1-values.yaml appSpec entry for argocd, since once GitOps sync takes over, Argo renders using that same name.

3. Get in

kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath='{.data.password}' | base64 -d; echo

Username is always admin. This secret only gets created by a first-install Helm hook — it won't reappear on a later helm upgrade against an existing release, only right after step 2 on a genuinely fresh install (which is why step 0's full uninstall matters if you're trying to get back to a known state rather than just patching forward).

If it's missing later and you're locked out:

kubectl -n argocd exec -it deploy/argocd-server -- argocd admin initial-password -n argocd
# or, as a last resort, set it directly:
kubectl -n argocd exec -it deploy/argocd-server -- argocd account bcrypt --password '<new-password>'
kubectl -n argocd patch secret argocd-secret \
  -p '{"stringData": {"admin.password": "<hash-from-above>", "admin.passwordMtime": "'$(date +%FT%T%Z)'"}}'

Reach the UI at argocd.192.168.1.7.nip.io (Contour ingress, plain HTTP — --insecure is set in custom-values.yaml on purpose, see that file).

4. Give it repos to sync from

Push devops-infra-argo-config and devops-infra-helm-charts to Gitea. If either repo isn't anonymous-read, ArgoCD needs a credential for both — the incubator Application sources devops-infra-argo-config directly, and every child Application's $values ref sources devops-infra-helm-charts. See devops-infra-argo-config/gitea-repo-credentials.example.yaml for the Secret template (copy it, fill in a real Gitea access token, kubectl apply, never commit the filled-in version).

5. Hand control to GitOps

From devops-infra-argo-config:

kubectl apply -f app-of-projects.yaml                              # creates the "devops" AppProject
kubectl apply -f incubator/incubator-infra-k8s-admin-prd-ase1.yaml  # renders argocd + gitea Applications

Check it landed:

kubectl -n argocd get applications

Expect to see app-of-projects, incubator-infra-k8s-admin-prd-ase1, argocd-admin-prd, and gitea. Everything in this repo is manual-sync only on purpose (no app has an automated: syncPolicy) — click Sync in the UI (or argocd app sync <name> via CLI) rather than expecting anything to reconcile on its own. argocd-admin-prd going green means ArgoCD is now managing its own Helm release via Git instead of the imperative commands above — from here on, values changes go through helm-overrides/k8s-admin-prd-ase1/argocd-admin-prd/custom-values.yaml, not a manual helm upgrade.

Gotchas hit getting here (don't relitigate these)

• spec.project: devops on child Applications requires the devops AppProject to exist first — that's what step 5's app-of-projects.yaml apply is for. Skipping it means gitea's (and any future app's) Application object exists in Git/gets rendered but fails to validate.
• Deleting a ConfigMap this chart owns doesn't self-heal — same manual-sync-only reasoning as above. Re-running step 2's helm upgrade --install recreates anything missing from the release; it just won't regenerate argocd-initial-admin-secret (see step 3).
• Gitea-specific adoption issues (the value→valueFrom env var migration, the single-node RollingUpdate LevelDB lock collision) are documented in helm-overrides/k8s-admin-prd-ase1/gitea/custom-values.yaml instead — not an ArgoCD problem, don't duplicate the notes here.