Mukul SharmaandClaude Opus 5
bf814854fd
toolshed can now run a one-off command against an app — a migration, a seed, a backfill — in a throwaway Job built from the app's own image with its own configuration (internal/deploy.Client.RunCommand). It exists because a user with a provisioned database otherwise has no way to reach it: the operator can exec into the Postgres pod, a user cannot, and the premise is that nobody needs kubectl. Without this the deployer gets "forbidden" the moment anyone runs one — the two-repositories drift internal/deploy/kubernetes.go's own package doc warns about, and the third feature in a row to need both copies changed together. Delete is included because the Job is removed once its output has been captured into the run record; without it they would accumulate one per command forever. Notably absent is pods/exec, which an exec-into-the- running-pod design would have required — a far more dangerous grant on the one credential here whose compromise means the cluster, and one reason the Job approach was chosen over it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wajog7nELA3i8JWTjxYGHF
152 lines
6.6 KiB
YAML
152 lines
6.6 KiB
YAML
# Cluster write access for toolshed's deployer, and only deployer.
|
|
#
|
|
# Lives here rather than in toolshed's own Helm chart because these are
|
|
# cluster-scoped, and toolshed's Application runs in the `webapp` project,
|
|
# whose clusterResourceWhitelist deliberately allows only Namespace. Widening
|
|
# that project to permit ClusterRole and ClusterRoleBinding would let *any*
|
|
# Application in it — every demo app — create cluster-wide RBAC, which is a
|
|
# privilege-escalation surface in the project that holds ordinary apps. The
|
|
# narrow fix is to put the two cluster-scoped objects where platform-level
|
|
# cluster resources already live.
|
|
#
|
|
# The ServiceAccount they bind to is namespaced, so it stays in toolshed's
|
|
# chart alongside the Deployment that uses it. If that ever moves, this
|
|
# binding's subject has to move with it.
|
|
#
|
|
# Scoped to the resource kinds toolshed creates for an app. Not cluster-admin
|
|
# and not a wildcard: this is the one credential in the system whose
|
|
# compromise means the cluster, so what it can do should fit on one screen.
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: ClusterRole
|
|
metadata:
|
|
name: toolshed-deployer
|
|
labels:
|
|
app.kubernetes.io/part-of: toolshed
|
|
rules:
|
|
# Namespaces are cluster-scoped, and one is created per app.
|
|
- apiGroups: [""]
|
|
resources: ["namespaces"]
|
|
verbs: ["get", "list", "watch", "create", "delete"]
|
|
# Everything an app needs inside its own namespace.
|
|
- apiGroups: [""]
|
|
resources: ["services", "resourcequotas"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
# An app's configuration, delivered as a Secret so values never appear in
|
|
# the pod spec.
|
|
#
|
|
# Deliberately without list or watch. Kubernetes RBAC cannot scope a
|
|
# ClusterRole to a namespace pattern, so this necessarily covers every
|
|
# namespace — but without list, deployer cannot enumerate the cluster's
|
|
# secrets, only address ones by a name it already knows. That narrows the
|
|
# blast radius without removing it: get on a known name still reaches any
|
|
# secret in the cluster.
|
|
#
|
|
# The proper fix, when this has tenants who are not the operator, is a
|
|
# RoleBinding created per app namespace instead of one ClusterRole. That
|
|
# needs deployer to hold permission to create RoleBindings, which is its
|
|
# own escalation path and wants thinking about rather than adding here.
|
|
- apiGroups: [""]
|
|
resources: ["secrets"]
|
|
verbs: ["get", "create", "update", "patch", "delete"]
|
|
# Read-only. Pods are listed to report why a rollout failed, never changed,
|
|
# and their output is read so an app's own logs can be shown in the
|
|
# dashboard without anyone reaching for kubectl.
|
|
- apiGroups: [""]
|
|
resources: ["pods", "pods/log"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["apps"]
|
|
resources: ["deployments"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
# A one-off command run against an app executes in a throwaway Job built
|
|
# from the app's own image — see internal/deploy.Client.RunCommand. Delete
|
|
# is needed because the Job is removed once its output has been captured;
|
|
# without it they accumulate one per command forever.
|
|
- apiGroups: ["batch"]
|
|
resources: ["jobs"]
|
|
verbs: ["get", "list", "watch", "create", "delete"]
|
|
# An app with persistent storage is a StatefulSet instead of a Deployment
|
|
# (internal/deploy.Client.ensureStatefulSet). Delete is needed on both:
|
|
# switching an app between stateless and stateful must remove whichever
|
|
# controller it no longer is, or the two fight over the same pods.
|
|
- apiGroups: ["apps"]
|
|
resources: ["statefulsets"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
# Read-only. The claims themselves are created by the StatefulSet's own
|
|
# volumeClaimTemplates, never directly here — this is only to observe
|
|
# them, and deleting one would destroy an app's data.
|
|
- apiGroups: [""]
|
|
resources: ["persistentvolumeclaims"]
|
|
verbs: ["get", "list", "watch"]
|
|
# The policy that stops one app reaching another.
|
|
- apiGroups: ["networking.k8s.io"]
|
|
resources: ["networkpolicies"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
# Created only for an app with autoscaling enabled (max replicas set
|
|
# above min); removed again if it's turned back off. See toolshed's own
|
|
# internal/deploy.Client.ensureAutoscaler. Added alongside that feature —
|
|
# keep this file and toolshed's deploy/helm/toolshed/templates/rbac.yaml
|
|
# in sync, per internal/deploy/kubernetes.go's own package doc warning
|
|
# that the two are unsynchronized copies in two repositories.
|
|
- apiGroups: ["autoscaling"]
|
|
resources: ["horizontalpodautoscalers"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: ClusterRoleBinding
|
|
metadata:
|
|
name: toolshed-deployer
|
|
labels:
|
|
app.kubernetes.io/part-of: toolshed
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: toolshed-deployer
|
|
namespace: toolshed
|
|
roleRef:
|
|
apiGroup: rbac.authorization.k8s.io
|
|
kind: ClusterRole
|
|
name: toolshed-deployer
|
|
---
|
|
# A custom domain's Ingress and TLS secret live in the gateway's own
|
|
# namespace (toolshed, same as above) — never an app's namespace. Scoped
|
|
# with a namespaced Role/RoleBinding rather than widening the ClusterRole
|
|
# above: Ingress management here only ever targets this one fixed
|
|
# namespace, unlike the per-app namespaces the ClusterRole necessarily
|
|
# spans. Added alongside toolshed's custom-domains feature — see
|
|
# internal/deploy.Client.EnsureDomainIngress/DomainCertReady/
|
|
# RemoveDomainIngress and this file's own header comment about keeping it
|
|
# and deploy/helm/toolshed/templates/rbac.yaml in sync.
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: Role
|
|
metadata:
|
|
name: toolshed-deployer-ingress
|
|
namespace: toolshed
|
|
labels:
|
|
app.kubernetes.io/part-of: toolshed
|
|
rules:
|
|
- apiGroups: ["networking.k8s.io"]
|
|
resources: ["ingresses"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
# Reads the TLS secret cert-manager's ingress-shim populates for a domain
|
|
# Ingress, and deletes it (and the Ingress above) when a domain is
|
|
# removed. Never create/update — cert-manager, not deployer, writes this
|
|
# secret.
|
|
- apiGroups: [""]
|
|
resources: ["secrets"]
|
|
verbs: ["get", "list", "watch", "delete"]
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: RoleBinding
|
|
metadata:
|
|
name: toolshed-deployer-ingress
|
|
namespace: toolshed
|
|
labels:
|
|
app.kubernetes.io/part-of: toolshed
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: toolshed-deployer
|
|
namespace: toolshed
|
|
roleRef:
|
|
apiGroup: rbac.authorization.k8s.io
|
|
kind: Role
|
|
name: toolshed-deployer-ingress
|