Add a Redis chart for toolshed's managed cache add-on
toolshed provisions a per-app Redis ACL user, scoped to its own key prefix, on request (internal/dbprovision.EnsureRedisUser). Nothing in this cluster ran Redis — Harbor's internal one is Harbor's and is not ACL configured — so there was nowhere for that to point. Hand-written rather than vendoring Bitnami's, same reasoning as the sibling postgresql chart: Broadcom has been retiring and freezing images behind that repo (infra issue #4, where it broke Contour twice), and Redis publishes no official chart either. The authentication design is the part worth reading before changing anything. Redis is started with an ACL file and NO requirepass, and that distinction is a security property rather than a style choice: - toolshed persists provisioned users with ACL SAVE, which requires an aclfile. Without it every provisioned user is lost on the next restart. - But ACL SAVE also serialises the default user. With requirepass, the saved entry comes back as `user default on nopass ~* &* +@all`, and after the next restart the ACL file wins — leaving Redis open to UNAUTHENTICATED access with full permissions. Verified directly: with requirepass, the restarted server answered an unauthenticated PING with PONG and served a key. So the default user is defined in the ACL file instead, seeded once by an init container that deliberately never overwrites an existing file — overwriting would delete every user toolshed had provisioned into it, reintroducing the same lockout from the other end. The documented consequence is that rotating the admin password in Vault does not propagate on its own; that needs ACL SETUSER default + ACL SAVE against the running server. Sized for a node at its ceiling: 32Mi requested, 96Mi limit, maxmemory 48mb. The limit sits above maxmemory on purpose, so Redis reaches its own eviction policy rather than being OOM-killed, which would lose the whole instance instead of the coldest keys. Snapshotting is off — what must survive a restart is the ACL file, which ACL SAVE writes independently of RDB, and cached values are by definition reconstructible. allkeys-lru because this backs a connection kind called "cache" and eviction under pressure is that contract; values.yaml says plainly that an app using Redis as its only copy of something wants noeviction instead. Verified: helm template, then a real deploy to a k3d cluster — provisioned users through toolshed's own code, deleted the pod, and confirmed all five came back with their key patterns intact, the init container declined to overwrite, unauthenticated access got NOAUTH, and a user writing outside its prefix got NOPERM. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wajog7nELA3i8JWTjxYGHF
This commit is contained in:
Mukul Sharma
co-authored by
Claude Opus 5
Claude Opus 5
parent
0c68312765
commit
76b4ddd2de
@@ -0,0 +1,127 @@
|
||||
{{- $name := .Values.fullnameOverride | default "redis" -}}
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
labels:
|
||||
app: {{ $name }}
|
||||
spec:
|
||||
serviceName: {{ $name }}
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: {{ $name }}
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: {{ $name }}
|
||||
spec:
|
||||
securityContext:
|
||||
# The official image runs as the redis user (uid 999 on the Alpine
|
||||
# variant). fsGroup makes the provisioned volume group-writable so
|
||||
# Redis can write the ACL file it is given — without it ACL SAVE
|
||||
# fails at provisioning time with a permission error.
|
||||
fsGroup: 999
|
||||
terminationGracePeriodSeconds: 30
|
||||
initContainers:
|
||||
# Seeds the ACL file with the default (admin) user on first boot
|
||||
# only. Redis will not start with an --aclfile that does not exist,
|
||||
# and the default user has to be defined there rather than by
|
||||
# requirepass — see the long note in values.yaml for why that
|
||||
# distinction is a security property and not a preference.
|
||||
#
|
||||
# Never overwrites an existing file. That file is rewritten by ACL
|
||||
# SAVE every time toolshed provisions an app user, so recreating it
|
||||
# on every pod start would silently delete every provisioned user
|
||||
# and lock those apps out — the exact failure this whole design
|
||||
# exists to prevent, reintroduced from the other end.
|
||||
#
|
||||
# Consequence worth knowing: rotating the admin password in Vault
|
||||
# does NOT propagate here, because this only ever runs against a
|
||||
# missing file. Rotating means `ACL SETUSER default >newpassword`
|
||||
# followed by `ACL SAVE` against the running server.
|
||||
- name: seed-acl
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
env:
|
||||
- name: REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ .Values.existingSecret }}
|
||||
key: {{ .Values.secretKeys.password }}
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
if [ -f /data/users.acl ]; then
|
||||
echo "ACL file already present; leaving it alone."
|
||||
exit 0
|
||||
fi
|
||||
echo "user default on >$REDIS_PASSWORD ~* &* +@all" > /data/users.acl
|
||||
chmod 600 /data/users.acl
|
||||
echo "Seeded ACL file with the default user."
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
containers:
|
||||
- name: redis
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
args:
|
||||
- redis-server
|
||||
- --aclfile
|
||||
- /data/users.acl
|
||||
- --maxmemory
|
||||
- {{ .Values.config.maxmemory | quote }}
|
||||
- --maxmemory-policy
|
||||
- {{ .Values.config.maxmemoryPolicy | quote }}
|
||||
- --save
|
||||
- {{ .Values.config.save | quote }}
|
||||
ports:
|
||||
- name: redis
|
||||
containerPort: 6379
|
||||
protocol: TCP
|
||||
# Authenticated probes: with the ACL file in place an
|
||||
# unauthenticated PING is correctly refused with NOAUTH, so a
|
||||
# bare `redis-cli ping` would mark a perfectly healthy server as
|
||||
# failing. Run through a shell so the environment expands —
|
||||
# Kubernetes does not substitute $(VAR) inside exec probe
|
||||
# commands.
|
||||
env:
|
||||
- name: REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ .Values.existingSecret }}
|
||||
key: {{ .Values.secretKeys.password }}
|
||||
readinessProbe:
|
||||
exec:
|
||||
command: ["sh", "-c", 'redis-cli --no-auth-warning -a "$REDIS_PASSWORD" ping | grep -q PONG']
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 6
|
||||
livenessProbe:
|
||||
exec:
|
||||
command: ["sh", "-c", 'redis-cli --no-auth-warning -a "$REDIS_PASSWORD" ping | grep -q PONG']
|
||||
initialDelaySeconds: 20
|
||||
periodSeconds: 20
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 6
|
||||
resources:
|
||||
{{- toYaml .Values.resources | nindent 12 }}
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
{{- if .Values.persistence.enabled }}
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
storageClassName: {{ .Values.persistence.storageClass | quote }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.persistence.size | quote }}
|
||||
{{- end }}
|
||||
Reference in New Issue
Block a user