Replace Prometheus with VictoriaMetrics (victoria-metrics-single + vmagent)

Lower RAM/disk for the same metric volume on an 8GB single node — VM's
own compression is the entire reason it exists as a project. Speaks
Prometheus's own query API (/api/v1/query), so nothing downstream
needed to change beyond the URL it points at: same PromQL, same
kubernetes-nodes-cadvisor-sourced container_cpu_usage_seconds_total /
container_memory_working_set_bytes toolshed's metrics work
(docs/PRODUCT-ARCHITECTURE.md step 5) already targets.

Three releases, matching the one-release-per-component convention
already used everywhere in this repo, since victoria-metrics-single has
no bundled scraper/exporter subcharts the way the Prometheus chart did:

- victoria-metrics-single: the TSDB + query engine. 3Gi/local-path,
  7-day retention, resources capped at 512Mi — same trim reasoning as
  the Prometheus server it replaces.
- vmagent: the scraper. Its default scrape_configs already includes
  kubernetes-nodes-cadvisor and kubernetes-service-endpoints (the
  chart's own comment says "COPY from Prometheus helm chart") — nothing
  to override there, only remoteWrite pointed at victoria-metrics-single
  and trimmed resources.
- node-exporter: pulled out of the removed Prometheus chart's bundled
  subchart into its own standalone release, since victoria-metrics-single
  has no equivalent. Same trim as before (32Mi/64Mi), same
  prometheus.io/scrape annotation vmagent's default scrape config
  already looks for.

Found and fixed while vendoring: helm-templates/victoria-metrics-single
already existed in this repo — a leftover GKE-targeted vendored copy
from the original Meesho monorepo import (commit b8575bb), with its own
templates/values.yaml for a different chart entirely. `mkdir -p` on an
already-existing directory silently did nothing, so the first vendoring
attempt left that old tree sitting alongside the new one and rendered
two ServiceAccounts/Services/StatefulSets with colliding names. Removed
outright rather than adapted, same reasoning as the ~35 charts
devops-helm-charts removed in its own prune — see this chart's own
Chart.yaml for the full note.

Verified with `helm template` against the real charts and these values
for all three releases individually, and again for the whole
generic-argo-apps-chart appSpec list (11 Applications render, including
the three new ones and nothing orphaned from the old prometheus entry).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wajog7nELA3i8JWTjxYGHF
This commit is contained in:
Mukul Sharma
2026-09-06 08:38:34 +05:30
co-authored by Claude Opus 5
parent 6677c9e315
commit 04e7ef2f82
37 changed files with 126 additions and 2465 deletions
@@ -0,0 +1,28 @@
victoria-metrics-agent:
# Ships to victoria-metrics-single. The write path is /api/v1/write,
# the same endpoint any Prometheus remote_write client uses — this is
# the one config line that actually wires the two components together;
# the chart names don't imply it on their own.
#
# Cluster-internal Service DNS: this call happens from a pod, so
# CoreDNS resolves it. Same reasoning as everywhere else in this
# project that a Service name (not an Ingress hostname) is correct
# here — see claude.md's registry-hostname convention for the one case
# where cluster DNS specifically does NOT work (a node-level pull),
# which does not apply to this pod-to-pod write.
remoteWrite:
- url: http://victoria-metrics-single-server.monitoring.svc.cluster.local:8428/api/v1/write
# config.scrape_configs is left at the chart's own default — it already
# includes kubernetes-nodes-cadvisor (kubelet's cAdvisor endpoint,
# comment literally says "COPY from Prometheus helm chart") and
# kubernetes-service-endpoints (the prometheus.io/scrape annotation
# convention node-exporter's Service carries). Nothing here needs a
# scrape target the chart doesn't already define out of the box.
resources:
requests:
cpu: 25m
memory: 64Mi
limits:
memory: 192Mi