Mukul SharmaandClaude Opus 5
0c68312765
Provisioned rather than built by hand in Grafana's UI — same reasoning as the datasource: survives a pod restart, and a git diff shows what changed. Four rows: ingress (Envoy total RPS + connections + response class breakdown), service level (CPU/memory by namespace, filterable via a $namespace template variable, plus a current-usage table), cluster utilization (used vs actual node capacity, not an assumed limit), and total resources (cores/memory/pods/disk). Two scrape gaps found and fixed to make this possible, both in vmagent: - Contour's own ingress Envoy (projectcontour namespace — the actual data plane for everything routed through this homelab, hostPort 80/443) was not being scraped at all. Confirmed live: Cilium's separate embedded Envoy (kube-system, its own L7 policy proxy) was already flowing, via the annotation-based kubernetes-pods job — which is what first showed envoy_* metrics existed in this cluster at all — but Contour's Envoy carries no such annotation. Added an explicit job targeting the projectcontour namespace by container port (8002, the official chart's fixed Envoy metrics port) rather than guessing at pod labels this cluster's auto-detected object names may not match. - node-exporter, deployed two commits ago, was never actually being scraped either: confirmed live that kubernetes-service-endpoints (role: endpointslice, keyed on the Service's scrape annotation — where that chart puts it) finds nothing in this cluster at all, not merely down. Rather than chase why, added the same fix as Envoy: target the pod directly by its declared container port (9100). Verified against the live deployment (queried through vmui) before writing a single panel: envoy_http_downstream_rq_total, envoy_http_downstream_rq_xx, container_cpu_usage_seconds_total, container_memory_working_set_bytes, machine_cpu_cores and machine_memory_bytes all confirmed present with real data. The one exception is the "Disk free" panel, which depends on the node-exporter scrape fix landing in this same change — noted in the values file's own comment as unverified until it actually deploys. Also verified with `helm template`: the dashboard JSON round-trips through the YAML values file and the chart's own ConfigMap templating intact (19 panels both times), and vmagent's scrape_configs list still carries all 8 chart defaults plus both new jobs — nothing lost by using extraScrapeConfigs instead of overriding the full list by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wajog7nELA3i8JWTjxYGHF
devops-infra-helm-charts
This branch (main) is part of the restructuring process for the gcp-devops-admin repository, aimed at organizing helmcharts of all infrastructure tools and their corresponding value files. The purpose of this repository is to centralize and manage these resources efficiently.
Directory Structure
helm-templates
This directory is intended for caching or forking helm charts locally. If there's a need to modify or customize any helm chart, it can be done here. Otherwise, the charts will be used directly from the provider.
helm-overrides
The helm-overrides folder stores custom values files for helm charts. These files can be used to override the default values provided by the helm charts, whether they are forked or used directly from the provider.
cluster_name
Each tool within the repository may have different values based on the specific clusters. This directory is used to manage configurations and values tailored to different clusters.
manifests
The manifests directory contains manifest files that need to be applied only once. Examples include service-to-service configurations, storage classes, and any other manifest-related files necessary for the operation of the infrastructure tools.
Additional Notes
Please ensure that all changes made to this branch align with the restructuring objectives and follow the best practices for managing helm charts and infrastructure-related configurations.
For any questions or concerns, please reach out to the designated repository maintainers.