Skip to content
HoneyBearTechPublic

About

Homelab monitoring as code: Grafana dashboards and alert rules, Prometheus, Loki and exporters for Docker hosts, the media stack, network, NAS, UPS, smart home, GitLab CI and Proxmox VE, shipped as signed Docker images with a one-container agent per host.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

103 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Argus

CI Release images CodeQL OpenSSF Scorecard OpenSSF Best Practices OpenSSF Baseline Docker Pulls Version Agent image size Grafana License

Grafana dashboards as code for a homelab: version-controlled JSON dashboards and alert rules for Docker hosts and their logs, the Plex/arr media stack, reverse proxy traffic, DNS, the network and internet connection, NAS storage, UPS power, the smart home, GitLab CI, Steerhouse (an Ansible web UI), Tdarr, Plex's current streams (through Tautulli) and the Proxmox VE cluster — edited in VS Code, provisioned automatically, and shipped as Docker images with a one-container agent for each host.

How it works

Grafana, Prometheus, blackbox_exporter, Loki and exporters for Pi-hole, UniFi, SNMP, the media apps, Tdarr, Tautulli (through Varken), GitLab CI and Proxmox run on one server from docker-compose.yml (the optional ones behind Compose profiles, and Loki on another host if you prefer); an Argus agent on every monitored host pushes its metrics and logs there. Grafana loads every dashboard under dashboards/ through file provisioning — each subfolder becomes a Grafana folder, and UI edits are blocked so this repo stays the source of truth. The same compose file runs production and a local sandbox; per-install settings come from a gitignored .env.

dashboards/          dashboard JSON, one folder per Grafana folder
provisioning/        Grafana datasources (pinned uids) and the dashboard provider
prometheus/          Prometheus config — scrape jobs, no addresses
blackbox/            blackbox_exporter probe modules (HTTP service check, DNS resolve)
agent/               the Argus agent image (Grafana Alloy + config) and its compose file, one per monitored host
logs/                Loki config, plus a compose file to run Loki on a separate host with more disk
images/              Dockerfiles for the published server images
targets.example/     placeholder scrape/probe targets; real ones go in targets/ (gitignored)
secrets.example/     placeholder credentials Prometheus scrapes with; real ones go in secrets/ (gitignored)
scripts/             repo checks and the PromQL test builder, run in CI; the Steerhouse dashboard port
tests/               unit tests (pytest) and PromQL tests (tests/promql/, run with promtool)
docs/                user, security and project documentation

Dashboards

Dashboard Folder Answers Data
Homelab Overview (home page) overview Is everything up right now, and if not, what? Uptime Kuma /metrics, blackbox_exporter HTTP + DNS probes
DNS / Pi-hole dns Is DNS healthy and consistent, and what's being blocked? pihole-exporter (Pi-hole v6 API), blackbox DNS probes
Host Health hosts Is any machine running out of disk, memory or CPU? Argus agent on each host (embedded node_exporter)
GPUs hosts Are the GPUs doing the transcoding, and are they healthy? nvidia_gpu_exporter (nvidia-smi) beside the agent on a GPU host (agent/compose.yml, gpu profile)
Docker Containers hosts Which containers are restarting, stopped, or eating resources? Argus agent on each host (embedded cAdvisor)
Network / UniFi network How's the internet connection, and what's on the network? unpoller with a read-only UniFi account
Internet / ISP network Is the ISP delivering the speed we pay for, and how often does the connection drop? unpoller: the UniFi gateway's own speed tests and the plan speeds set in UniFi
Media Stack media Who's watching what, is the download pipeline flowing, and is the library healthy? exportarr (*arr, SABnzbd), qbittorrent-exporter; Plex exporter beside Plex (agent/compose.yml, plex profile)
Tautulli media What's playing on Plex right now, and how is it being delivered? Varken's Prometheus output, reading Tautulli's API (varken profile): streams, transcode decisions, bandwidth, sessions by quality and platform, library sizes
Varken media What's on Plex, who's watching from where, and what are Sonarr, Radarr and Seerr waiting on? The original Varken dashboard, ported to Prometheus, with Seerr in place of Ombi and without SickChill: Varken 3.0's metrics with stream_details on (viewer names, titles and cities) and GeoIP on (a MaxMind account: maxmind_license_key secret, account ID in varken.env) for the map; Sonarr and Radarr (and their 4K instances, with an HD/4K column) and Seerr's requests and issues through Varken too
Varken [Classic] media The same, in the original Varken dashboard's look The Varken dashboard's queries, restyled like the original "Varken [Official]" (coloured tiles and rows, heat strip, US map)
Tdarr media Is Tdarr keeping up, and how much space has it saved? tdarr-exporter through Tdarr's API (tdarr profile); the Tdarr container's CPU, memory and logs from the Argus agent
NAS / Storage storage Are the NAS units healthy, and how fast are they filling? snmp-exporter (bundled synology + standard MIB modules, and Argus' synology_flashcache for SSD caches), SNMPv3
Smart Home home What's the house doing — temperatures, HVAC, energy? Home Assistant's Prometheus integration (long-lived token)
Reverse Proxy Traffic network What's hitting the services behind the reverse proxy, and is anything erroring? Nginx Proxy Manager access logs → Alloy → Loki
Logs hosts What did a host or container log around the time something broke? Docker container logs → Alloy → Loki
Deploys ci Did every merge reach its host, and is every host's deploy agent alive? homelab-pull's metrics and deploy log (a pull-based deploy agent: one stack per host, deployed from its git branch), read by the Argus agent (HOMELAB_PULL_DIR); deploy markers on Host Health and Docker Containers
GitLab CI ci Are pipelines healthy, are the runners keeping up, and is GitLab itself healthy? gitlab-ci-pipelines-exporter with a read_api token (gitlab profile); GitLab Runner's metrics endpoint; a subset of GitLab's bundled Prometheus (Omnibus), read through /federate
Steerhouse: overview & runs ci Is Steerhouse healthy right now, and how have its Ansible runs gone: which failed, and how long did they take? Steerhouse's token-protected /metrics (port 8002) and its database's read-only analytics views, as the grafana_ro role over TLS; the three Steerhouse dashboards are Steerhouse's own, ported by scripts/port_steerhouse.py
Steerhouse: execution, queue & workers ci Is Steerhouse keeping up: are runs waiting for a worker, are notifications going out, and is its API healthy? The same
Steerhouse: security & access ci Is anyone trying to get into Steerhouse, and who changed users, access or keys? The same
Proxmox VE proxmox Are the Proxmox nodes overcommitted, are the guests that should run running, and how is storage filling? prometheus-pve-exporter with a read-only API token (proxmox profile)
Server Hardware proxmox Is the server hardware healthy: temperatures, fans, power supplies, batteries, power draw, room in the BMCs' event logs, and add-in GPUs? prometheus-ipmi-exporter on each server (Debian package), reading its own BMC through /dev/ipmi0 — see docs/ipmi.md; add-in GPUs, which the BMC can't see, from node_exporter's hwmon collector (Debian package, hwmon only) — see docs/hwmon.md
UPS / Power power How long would the lab survive a power cut, and is the UPS healthy? PeaNUT /api/v1/metrics

Alerts

Grafana-managed alert rules live in provisioning/alerting/rules.yaml (edit there — UI changes are blocked) and notify a Discord channel through the webhook in secrets/discord_webhook_url (its template points nowhere, so a sandbox can't post). Each rule is a raw PromQL measurement (A) plus a threshold (C), one alert per series:

Group Rules
Availability host stopped reporting (its agent went silent), service down (only services that were up for at least an hour in the last 7 days), DNS server not answering, monitoring target down (not a host's GPU exporter while that host says it's on a non-NVIDIA GPU), Proxmox node without cluster quorum, Proxmox QDevice disconnected for 5 minutes, GPU exporter reporting no GPU (driver not loaded)
Capacity local disk > 90 % (network shares are covered by the NAS pool rule), NAS storage pool > 90 %, certificate < 14 days, Proxmox node memory > 90 % used, Proxmox node memory overcommitted (running guests allocated > 100 %)
Power and hardware UPS on battery, UPS battery < 50 %, NAS disk unhealthy, NAS pool degraded, NAS disk > 55 °C, network device > 85 °C, GPU > 85 °C (NVIDIA, and Intel cards read by the server's driver), a BMC sensor critical for 5 minutes (a failed battery, power supply or fan, or a reading past its critical limit; empty slots and Supermicro's LED pattern sensors don't count), a BMC sensor in warning for 15 minutes, a BMC that can't be read for 10 minutes, a BMC event log with < 1 KiB left
Network internet on backup WAN, internet dropped, internet much slower than usual (latest speed test under half its 14-day median), no speed test in 8 days, Pi-hole blocklists out of sync
Containers container restarted more than 3 times in an hour
Logs a site returning > 20 server errors (5xx) in 10 minutes, log pipeline stalled (Loki receiving almost nothing for 15 minutes)
Tautulli Varken can't read Tautulli for 10 minutes (down, wrong API key or an unreadable reply), Tautulli data stale for 10 minutes while Varken still reports it up (a hanging request)
Tdarr new transcode or health-check errors in the last hour, files queued but no transcode or health check finished in a day
Deploys a deploy agent that hasn't finished a run for 50 minutes, a deploy failing for 15 minutes, every run of an agent failing for 15 minutes (fetch or settings), deploys paused for 2 hours (the checkout isn't on its branch)
GitLab a GitLab component (Puma, Sidekiq, Gitaly, PostgreSQL, …) down for 5 minutes, the latest pipeline on a watched branch failed
Steerhouse more than 30 % of the runs that ended in the last hour failed or timed out (for 15 minutes), over 30 failed sign-ins in 15 minutes, a notification waiting over 15 minutes to be sent, Steerhouse's API unable to read its database for 5 minutes. Single events (a failed run, a worker offline, a lockout) are Steerhouse's own notifications; its metrics not arriving is "monitoring target down"

Alerts are grouped per rule and repeat every 12 hours while firing.

Running it

Requirements: Docker with Compose v2.24 or later, on amd64 or arm64. One host runs the server; every host you want to watch runs the agent.

Argus ships as images for amd64 and arm64 on GHCR (ghcr.io/honeybeartech/argus-*) and Docker Hub (honeybeartech/argus-*; set ARGUS_REGISTRY=docker.io/honeybeartech in .env to use it). The Grafana image has every dashboard, datasource and alert rule baked in; the Prometheus, blackbox and Loki images carry their configs. What stays on the host is what's specific to your network: scrape targets, credentials and .env.

Server (one host — Grafana, Prometheus, blackbox_exporter, Loki):

git clone --depth 1 https://github.com/HoneyBearTech/Argus.git && cd Argus
cp .env.example .env              # set GRAFANA_ROOT_URL
mkdir -p targets secrets && cp targets.example/*.json targets/ && cp secrets.example/* secrets/
openssl rand -base64 24 > secrets/grafana_admin_password   # Grafana's admin password (user admin)
# put your real addresses in targets/ and credentials in secrets/ — delete what you don't use
docker compose up -d

Then run the agent on each host you want to watch (next section). Grafana is on port 3000; the home page is the Homelab Overview. Edits to targets/*.json are picked up by Prometheus without a restart.

Host Health, Docker Containers and Logs work with just the agents. Every other dashboard lights up as you connect the service it reads. COMPOSE_PROFILES starts as loki: add profiles to that list (loki,snmp) rather than replacing it, or Loki stops (unless it runs on another host, with LOKI_URL set):

To see Add to targets/ Credentials in secrets/ Add to COMPOSE_PROFILES in .env
Service up/down, certificates (Overview) blackbox_http.json (URLs), blackbox_dns.json (DNS servers) — —
Uptime Kuma monitors (Overview) uptime_kuma.json uptime_kuma_api_key —
DNS / Pi-hole blackbox_dns.json pihole-exporter.env —
Network / UniFi, Internet / ISP — unpoller.env (console URL + read-only local account), unifi_password —
NAS / Storage snmp.json (add synology_flashcache to a Synology's module list for its SSD cache) snmp-auth.yml (SNMPv3) snmp
Smart Home homeassistant.json home_assistant_token —
UPS / Power peanut.json peanut_username, peanut_password (PeaNUT 6's login; PeaNUT 5 ignores them) —
Media Stack (*arr, SABnzbd, qBittorrent) media.json (enabled exporters only) <app>.env (URL), <app>_api_key (qBittorrent: qbittorrent.env with URL + user, qbittorrent_password) the app's profile, e.g. radarr,sonarr
Tautulli varken.json (varken:9180) varken.env (Tautulli's, Sonarr's, Radarr's, Lidarr's and Seerr's addresses, SSL settings), tautulli_api_key and the Sonarr/Radarr (and 4K) and Lidarr API keys exportarr uses, seerr_api_key; and CONFIG_ROOT/varken/config owned by uid 1000 varken
Tdarr tdarr.json (tdarr-exporter:9090) tdarr.env (Tdarr's URL; and TDARR_API_KEY once Tdarr's own login is on, which the exporter reads only from its environment) tdarr
Server Hardware ipmi.json (<server>:9290, with a host label) ipmi_password (the exporters' basic auth; set up each server with deploy/ipmi/, see docs/ipmi.md) —
Server Hardware (add-in GPUs) hwmon.json (<server>:9100, with the same host label) hwmon_password (node_exporter's basic auth; set up each server with deploy/hwmon/, see docs/hwmon.md) —
GPUs gpu.json (<gpu host>:9835, with a host label) — gpu, in that host's agent .env (needs the NVIDIA Container Toolkit)
Media Stack (Plex) plex.json PLEX_TOKEN in the Plex host's agent .env plex, in that host's agent .env
Loki's own health loki.json (loki:3100 for the bundled one) — —
GitLab CI (pipelines) gitlab.json gitlab.yml (GitLab's URL and a read_api token) gitlab
GitLab CI (runners) gitlab_runner.json (<runner>:9252, one target per runner); set listen_address = ":9252" in each runner's config.toml — —
GitLab CI (GitLab's health) gitlab_server.json (<gitlab>:9090); set prometheus['listen_address'] = '0.0.0.0:9090' in gitlab.rb and run gitlab-ctl reconfigure (restarts Puma and Sidekiq: a minute or two of 502s) — —
Steerhouse steerhouse.json (<steerhouse host>:8002, the backend's metrics port, which Steerhouse serves once METRICS_TOKEN and METRICS_HOST are set); and STEERHOUSE_DB_ADDRESS in .env (its PostgreSQL, host:port) steerhouse_metrics_token (Steerhouse's METRICS_TOKEN), steerhouse_db_password (the grafana_ro role's password: see docs/steerhouse.md), steerhouse_db_ca (the CA certificate that signed the database's TLS certificate, which must name the address) —
Proxmox VE pve.json: each node's API address (port 8006 by default), which the node's certificate must name; in a cluster, label every node but one "cluster": "0" pve.yml (an API token with the PVEAuditor role; the commands are in the template), pve-ca.pem (the cluster's CA, /etc/pve/pve-root-ca.pem on any node) proxmox

To upgrade, pull new images (docker compose pull && docker compose up -d), or pin ARGUS_VERSION in .env to a release; see docs/upgrading.md. New to Argus? The quick start runs it on one machine in about ten minutes.

Security: Prometheus (9090) and Loki (3101) accept pushes without authentication, so agents anywhere on the network can reach them; Prometheus can require a password. Run Argus on a trusted network, and put Grafana behind a reverse proxy with TLS if it's reachable from outside (docker-compose.proxy.yml puts Grafana on the network of a proxy running in Docker on the same host, so its own port can listen on loopback only). What Argus does and doesn't protect against is in docs/security.md.

Developing — to edit dashboards, alert rules or configs live, run from the checkout instead of the published images by setting COMPOSE_FILE=docker-compose.yml:docker-compose.source.yml in .env. Grafana then reads dashboards/ and provisioning/ straight from the repo (dashboard edits show up within 30 seconds), and docker compose up -d --build rebuilds the images. In this mode, deploying is git pull on the server; restart the stack only when docker-compose.yml, provisioning/ or a config changes. A systemd timer can do that for you once CI passes: see docs/auto-deploy.md.

Releases are published by pushing a signed v*.*.* tag (.github/workflows/release.yml): the images are signed with cosign and carry an SBOM and provenance, and the GitHub Release gets its notes from CHANGELOG.md and a signed checksum file. docs/verifying-releases.md shows how to check them.

Adding a host

Each monitored host runs one container, the Argus agent (agent/): Grafana Alloy with embedded node_exporter and cAdvisor that pushes host metrics, container metrics and container logs to the Argus server. Nothing on the server needs to change — the host appears in the dashboards as soon as the agent starts, and it only makes outbound connections (no firewall rules on the host).

docker run -d --name argus-agent --restart unless-stopped \
  --network host --pid host --read-only \
  --cap-drop ALL --cap-add DAC_OVERRIDE --security-opt no-new-privileges:true \
  -e ARGUS_HOST=my-host \
  -e ARGUS_PROMETHEUS_URL=http://argus-server:9090 \
  -e ARGUS_LOKI_URL=http://argus-server:3101 \
  -v /:/rootfs:ro,rslave -v /sys:/sys:ro -v /run/udev:/run/udev:ro -v /dev/disk:/dev/disk:ro \
  -v /var/lib/docker:/var/lib/docker:ro -v /var/run/docker.sock:/var/run/docker.sock:ro \
  -v /run/containerd:/run/containerd:ro -v argus-agent-data:/var/lib/alloy/data \
  ghcr.io/honeybeartech/argus-agent:latest

It runs as root in the host's network and PID namespaces with the host mounted read-only, but not privileged: its only capability is DAC_OVERRIDE (to read files that belong to other users), and its own filesystem is read-only, so it writes only its data volume. Why it needs each of these is in docs/security.md.

Or copy agent/compose.yml and agent/.env.example (as .env) to the host and run docker compose up -d; pin ARGUS_AGENT_IMAGE to a release tag for predictable upgrades. The compose file also covers four extras:

  • Nginx Proxy Manager access logs (Reverse Proxy Traffic dashboard): on the host running NPM, set NPM_LOG_DIR to its data/logs directory.
  • homelab-pull deploys (Deploys dashboard): on a host whose Docker Compose stack homelab-pull (0.4.0 or later) deploys, set HOMELAB_PULL_DIR to its state directory (~/.local/state/homelab-pull of the user it runs as). The agent reads its metrics files (metrics/*.prom, node_exporter's textfile collector) and each stack's deploys.log.
  • NVIDIA GPUs (GPUs dashboard): on a host with an NVIDIA GPU, the NVIDIA Container Toolkit 1.18 or later and Docker Engine 29.2 or later, add the gpu profile, and list that host in the server's targets/gpu.json. From 29.2, Docker serves the exporter's driver: nvidia reservation through CDI (the toolkit's nvidia-cdi-hook, with nvidia-cdi-refresh keeping the CDI spec current). An older Engine uses NVIDIA's runtime hook, and the container loses the GPU at every systemctl daemon-reload (apt and unattended-upgrades run one): nvidia-smi fails with "Failed to initialize NVML: Unknown Error" (NVIDIA container-toolkit issue #48) and the "GPU not reporting" alert fires. To check a host: sudo systemctl daemon-reload, then docker exec argus-gpu-exporter nvidia-smi -L still lists the GPU. A host whose GPU can change (a VM that fails over to a machine with an Intel GPU and stops its exporter there) can say which GPU it booted with, homelab_gpu_selected{vendor="intel"} 1, so its stopped exporter doesn't raise "Monitoring target down" (docs/interfaces.md).
  • Plex (Media Stack): on the Plex host (amd64), add the plex profile with PLEX_SERVER and PLEX_TOKEN, and list that host in the server's targets/plex.json.

The agent needs the server's Prometheus (9090) and Loki (3101) to be reachable from the host; those are the defaults. If Prometheus requires a password, add -e ARGUS_PROMETHEUS_PASSWORD=... (or set it in the agent's .env).

Adding or changing a dashboard

  1. Build or edit it in the sandbox Grafana, then Export → Export as JSON (leave "Export for sharing externally" off), or edit the JSON directly in VS Code.
  2. Save it under dashboards/<folder>/<name>.json with a stable, hand-picked uid, "id": null, the argus tag, and a description saying what question it answers.
  3. Reference datasources by uid (prometheus) — never by name or URL.
  4. Run python3 scripts/check_dashboards.py. It also fails on private IP addresses and on internal names listed one regex per line in a gitignored .forbidden-patterns file (CI reads the same list from the ARGUS_FORBIDDEN_PATTERNS secret): this repo is public, so addresses stay in targets/ and .env.

Checks

CI (.github/workflows/ci.yml) runs on every push and pull request: linters for Python, YAML, workflows, Dockerfiles and the agent config; unit tests with a coverage floor; the dashboard and public-repo checks; promtool check config and the PromQL tests, which run the shipped alert rule and dashboard queries against synthetic series; the blackbox, Loki and Alloy config checks; it builds every image, starts Grafana to check that every dashboard, alert rule and datasource loads, and validates the compose files.

Alongside it: CodeQL analyses the Python checks and the workflows on every push and pull request and weekly, a weekly Trivy scan reports known vulnerabilities in the five images, OpenSSF Scorecard scores the repo's supply-chain practices weekly (see the badge above), and Dependabot opens weekly PRs for the upstream images, GitHub Actions and the check tools. Image updates are merged by hand after a sandbox run. How dependencies are handled is in docs/dependencies.md.

Documentation

docs/ has the quick start, architecture, interfaces, upgrading, security requirements and assurance case, dependency policy and roadmap. Changes are listed in CHANGELOG.md.

Contributing

Bug reports and ideas go in GitHub Issues; see CONTRIBUTING.md for the development setup, coding standards, test policy and sign-off, SECURITY.md for reporting vulnerabilities privately, and SUPPORT.md for which versions are supported. The project's governance and code of conduct apply to everyone taking part.

License

MIT

About

Homelab monitoring as code: Grafana dashboards and alert rules, Prometheus, Loki and exporters for Docker hosts, the media stack, network, NAS, UPS, smart home, GitLab CI and Proxmox VE, shipped as signed Docker images with a one-container agent per host.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages