feat: CO2-only regulation via Wattnet by default

.gitignore

@@ -1,7 +1,7 @@
 __pycache__
 config.ini
 config/.env
-
+dev/
 # Python build / install artifacts (pip install ., setup.py build)
 build/
 dist/
@@ -10,6 +10,7 @@ src/*.egg-info/
 *.egg
 *.whl
 
+
 # Debian packaging build artifacts (dpkg-buildpackage / debhelper / pybuild)
 .pybuild/
 debian/.debhelper/

README.md

@@ -91,7 +91,7 @@ sudo apt install -y build-essential devscripts debhelper dh-python \
                     python3-all python3-setuptools fakeroot
 sudo apt install -y ipmitool cpufrequtils python3-numpy python3-requests
 make deb
-sudo dpkg -i ../hpc-eff_*.deb          # this also enables the cron job
+sudo dpkg -i ../hpc-eff_*.deb          # does not enable cron; see below
 ```
 
 Then configure and start:
@@ -149,7 +149,7 @@ JSON on stdout. Contract, built-in plugins, security rules and starter files:
 | Signal | Source | Notes |
 |---|---|---|
 | Electricity price | [spotovaelektrina.cz](https://spotovaelektrina.cz) (Czech OTE spot market) | Current price via API; the 12-month baseline is scraped from HTML |
-| Carbon intensity | [Nowtricity](https://www.nowtricity.com/) (default) or [Wattnet](https://api.wattnet.eu) | Selectable via `[CO2_API] TYPE`; Nowtricity needs an API key |
+| Carbon intensity | [Wattnet](https://api.wattnet.eu) (default) or [Nowtricity](https://www.nowtricity.com/) | Selectable via `[CO2_API] TYPE`; Nowtricity needs an API key, Wattnet a login (email + password, register at [api.wattnet.eu/token-request](https://api.wattnet.eu/token-request/)) |
 | Temperature | IPMI sensor, HTTP API, or your own Python module | "Bring your own reader": see [docs/configuration.md](docs/configuration.md#temperature_source-pluggable-temperature-reading) |
 
 ---

debian/postinst

@@ -6,11 +6,12 @@ set -e
 
 case "$1" in
     configure)
-        # Enable the cron job when package is installed
-        if [ -x /usr/bin/hpc-eff ]; then
-            echo "Setting up cron job for hpc-eff..."
-            /usr/bin/hpc-eff --enable 2>/dev/null || true
-        fi
+        # Cron is NOT enabled here: the shipped config has placeholder API keys,
+        # so regulating before the admin configures it would cap the CPU on no
+        # data. Upgrades keep an existing /etc/cron.d/hpc-eff untouched.
+        echo "hpc-eff installed. Set WATTNET_EMAIL and WATTNET_PASSWORD in"
+        echo "/etc/hpc_eff/config.ini (chmod 600), test with 'sudo hpc-eff', then"
+        echo "enable cron with 'sudo hpc-eff --enable'."
         ;;
     abort-upgrade|abort-remove|abort-deconfigure)
         ;;

docs/configuration.md

@@ -80,12 +80,12 @@ only, never acted on.
 
 ```ini
 [CO2_API]
-TYPE=nowtricity
+TYPE=wattnet
 ```
 
 `TYPE` selects the backend; the other backend's keys are ignored.
 
-### `TYPE=nowtricity` (default)
+### `TYPE=nowtricity`
 
 | Key | Default |
 |---|---|
@@ -98,22 +98,81 @@ Two endpoints are called per run: `emissions-previous-24h/<zone>/` and
 node per day: check that against your plan's quota before a cluster-wide
 rollout.
 
-An API key for the default `nowtricity` backend is sent as `X-Api-Key`; get one
+An API key for the `nowtricity` backend is sent as `X-Api-Key`; get one
 at nowtricity.com.
 
-### `TYPE=wattnet`
+### `TYPE=wattnet` (default)
+
+**Wattnet requires a login (email and password). There are no API keys.**
 
 | Key | Default |
 |---|---|
+| `WATTNET_EMAIL` | none, **required** |
+| `WATTNET_PASSWORD` | none, **required** |
+| `WATTNET_TOKEN_CACHE` | `/var/lib/hpc_eff/wattnet_token.json` |
+| `WATTNET_TOKEN_URL` | `https://api.wattnet.eu/token-request/get_token` |
 | `WATTNET_URL` | `https://api.wattnet.eu/v1/footprints` |
 | `WATTNET_ZONE` | `CZ` |
 | `WATTNET_FOOTPRINT_TYPE` | `carbon` |
 | `WATTNET_SCOPE` | `operational` |
 
-The 24-hour window is computed automatically. Wattnet returns 15-minute
-samples, which are averaged into 24 hourly values; the newest hourly average
-doubles as the "current" value. **If fewer than 24 hourly values come back the
-call fails** and the run falls back to the price rating.
+```ini
+[CO2_API]
+TYPE=wattnet
+WATTNET_EMAIL=throttlepoint@your-site.example
+WATTNET_PASSWORD=<password>
+```
+
+#### Getting a login
+
+1. Request API access at [wattnet.eu/contact](https://wattnet.eu/contact).
+   Registration only works once your email is approved.
+2. Register once with the token service at
+   [https://api.wattnet.eu/token-request/](https://api.wattnet.eu/token-request/):
+   ```bash
+   curl -X POST https://api.wattnet.eu/token-request/register \
+        -H "Content-Type: application/json" \
+        -d '{"email": "<email>", "password": "<password>"}'
+   ```
+3. Put the same email and password in `[CO2_API]`.
+
+Wattnet suggests a dedicated account for unattended services, not a personal
+one. The email does not need to be a real mailbox.
+
+#### How the token is handled
+
+Wattnet tokens are valid for 24 hours and there are no refresh tokens. On the
+first run ThrottlePoint logs in at `/token-request/get_token` and caches the
+token in `WATTNET_TOKEN_CACHE` (mode 600). Later runs reuse it and log in again
+about 10 minutes before it expires: one login per node per day, as Wattnet
+asks. A wrong email or password fails the CO₂ fetch with
+`Wattnet login rejected`.
+
+The config file now holds a password: make it readable by root only
+(`chmod 600 /etc/hpc_eff/config.ini`).
+
+Full API documentation: [docs.wattnet.eu](https://docs.wattnet.eu/user-guide/api/api-access/).
+
+#### Data
+
+The 24-hour window is computed automatically. Wattnet splits the response into
+several series by data state: final data ends about 4 hours ago, newer data is
+pending or a preview and may still be revised slightly. All series are merged
+(final data wins on overlap) so the newest point is about now. The 15-minute
+samples are averaged per clock hour; the newest, possibly partial, hour is the
+"current" value. **If fewer than 24 hourly values come back the call fails**,
+and the CO₂ fallback below applies.
+
+### When the CO₂ fetch fails
+
+The last real CO₂ grade is reused if it is at most `MAX_AGE_MIN` minutes old
+(default 120, i.e. one missed hourly value plus slack). Older or none: rating 1,
+no slowdown without evidence. Each failure logs a WARNING and a
+`CO2 unavailable (...)` summary in `state.json`.
+
+| Key | Default |
+|---|---|
+| `MAX_AGE_MIN` | `120` |
 
 ---
 

docs/deployment.md

@@ -43,8 +43,9 @@ The four things to set beyond the mode itself:
 
 | Section / key | Set it to | Notes |
 |---|---|---|
-| `[CO2_API] TYPE` | `nowtricity` or `wattnet` | Picks the backend. |
-| `[CO2_API] NOWTRICITY_API_KEY` | your key | Also set `NOWTRICITY_USER_AGENT` to something identifying your site. |
+| `[CO2_API] TYPE` | `wattnet` (default) or `nowtricity` | Picks the backend. |
+| `[CO2_API] NOWTRICITY_API_KEY` | your key | Only for `TYPE=nowtricity`. Also set `NOWTRICITY_USER_AGENT` to something identifying your site. |
+| `[CO2_API] WATTNET_EMAIL`, `WATTNET_PASSWORD` | your Wattnet login | Only for `TYPE=wattnet`. Wattnet has no API keys; register at [api.wattnet.eu/token-request](https://api.wattnet.eu/token-request/), see [configuration.md](configuration.md#typewattnet-default). |
 | `[frequency_tables]` | frequencies for your CPU, MHz, highest first | See [the frequency table](#co2-mode-the-frequency-table). |
 | `[aggregation]` | `rating_type` and weights | See below. |
 
@@ -61,11 +62,10 @@ DEBUG=yes
 control_mode=co2
 
 [CO2_API]
-TYPE=nowtricity
-NOWTRICITY_USER_AGENT=fzu-throttlepoint
-NOWTRICITY_API_KEY=<your key>
-NOWTRICITY_BASE_URL=https://www.nowtricity.com/api
-NOWTRICITY_ZONE=czech-republic
+TYPE=wattnet
+WATTNET_EMAIL=<your Wattnet login>
+WATTNET_PASSWORD=<password>
+WATTNET_ZONE=CZ
 
 [aggregation]
 rating_type=average

docs/install.md

@@ -6,7 +6,8 @@ configure and start it.
 
 ThrottlePoint is not a daemon. It is a short-lived program that root runs
 periodically from `/etc/cron.d/hpc-eff`, every 10 minutes by default. What it
-does on each run depends on `[MODE] control_mode`; see
+does on each run depends on the shipped `[PIPELINE]` (CO₂ only by default) or,
+if you remove it, on `[MODE] control_mode`; see
 [regulation-modes.md](regulation-modes.md).
 
 ---
@@ -23,12 +24,12 @@ Required in both modes:
 | A writable cpufreq interface | Otherwise the cap silently does nothing | `cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq` |
 | `ipmitool` + DCMI-capable BMC | Power reading for the log (optional) | `ipmitool dcmi power reading` |
 
-Additionally, for `control_mode = co2`:
+Additionally, for CO₂ regulation (the default):
 
 | Requirement | Why | Check |
 |---|---|---|
 | Outbound HTTPS | Price and CO₂ data | see below |
-| A Nowtricity API key | CO₂ rating | https://www.nowtricity.com/ |
+| A Wattnet login (default) **or** a Nowtricity API key | CO₂ rating | Wattnet: email + password, registered at https://api.wattnet.eu/token-request/ (see [configuration.md](configuration.md#typewattnet-default)) · Nowtricity: https://www.nowtricity.com/ |
 
 Additionally, for `control_mode = temperature`:
 
@@ -46,8 +47,8 @@ config; set `https_proxy` in the cron environment if you need one.
 | Host | Purpose | Required? |
 |---|---|---|
 | `spotovaelektrina.cz` | Current price and historical monthly averages | Yes: price rating |
-| `www.nowtricity.com` | CO₂ intensity (default backend) | Only if `TYPE=nowtricity` |
-| `api.wattnet.eu` | CO₂ intensity (alternative backend) | Only if `TYPE=wattnet` |
+| `www.nowtricity.com` | CO₂ intensity (alternative backend) | Only if `TYPE=nowtricity` |
+| `api.wattnet.eu` | CO₂ intensity and the daily token login (default backend) | Only if `TYPE=wattnet` |
 
 Pre-flight check:
 
@@ -138,10 +139,10 @@ gh run download <run-id> -n hpc-eff-deb   # or hpc-eff-rpm
 | `/var/lib/hpc_eff/state.json` | Current state and rolling history, first run |
 | `/etc/cron.d/hpc-eff` | Created by `hpc-eff --enable` |
 
-> **Caution:** the DEB `postinst` runs `hpc-eff --enable` automatically, so on
-> Debian and Ubuntu the cron job is live the moment you install, before you
-> have configured anything. The RPM does not do this. On Debian, either
-> configure immediately or run `sudo hpc-eff --disable` first.
+> **Note:** neither the DEB nor the RPM enables cron on install. Nothing
+> regulates until you run `sudo hpc-eff --enable`, after setting the API key
+> and checking one manual `sudo hpc-eff` run. Upgrading keeps an existing
+> cron job.
 
 ---
 

docs/troubleshooting.md

@@ -99,6 +99,9 @@ Test each source directly:
 curl -v https://spotovaelektrina.cz/api/v1/price/get-actual-price-czk
 curl -v -H "X-Api-Key: $KEY" -H "User-Agent: test" \
      https://www.nowtricity.com/api/current-emissions/czech-republic/
+# Wattnet: log in, then query with the returned token
+curl -X POST https://api.wattnet.eu/token-request/get_token \
+     -H "Content-Type: application/json" -d '{"email": "<email>", "password": "<password>"}'
 ```
 
 Usual causes: no outbound HTTPS from compute nodes (very common on HPC
@@ -123,6 +126,9 @@ is why the node keeps working and nothing looks broken.
 |---|---|
 | `nowtricity` | Missing/expired `NOWTRICITY_API_KEY`, or the quota is exhausted. Each run makes **two** calls: 288/day/node at a 10-minute interval. |
 | `nowtricity` | Wrong `NOWTRICITY_ZONE` slug. |
+| `wattnet` | `Wattnet needs WATTNET_EMAIL and WATTNET_PASSWORD`: the login is not set. |
+| `wattnet` | `Wattnet login rejected`: wrong email/password, or the account is not registered at [api.wattnet.eu/token-request](https://api.wattnet.eu/token-request/). |
+| `wattnet` | `Wattnet rejected a freshly issued token`: login works but the account has no API access. Contact Wattnet. (A rejected *cached* token is handled automatically: Wattnet answers it with a redirect to its login page, and ThrottlePoint logs in again once.) |
 | `wattnet` | Fewer than 24 hourly values returned; the call raises rather than working with a partial window. |
 | both | Egress blocked to that host specifically. |
 

src/hpc_eff/config.ini.example

@@ -5,8 +5,8 @@ POWERREADINGCMD=ipmitool dcmi power reading
 DEBUG=yes
 
 [CO2_API]
-# Which CO2 intensity API to use: nowtricity | wattnet
-TYPE=nowtricity
+# Which CO2 intensity API to use: wattnet (default) | nowtricity
+TYPE=wattnet
 
 # Nowtricity API credentials (https://www.nowtricity.com/)
 NOWTRICITY_USER_AGENT=UserAgentString
@@ -17,11 +17,22 @@ NOWTRICITY_ZONE=czech-republic
 # Wattnet API settings (used only when TYPE=wattnet)
 # Start/end timestamps are calculated automatically for the trailing 24h
 WATTNET_URL=https://api.wattnet.eu/v1/footprints
-WATTNET_API_KEY=***
+# Login. Wattnet has no API keys: tokens last 24 h and cannot be refreshed,
+# so ThrottlePoint requests one with these credentials and caches it in
+# WATTNET_TOKEN_CACHE until shortly before it expires. Wattnet suggests a
+# dedicated account (the email need not be a real mailbox), not a personal one.
+WATTNET_EMAIL=
+WATTNET_PASSWORD=
+#WATTNET_TOKEN_CACHE=/var/lib/hpc_eff/wattnet_token.json
 WATTNET_ZONE=CZ
 WATTNET_FOOTPRINT_TYPE=carbon
 WATTNET_SCOPE=operational
 
+# When the CO2 API fails, reuse the last real grade if it is at most this many
+# minutes old (data is hourly, so 120 = one missed hour plus slack). Older or
+# none: rating 1, i.e. no slowdown without evidence.
+MAX_AGE_MIN=120
+
 [logging]
 db_path=/var/lib/hpc_eff/history.db
 log_level=INFO
@@ -154,12 +165,15 @@ weight_co2=0.4
 # Built-ins:  read: power, cpufreq | score: price, co2, combine |
 #             act: cpu_freq, cpu_thermo, gpu_power | record: db_log, state_json
 #
-# Example: regulate by CO2 alone (Nowtricity or Wattnet per [CO2_API]), no price:
-#
-#[PIPELINE]
-#score = co2
-#act   = cpu_freq
-#
+# DEFAULT: regulate by CO2 alone (Nowtricity or Wattnet per [CO2_API]), no price.
+# While this section is present, [MODE] control_mode is ignored. To use the
+# price+CO2 blend or temperature mode instead, comment this section out.
+
+[PIPELINE]
+read  = power, cpufreq
+score = co2
+act   = cpu_freq
+
 # Example: swap in your own executable. A [plugin:NAME] section with `exec`
 # (absolute path, root-owned, not group/world-writable) overrides a built-in
 # of the same name or defines a new plugin. Other keys are passed to the plugin.

src/hpc_eff/pipeline/builtins.py

@@ -8,7 +8,9 @@
 """
 
 import json
+import logging
 import traceback
+from datetime import datetime, timezone
 from pathlib import Path
 
 from ..utils.energy_price import get_current_energy_price, get_averages_year, classify_price_by_median
@@ -146,8 +148,37 @@ def price(req, config, rt):
     return {"set": values, "state": state}
 
 
+def _last_fresh_co2_grade(config):
+    """(grade, age in minutes) of the newest real CO2 grade in state.json, or None.
+
+    Grid CO2 is published hourly, so a grade up to [CO2_API] MAX_AGE_MIN old
+    (default 120) still describes the grid; older ones are a guess.
+    """
+    max_age = config.getint("CO2_API", "MAX_AGE_MIN", fallback=120)
+    path = config.get("logging", "state_json_path", fallback=DEFAULT_STATE_PATH)
+    try:
+        with open(path, "r") as f:
+            history = json.load(f).get("history", [])
+    except Exception:
+        return None
+    now = datetime.now(timezone.utc)
+    for entry in history:
+        if entry.get("co2_fallback") or not isinstance(entry.get("co2_grade"), int):
+            continue
+        try:
+            ts = datetime.strptime(entry["timestamp"], "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=timezone.utc)
+        except (KeyError, ValueError):
+            continue
+        age_min = int((now - ts).total_seconds() // 60)
+        if age_min <= max_age:
+            return entry["co2_grade"], age_min
+        return None
+    return None
+
+
 def co2(req, config, rt):
     """CO2 intensity (Nowtricity or Wattnet per [CO2_API]), graded 1-10 as a 24 h percentile."""
+    co2_error = None
     try:
         history, current, median, grade = co2_value(config)
         rt.debug_log(f"Last 24 hours CO2 values (g CO2eq/kWh): {history}")
@@ -157,6 +188,16 @@ def co2(req, config, rt):
     except Exception as e:
         current, median, grade = None, None, "unknown"
         rt.debug_log(f"Error fetching co2 values and rating: {e}")
+        logging.getLogger("hpc_eff").warning("CO2 data unavailable: %s", e)
+        # The reason goes into state.json: cron discards the log output
+        reason = " ".join(str(e).split())[:160]
+        co2_error = f"CO2 unavailable ({type(e).__name__}: {reason})"
+        last = _last_fresh_co2_grade(config)
+        if last is not None:
+            grade, age_min = last
+            co2_error += f"; reusing grade {grade} from {age_min} min ago"
+        else:
+            co2_error += "; no CO2 data within max age, rating 1 (no slowdown)"
 
     values = {
         "co2_current": current,
@@ -166,9 +207,17 @@ def co2(req, config, rt):
         "rating_co2": grade if isinstance(grade, int) else None,
     }
     state = dict(values)
+    # A fallback grade is never reused itself, so a long outage cannot chain it forward
+    state["co2_fallback"] = co2_error is not None
+    if co2_error:
+        state["summary"] = co2_error
     if values["rating_co2"] is not None:
         # On its own, the CO2 grade is the rating. `combine`, when listed, overrides it.
         values["rating"] = state["rating"] = values["rating_co2"]
+    elif co2_error:
+        # No recent data: do not slow the node down without evidence. rating_co2
+        # stays None, so `combine` (when listed) falls back to price as before.
+        values["rating"] = state["rating"] = 1
     return {"set": values, "state": state}