Skip to content

refactor(repairs): track device faults per gateway and raise them as self-clearing repair issues - #593

Merged
GreenGrassBlueOcean merged 10 commits into
OpenWebNet-HA:v2-phase1-architecturefrom
GreenGrassBlueOcean:rfc/actuator-hardware-fault-handling
Oct 3, 2026
Merged

GreenGrassBlueOcean merged 10 commits into
OpenWebNet-HA:v2-phase1-architecturefrom
GreenGrassBlueOcean:rfc/actuator-hardware-fault-handling

Conversation

@GreenGrassBlueOcean

@GreenGrassBlueOcean GreenGrassBlueOcean commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Hardware faults on the bus show up as self-clearing Home Assistant repair issues, raised by one tracker per gateway instead of code inside each entity. Discussion: #594.

The first version of this PR raised actuator_hardware_fault from inside MyHOMELight.handle_event. That missed switches and addresses with no entity, left the issue behind when the entity was deleted, and would have been copied into every platform. Its text also stated causes (F414, open load, T5H fuse) that the evidence does not show. This revision replaces it:

  • device_health.py: DeviceHealth, one per gateway (handler.device_health). It is the only code that creates or deletes per-device fault issues. It writes an issue once per change, so a stuck actuator answering every poll does not rewrite the registry.
  • Fed from the bus: GatewayEventDispatcher.process_message passes every WHO 1/1001 frame to it, so lights and switches with no entity still raise their issue. On a shared bus only the gateway owning WHO 1 raises lighting faults; WHO 1001 is gated as WHO 1.
  • Lighting rule: a WHO 1 status outside the published table raises unmapped_device_status. The next normal status (on, off or a level) clears it.
  • Documented events are not faults: WHAT 32 (Toggle), 34 (movement detected) and 39 (end of movement detected) are defined by the ZigBee OpenWebNet spec 4.0 variant of WHO 1. They neither raise nor clear a fault (DOCUMENTED_EVENTS).
  • WHO 1001 mask = evidence, never a trigger: a WHO 1001 DIMENSION 7/11 mask seen within 10 s of the odd status for the same address, in either order, is attached exactly as received (unmapped_device_status_autodiag). It is not decoded and does not raise an issue on its own (see Evidence below).
  • Entity side (MyHOMEEntity):
    • names the device in its issues (WHO x WHERE y until an entity exists);
    • drops them only when the owner deletes the entity from the registry (async_removed_from_registry), so neither a reload nor an entity_id rename clears them; an address shared by several entities keeps its issues until the last one is removed;
    • offers _report_fault / _clear_fault for faults only an entity can see.
  • Warm standby: while the primary is offline the standby feeds what it sees to the primary's tracker (health_owner()), so the primary's entry owns every issue and a recovery seen during the failover or after the failback clears it.
  • clear() always asks the registry: an issue raised before a reload (unknown to the new tracker) still clears.
  • Climate unresponsive_zone moved onto the tracker; same text, {zone} → {device}. The issue still passes zone too, so a translation that kept {zone} (e.g. in Crowdin) renders.
  • Lifecycle: issues are cleared when the entry unloads and raised again by the next setup if the fault is still there. They are non-persistent. Issue ids are device_fault_<entry_id>_<kind>_<who>_<where>, so the existing _<entry_id>_ sweep in async_remove_entry covers them.
  • Diagnostics list the active faults under gateway.device_faults, without device names.
  • Text and docs: neutral wording in en/fr/it/nl, plus a new Unmapped Device Status section in docs/diagnostics/repair-issues.md, including what to do with an actuator that will not be repaired soon (disable the entity, don't delete it; "Ignore" on the repair is temporary).

Evidence and protocol checks

  • WHAT 19 is absent from WHO_1.pdf, the Encyclopedia and OPEN.db. The only capture is one actuator on an MH200 (EVID-MH200-WHAT19-FAULT, fix(light): keep the last state when a light reports an unknown WHAT #456):
    *1*19*74##
    *#1001*74*11*111110111111111111110111##
    
  • openwebnet-mcp parse_and_validate_frame flags "Dimension 11 is not officially documented for WHO=1001". OPEN.db describes a 24-bit autodiagnostic mask but no per-bit meanings. The Encyclopedia says the WHO 1004 active-low fault labels must not be copied to WHO 1001.
  • A healthy MyHomeServer1 device interview in the cover-diagnostic traces answers *#1001*0*7*111111111111111101101111##. Zero bits do not mean a fault, which is why the mask is evidence only.
  • In the MH200 trace of 2026-09-26 the mask arrived 3.45 s before the status, which is why the evidence window works in both directions.
  • WHAT 32–39: no capture holds a direct WHO 1 status with any of them. 32/34/39 are taken from the ZigBee variant spec; WHAT 33 is in no source we hold, so it stays a fault (the test pins that assumption).

Tests

  • tests/test_device_health.py covers:
    • frame classification and issue placeholders;
    • writing once per change and clearing once;
    • evidence before and after the status, and a stale mask that is no longer attached;
    • a mask alone, F422 interface addresses, and area/group/general frames;
    • renaming, entity removal with shared addresses, clear_all, clearing an issue the tracker does not hold;
    • warm-standby ownership during failover and after failback;
    • shared-bus gating through process_message.
  • tests/test_device_health_corpus.py:
    • replays the 63 captures under tests/fixtures/traces (MH200, MH200N, MH201, MH202, F454, F455, F461, H4890, MyHomeServer1), 30 of which carry WHO 1/1001 frames. Only the two MH200 address 74 captures may raise a fault, with the expected evidence;
    • no frame of the vendored Encyclopedia golden corpus may raise one;
    • every published WHO 1 WHAT (0–18, 20–29, 30, 31) must not count as a fault; 19/33/50/99 must; 32/34/39 neither raise nor clear one.
  • test_poll_health and test_climate_central_discovery moved to the new issue ids. test_where_usage_guard lists the reviewed WHERE read in message_where.
  • Results on the current base (WSL, core 2026.9.2):
    • full suite: 3200 passed, 4 failed. The 4 failures (test_component_alarm, test_issue_311, test_issue_466_h4890_trace, test_issue_564_mh202_alarm_trace) also fail on the base branch and are unrelated (alarm decoding);
    • coverage 99.9% (the 2 missed lines are in the untouched alarm_control_panel.py);
    • ruff clean, mypy clean on the changed modules, scripts/verify_ha_standards.py passes.

Not covered yet

  • One fault example: the only positive is one actuator on one MH200 firmware. Bench captures with the load removed (F411/F414/F418U2 on MH200, MH200N, F454, MyHomeServer1; *#1*W## and *#1001*W*7## before and after) would show whether WHAT 19 is universal and which mask bits change against a healthy baseline.

  • Permanent silence: a faulty actuator that will not be repaired cannot yet be silenced for good. An ignored_addresses option (skip discovery, entities and health tracking for listed WHO/WHERE pairs) would be a separate PR.

  • Next classifiers: WHO 1004 heating faults and WHO 18 DIMENSION 250 Stop&Go breakers are specified in the published docs. The corpus has no frames of either yet, so each waits for a capture.

  • V2 Architecture: New entities implement handle_event() and do not manually subscribe via async_dispatcher_connect. (No new entities.)

🤖 Generated with Claude Code

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@GreenGrassBlueOcean GreenGrassBlueOcean changed the title rfc(repairs): surface actuator hardware anomalies as self-clearing repair issues refactor(repairs): track device faults per gateway and raise them as self-clearing repair issues Oct 1, 2026
@xtimmy86x

Copy link
Copy Markdown
Contributor

Thanks for the refactor. Moving device health tracking to the gateway makes sense, especially for switches and addresses without a configured entity. Keeping WHO 1001 masks as raw evidence without interpreting undocumented bits is also the right approach.

Before merging, I would address two points:

  1. Repair ownership during standby failover and failback. Each gateway has its own DeviceHealth, while _bridge_to_primary() only forwards messages to the primary’s entity dispatcher. A fault raised by the primary therefore remains active if the device recovers while the standby is handling frames. Conversely, a fault raised by the standby can remain after the primary reconnects, because the standby stops observing WHO 1. Please keep repair ownership consistent across these transitions and add regression tests for recovery during both failover and failback.

  2. The 10-second evidence window is only enforced in one direction. _recent_autodiag() rejects an old mask when the mask precedes the fault, but _observe_lighting_autodiag() attaches any subsequent mask to an active fault without checking elapsed time. Please apply the window in both directions and test a mask arriving more than 10 seconds after the last anomalous status.

Two smaller documentation points:

  • Removing an entity clears its repair, but the next anomalous bus frame recreates it because unconfigured addresses are monitored too. Please make this explicit.
  • The repair text says Home Assistant shows the state as unknown, but lights can retain their last known state and switches retain their previous ON/OFF state. “The received status does not allow Home Assistant to determine the current device state” would be more accurate.

The overall architecture looks good; these changes would make the repair lifecycle and evidence association more reliable.

@Interstellar0verdrive Interstellar0verdrive left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reworking this. Moving fault tracking to one tracker per gateway is the right architecture: it covers switches and unconfigured addresses, removes the per-platform duplication, and keeping the WHO 1001 mask as raw evidence is exactly how undocumented bits should be handled. I checked that WHO 2 handling and cover calibration are unaffected, and that the dispatcher still delivers frames to entities in the same order.

I agree with all four points @xtimmy86x raised (failover/failback ownership, the one-directional evidence window, the entity-removal wording, and the "shows its state as unknown" text). I reproduced the failback case: once the primary reconnects, the standby stops observing WHO 1 and its issue never clears until a reload. I won't repeat those. These are the additional points I found, in order of importance.

1. The corpus test fails on the current base (blocking).
CI ran on ae94a1a before #596 was merged. With this branch merged onto the current v2-phase1-architecture (3ac21ff, merges without conflicts), two tests fail:

  • test_device_health_corpus.py::test_the_corpus_is_not_empty
  • test_device_health_corpus.py::test_capture_raises_exactly_its_known_faults[config_entry-myhome_F455.json]

_replay() calls OWNEvent.parse() on every frame of every capture, and the new F455 fixture contains *#13**0##, which OWNd 2.0.0b9 rejects with ValueError: Gateway dimension 0 requires 3 values. I suggest filtering WHO 1/1001 with the regex before parsing, so the replay does not depend on how other WHOs are parsed. Please rebase and re-run the suite.

2. Renaming an entity_id drops the device's faults.
MyHOMEEntity.async_will_remove_from_hass treats a missing registry entry for self.entity_id as "the owner deleted the entity". On an entity_id rename, Home Assistant updates the registry first and then calls async_remove(force_remove=True) while self.entity_id still holds the old id. The lookup therefore returns None and forget_address() withdraws an issue whose fault is still present. This pattern was already in the climate entity; this PR extends it to every platform that calls the base. Home Assistant has a dedicated hook for this, async_removed_from_registry(), which runs only on a real registry removal. Note that once the hook applies to sensors too, forget_address() needs to check that no other live entity still uses the same (WHO, WHERE): today removing one of two entities on an address drops the other's issue. That only doesn't happen for sensors/binary sensors because they override async_will_remove_from_hass without calling super().

3. Issues the tracker doesn't hold can no longer clear.
clear() only deletes an issue that is in _active. Any issue in the registry that the current tracker doesn't know about can't be withdrawn by the next normal status. That covers an issue raised by a frame processed between clear_all() and the listener task being cancelled during unload, and also the standby case above. ir.async_delete_issue on a missing id is already a cheap no-op, so I'd keep the edge trigger on create (that is where the savings are) and always delete on a normal status.

4. A keypad command clears the issue.
On the bus, *1*1*74## from a push button (or from Home Assistant itself) looks the same as the actuator's status. On a faulty actuator every command withdraws the issue and the next poll raises it again (three cycles give three creates and three deletes). The bus can't tell the two apart, so I'm not asking for a different rule. I am asking that the text and the docs say "clears when an on/off/level frame for this address is seen" rather than "when the device reports a normal state".

5. Re-raise after reload only works for configured addresses.
The docs say the issue "is raised again after a restart or reload if the actuator still reports the code". initial_discovery() only queries WHO 2, 4 and 16, so for an address with no entity nothing asks again. The issue comes back only if the actuator sends a frame by itself. Please qualify the sentence, or document that a bus sweep re-raises it.

6. Hot path.
health.observe() runs inside process_message without isolation. An exception there would skip the rest of the WHO 1 handling for that frame (area/group events, resync). I'd guard it with who_int in (1, 1001) before calling _is_active_for_who(), since on a standby that call scans the config entries for every frame of every WHO. I'd also wrap it in try/except with LOGGER.exception.

Questions

  • test_a_status_outside_the_published_table_is_a_fault asserts WHAT 32 and 33 as faults. Is there a source saying they are never used, or does the test pin a behaviour that would raise issues on real plants?
  • There is no real capture of lighting-management sensors in the corpus (OWNd handles WHAT 34). Do they emit other WHATs, for example end of movement? 33 of the 63 captures feed no WHO 1/1001 frame to the tracker, so "replays all 63 captures" overstates the coverage a bit.
  • unresponsive_zone changes its placeholder from {zone} to {device}. fr/it/nl have no translation today, but if Crowdin holds strings with {zone}, the next sync would bring back an unfilled placeholder. Could you check the source string there?

Smaller things

  • The EVIDENCE_WINDOW comment says the mask "followed the status reply by about 40 ms"; in the 2026-09-26 trace it arrived 3.45 s before. The PR body says so too.
  • name_address() stores _device_name and report() stores _display_name, so the same device can be named two ways.
  • Italian strings: the rest of it.json uses the second person ("Riconfigura la password", "Controlla il cablaggio"), while the new strings use the infinitive. Please align them. Also, "un attuatore luci in guasto" reads better as "un attuatore luci guasto", and "è mostrato così come ricevuto" as "viene mostrato così come è stato ricevuto".

I ran the suite on Python 3.14 with the latest pytest-homeassistant-custom-component. The head of this branch passes (3179 passed, 1 skipped, 100% coverage); merged onto the current base, the two tests above fail. I'm requesting changes for point 1 and the lifecycle points (2, 3, and the failover case already raised). The rest can be follow-ups if you prefer.

GreenGrassBlueOcean and others added 6 commits October 2, 2026 08:56
…epair issues

When an actuator reports an anomaly status (such as WHAT 19 on WHO 1), raise a Home Assistant repair issue (actuator_hardware_fault) linking to diagnostic guidance and auto-clear it when normal status returns.
…apped WHAT copy

- Slug '#' in WHERE addresses to maintain URL-safe issue identifiers
- Edge-trigger repair issue creation and deletion on state change
- Generalize copy to unmapped lighting WHAT codes, citing F414 WHAT 19 as the known hardware case
- Point learn_more_url at the live repair issues documentation anchor
- Add localized translations in fr.json, it.json, and nl.json
- Add comprehensive edge-triggering, slugging, and non-19 tests
…vice health tracker

Replace the light-only actuator_hardware_fault repair with DeviceHealth
(device_health.py): one tracker per gateway, the only code that creates or
deletes per-device fault issues. Issues are written once per change and
withdrawn on the next normal status, entity removal or entry unload.

- Frames are classified in GatewayEventDispatcher.process_message, so lights
  and switches with no entity still raise their issue; on a shared bus only the
  owner of WHO 1 raises lighting faults (WHO 1001 follows WHO 1).
- A WHO 1 status outside the published table raises unmapped_device_status.
  A WHO 1001 DIMENSION 7/11 mask within 10 s is attached raw as evidence
  (unmapped_device_status_autodiag); its bits are undocumented, so it is never
  decoded and never a trigger on its own.
- The entity base class names the device in its issues, drops them when the
  owner deletes the entity, and offers _report_fault/_clear_fault; the climate
  unresponsive_zone issue now goes through the tracker.
- Neutral issue text in en/fr/it/nl: no F414, fuse or open-load claims the
  evidence (EVID-MH200-WHAT19-FAULT) does not support. Diagnostics list the
  active faults.
- test_device_health_corpus replays all 63 captures: only the two MH200
  address 74 captures may raise a fault; the Encyclopedia golden corpus and
  every published WHO 1 WHAT must raise none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le, evidence window)

- corpus replay parses only WHO 1/1001 frames, so other WHOs' decoders cannot break it
- a warm standby carrying an offline primary files faults under the primary's tracker
- entity removal uses async_removed_from_registry (not a rename or reload) and the
  tracker keeps issues while another entity uses the address
- a normal status always asks the registry to delete the issue
- the 10 s evidence window holds in both directions
- guard and isolate health.observe on the WHO 1/1001 hot path
- repair text and docs: what clears the issue, what re-raises it, state wording
- it/fr/nl strings aligned

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ecovery

WHAT 32 (Toggle), 34 (movement) and 39 (end of movement) are defined by the
ZigBee OpenWebNet spec 4.0 and say nothing about the on/off state; OWNd
reports 32 and 39 as unknown_state, which raised a false repair. WHAT 33 is
in no source and stays a fault. Also give unresponsive_zone a zone
placeholder, so a translation that still uses {zone} renders. Document what
to do with an actuator that will not be repaired soon.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A disabled entity's status is not requested at startup, so the repair does
not come back on every restart. Also fixes a missing space in device_health.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GreenGrassBlueOcean
GreenGrassBlueOcean force-pushed the rfc/actuator-hardware-fault-handling branch 2 times, most recently from 6804784 to bc418db Compare October 2, 2026 09:31
The failover tests only checked what health_owner() returns. Two new tests
send the fault and the recovery through the dispatchers of a primary and its
warm standby: a recovery seen by the standby during the failover, and one
seen by the primary after the failback, both delete the primary's issue.

Also calls super() in async_removed_from_registry, fixes the stale
"~40 ms" mask timing in the test docstring, and points the docs' "clears
itself" sentence at the on/off/level rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GreenGrassBlueOcean

Copy link
Copy Markdown
Contributor Author

Thanks for the review, all four blocking points are addressed as of 9e860f8a (rebased on the current v2-phase1-architecture):

  • Corpus tests on current base: rebased onto the base that carries the F455 trace (Community Call for Real-World Gateway Traces: Help Us Eliminate Hardware Blind Spots! 📡 #466); the corpus test passes for all 63 captures, F455 included. The full suite has 3200 passed and 4 failed; the 4 are alarm-decoding tests (test_component_alarm, test_issue_311, test_issue_466_h4890_trace, test_issue_564_mh202_alarm_trace) that also fail on the base.
  • Failover/failback ownership: health_owner() sends what a warm standby sees to the primary's tracker while the primary is offline, so the primary's entry owns every issue. This is now tested end to end through the dispatchers of a primary and its standby:
    • test_a_recovery_seen_by_the_standby_during_a_failover_clears_the_primarys_issue: the fault and the recovery both arrive on the standby; the issue is filed and deleted under the primary's entry, never the standby's.
    • test_a_fault_raised_during_a_failover_clears_on_the_primary_after_the_failback: after the failback the standby's frames are ignored, and the primary's own recovery frame deletes the issue.
  • Rename dropping faults: issues are dropped only in async_removed_from_registry (which now calls super()), and neither a reload nor an entity_id rename reaches it. Ownership is tracked per unique id, so an address keeps its issues while another entity still uses it.
  • clear() only deleting held issues: clear() now always deletes from the issue registry, so an issue raised before a reload still clears (test_a_normal_status_clears_an_issue_the_tracker_does_not_hold).

On the other questions:

  • WHAT 32/33/34: the ZigBee OpenWebNet spec 4.0 variant of WHO 1 defines 32 Toggle, 34 movement detected and 39 end of movement detected. These are now treated as events that neither raise nor clear a fault. WHAT 33 is in no source we hold, so it stays a fault, and the test pins that as an assumption. No capture has any of 32–39.
  • Corpus wording: corrected in the PR body to "63 captures, 30 of which carry WHO 1/1001 frames". The mask timing is corrected too: in the 2026-09-26 MH200 trace the mask arrived 3.45 s before the status.
  • Crowdin {zone}: unresponsive_zone still passes zone alongside device, so a translation that kept {zone} renders either way.
  • Old unresponsive_zone_{unique_id} repairs: no migration needed. They were never persistent, and Home Assistant loads non-persistent issues as inactive at startup, so after the restart that installs the update the old repair is no longer shown.

Docs: docs/diagnostics/repair-issues.md now has a section on an actuator that will not be repaired soon (disable the entity, don't delete it; "Ignore" on the repair is temporary). Silencing one permanently is left for a separate ignored_addresses PR.

@GreenGrassBlueOcean

Copy link
Copy Markdown
Contributor Author

Follow-up: opened #611 for permanent device silencing via the ignored_addresses gateway option.

@Interstellar0verdrive Interstellar0verdrive left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough follow-up. I re-ran everything against 9e860f8. Every point I raised is addressed, and so are @xtimmy86x's:

  • the corpus replay now filters WHO 1/1001 before parsing, and the suite passes on the head (3205 passed, 1 skipped, 100% coverage) and merged onto the current base fb3cce9 (3254 passed, 1 skipped). I don't see the 4 alarm failures you mention, either on the head or merged with the current base, so #608 has probably fixed them;
  • async_removed_from_registry() fixes the entity_id rename. My repro of HA's rename path now keeps the issue;
  • health_owner() keeps one owner across failover and failback. My failback repro now clears on the primary and leaves nothing on the standby;
  • the evidence window holds in both directions, clear() always deletes from the registry, observe() is guarded and isolated, and the per-unique-id owners keep an address's issue while another entity still uses it;
  • 32/34/39 as documented events, the {zone} placeholder, the keypad and reload caveats in the docs, and the Italian strings all look good to me.

One small regression from the naming change, not blocking. name_address() now receives _display_name from every entity, sensors included. A zone's temperature sensor shares the climate's address (4, "71"), so when it is added after the climate it renames the unresponsive_zone issue to "Heating zone Zone 71 Temperature no longer answers" (I reproduced it). The issue describes the device, so I'd pass _device_name to name_address() and keep _display_name for the entity that calls report().

Two Italian nits, also not blocking: "lo stato ricevuto non consente a Home Assistant di determinare lo stato attuale" repeats "stato" ("il valore ricevuto…" reads better), and "un pulsante a muro" is more natural as "un pulsante a parete".

Approving. The naming fix can go in here or in a follow-up, as you prefer.

@GreenGrassBlueOcean
GreenGrassBlueOcean merged commit abea209 into OpenWebNet-HA:v2-phase1-architecture Oct 3, 2026
16 checks passed
@GreenGrassBlueOcean
GreenGrassBlueOcean deleted the rfc/actuator-hardware-fault-handling branch October 3, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants