Skip to content

Readable names for unlisted values, Connected variable & rate-limit recovery (builds 25–30) - #20

Merged
TillBrede merged 9 commits into
symcon:masterfrom
bumaas:pr/readable-names-rate-limit-recovery
Oct 8, 2026
Merged

TillBrede merged 9 commits into
symcon:masterfrom
bumaas:pr/readable-names-rate-limit-recovery

Conversation

@bumaas

@bumaas bumaas commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

This PR builds on the current master (build 24) and adds builds 25–30. It is the delta only; the tree matches our tested fork state. All 76 PHPUnit tests pass.

Motivation

Field reports showed raw API keys or "N/A" where a readable name belongs: option values and programs chosen at the appliance were missing from their profiles (the profiles hold only what the API lists as selectable), and the event profile was created once and never extended. Users also asked to tell a switched-off appliance from an offline one. A code review of the rate-limit path then found that the event stream could stay dead after a block: a restart during the block left the IO switched off for good, and the reconnect after a 401 re-sent the token the server had just rejected.

Changes

Device (Home Connect Device)

  • Readable names for values missing from a profile (builds 25, 27, 28): option values, events and programs chosen at the appliance get a missing profile association added when they arrive — known keys with their translated name, unknown ones with a readable name from the key (DelicatesSilk → "Delicates Silk"). The selection still offers exactly the values of the selected program. SaltNearlyEmpty added to the dishwasher events.
  • Connected variable (build 26): follows the DISCONNECTED/CONNECTED events and the /status answer the refresh requests anyway (SDK.Error.HomeAppliance.Connection.Initialization.Failed = offline). No additional request; OperationState and instance status are unchanged.
  • Throttled initialization is retried (build 30): an init skipped by the 30 s refresh throttle (e.g. appliance offline at setup, online right after) arms a RetryRefresh timer for the rest of the window instead of being dropped.
  • Numeric profile type (build 30): a variable follows an existing profile of the other numeric type (INTEGER vs. FLOAT) instead of being rejected.

Cloud (Home Connect Cloud)

  • Recovery after a rate-limit block (builds 29, 30): the keep-alive watchdog lifts an expired block whose timer was lost to a restart and re-activates the IO; if the stream cannot be resumed (no token), the block stays pending and is retried. The instance reports IS_ACTIVE only after the IO is running again, so children no longer see an inactive IO and stay inactive.
  • Single stream connect (build 30): one path fetches the token first and applies the IO once (previously twice, the first time with the old header); without a token the IO is left off, and it is no longer switched on while a block is active.
  • Rejected token is dropped (build 29): after a 401 invalid_token from the stream the cached access token is cleared, so the reconnect fetches a new one.
  • Quiet watchdog (build 30): no echo from the timer when the IO is switched off on purpose or no login exists — each one ended up as a log warning.

Configurator (Home Connect Configurator)

  • Lists only device instances of its own cloud instance (build 30); devices of another cloud were offered as deletable rows.

Tests

PHPUnit suite extended from 52 → 76 tests, with fixtures from real captures (coffee maker, offline washer, dishwasher, washer dryer). Each fix was checked red/green: the new tests fail on the code without the fix and pass with it. The fixtures added in builds 26–28 are reformatted for json-check.php; their content is unchanged. All green.

🤖 Generated with Claude Code

bumaas and others added 7 commits October 4, 2026 14:29
Option profiles (HomeConnect.<DeviceType>.Option.<Ident>) are shared by all
programs of a device type and hold the value list of the last loaded
program only, since that list also feeds the selection. A value reported
for another program (e.g. operated at the appliance, or before the program
was ever selected) had no association and showed the raw API key
(forum t/124612 #554/#556).

ensureOptionAssociation() now adds a missing association before an option
value is set (event, program refresh, buffered values): program keys such
as a favorite's BaseProgram reuse the name from the Programs profile, any
other value gets its last key snippet. The profile is still replaced on
the next program selection, so the selection keeps offering exactly the
values of the selected program.

Tests: 2 new regression tests in HomeConnectCoffeeTest based on the real
coffee maker responses. Before the fix 2 of 54 tests fail, after it all
54 pass (589 assertions). Also fixes the formatValue test helper, which
indexed profile associations with gaps.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every device instance gets a "Connected" variable (Yes/No), so a
visualization can tell a switched-off appliance (OperationState
Inactive, still connected) from one Home Connect reports as offline.

It follows the DISCONNECTED/CONNECTED events and is refreshed from the
/status request the refresh performs anyway: the error key
SDK.Error.HomeAppliance.Connection.Initialization.Failed ("HomeAppliance
is offline") means not connected, a regular answer means connected,
other errors (e.g. 429) leave the value unchanged. No additional request.
OperationState handling and instance status are unchanged.

Fixture is a real capture of an offline washer. Before the change 7 new
checks failed, after it 58 tests / 599 assertions pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The event profile (HomeConnect.Event.<DeviceType>) is created only once,
so events the module did not know showed "N/A" (Dishcare.Dishwasher.Event.
SaltNearlyEmpty, reported by pitti), and associations added in later
builds never reached existing installations (a dishwasher profile on a
live system lacked RinseAidNearlyEmpty).

ensureEventAssociation() now adds a missing association when an event
arrives: known events get their translated name, unknown ones a readable
name from the key (SomeEvent -> "Some Event"). The event list per device
type moved to getEventAssociations(), shared by profile creation and the
new check. SaltNearlyEmpty added ("Bitte Salz nachfüllen").

Tests: HomeConnectDishwasherEventTest with the reported event line and
real dishwasher captures. Before the fix 3 of 61 tests fail, after it
61 tests / 603 assertions pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API lists only the selectable programs (three on a WNC254A40 washer
dryer), but programs chosen at the appliance are reported via
SelectedProgram as well (about 25 according to forum t/124612 #572).
Without a profile association the Program variable showed the raw key.

SelectedProgram events and the selected-program refresh now add a
missing association via ensureProgramAssociation(). createPrograms()
re-adds the currently selected program after rebuilding the profile, so
it survives ApplyChanges. Keys without a known name get a readable name
from the key (DelicatesSilk -> "Delicates Silk"), shared with the event
profile via getReadableName().

Tests: HomeConnectWasherDryerTest with fixtures taken unchanged from the
user's debug dump (t/124612 #570). Before the fix 3 of 65 tests fail,
after it 65 tests / 610 assertions pass.

# doku:ok

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three findings from the code review of the rate-limit path:

- A restart during a block reset the RateLimit timer, while RateLimitUntil
  and the IO deactivated for the block persisted. Nothing re-activated the
  IO, so the event stream stayed dead until the user switched it on by
  hand. The keep-alive watchdog now lifts an expired block itself; an IO
  the user switched off (no block pending) is left alone.
- A 401 "invalid_token" from the stream reconnected with the cached access
  token the server had just rejected (still valid by its local expiry),
  costing a GET /events per attempt. The cached token is now dropped.
- ResetRateLimit reported IS_ACTIVE before the IO was re-activated.
  Children answering with HasActiveParent() could see the IO inactive, go
  inactive and, due to their LastParentStatus guard, never retry. The
  status is now set after the stream is registered.

testInvalidTokenReconnectsEventStream relied on the reused token and now
asserts that a fresh one is requested.

Red/green: without the fix 4 of 69 tests fail, with it all 69 pass
(621 assertions).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nect

Further findings from the code review, verified against the code:

- A device initialization skipped by the 30 s refresh throttle was never
  retried (e.g. an appliance offline at setup that comes online right
  after). A throttled init now arms the RetryRefresh timer for the rest of
  the window; a throttled value refresh still needs no retry.
- ForceRegisterServerEvents applied the IO twice, the first time with the
  old header, and switched it on even when no token could be fetched. One
  path (connectEventStream) now fetches the token first and applies once.
  It also no longer switches the IO on while a block is active.
  ResetRateLimit keeps an expired block pending if the stream cannot be
  resumed, so the watchdog retries instead of leaving the IO off.
- The keep-alive watchdog echoed "IO instance is not active" or "login is
  missing" from its timer, a log warning per attempt. It now stays silent
  when the IO is switched off on purpose or no login exists; the stream
  token refresh after invalid_token no longer echoes either.
- A numeric profile of the other numeric type (INTEGER vs. FLOAT) passed
  the type guard, and the variable was rejected. The variable now follows
  the existing profile's type.
- The configurator listed device instances of other cloud instances as
  deletable rows. It now lists only devices of its own cloud (or of none).

Adjusted tests: the watchdog tests now model a registered cloud (refresh
token present); the invalid_token test expects no output from ReceiveData.

Red/green: without the fix 9 of 76 tests fail, with it all 76 pass
(638 assertions).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixtures added in builds 26-28 were indented with two spaces; the
Check Style workflow (.style/json-check.php) expects PHP's pretty print
with four. Reformatted with `php .style/json-check.php fix`; the decoded
content of all 14 files is unchanged, the test suite stays green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@TillBrede TillBrede left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! I went through the whole diff (builds 25–30). Overall it's in good shape: every fix comes with a test that fails without it, the commit messages explain why each change was made, and the fixtures are real captures. I found one regression worth fixing before merge and a few small points. I didn't run the suite locally; CI is green.

Worth fixing: a pending block can leave the cloud stuck at status 201

Build 30 changes ResetRateLimit() so that the block stays pending when the stream can't be resumed. That happens when no token can be fetched: login missing or revoked, or the OAuth server is down. In that case RateLimitUntil stays set and the status stays STATUS_RATE_LIMITED.

The problem is what happens next:

  1. The token fetch fails because the login expired. The user sees this and re-registers.
  2. ProcessOAuthData calls ForceRegisterServerEvents, which calls connectEventStream(true, true). The stream comes back up.
  3. Nothing clears the block state. ForceRegisterServerEvents doesn't touch RateLimitUntil or the status.
  4. The watchdog only calls ResetRateLimit() when the keep-alive is stale. Keep-alives now arrive every 55 s, so that never happens.

Result: the event stream works, but the cloud stays at 201 and the device instances stay inactive. The form keeps showing "blocked until <a time in the past>". It only recovers when some REST request happens to succeed (a user action, a SelectedProgram event, opening the configurator). Before build 30 this couldn't happen, because the block was reset unconditionally.

Possible fix: split ResetRateLimit() into two steps, "connect" and "clear block + SetStatus(IS_ACTIVE)". Then run the second step after every successful connectEventStream() while RateLimitUntil !== 0 && !isRateLimitActive(). That covers ForceRegisterServerEvents, RegisterServerEvents and the 401 path. A test would be: block pending, then ForceRegisterServerEvents() with a valid token, then assert the status is IS_ACTIVE and RateLimitUntil is 0.

Minor points

  • Numeric profile type (createVariableFromConstraints): the variable now takes the type of an existing profile. When the existing profile is INTEGER and the API reports FLOAT, values like 4.5 °C get cut to 4. That's still better than the variable not being created at all, which is what happened before. createStates already avoids the clash by giving float profiles a .f suffix, and options/settings could do the same.
  • RetryRefresh can be lost: refreshDeviceState() clears the timer on every call. A throttled value refresh ($initializeDevice = false) in the same window clears it and doesn't re-arm it. In practice the next IS_ACTIVE status change triggers the init again, so the fix is just for robustness: only clear the timer when a refresh actually runs, or re-arm it when needsInitialization() is true.
  • Misleading debug label: the RetryRefresh timer goes through the RefreshDeviceState action, so a retry shows up in the debug log as Event:CONNECTED (deferred).
  • Stale comment in handleExpiredTokenFromStream: it still says "FetchAccessToken() in RegisterServerEvents"; that call is now in connectEventStream.
  • getReadableName() and digits: keys with digits followed by capitals split oddly, e.g. HotAir3D becomes "Hot Air3 D". Cosmetic.

Looks good

  • connectEventStream(): one code path, the token is fetched before anything is applied, the IO is applied once, and timer calls produce no echo.
  • ForceRegisterServerEvents no longer switches the IO on during a block. Before, that cost a GET /events that came back as 429.
  • After a 401, the rejected token is dropped, and IS_ACTIVE is only set once the IO is running again.
  • The watchdog clears a block that expired across a restart, and it leaves an IO alone when the user switched it off.
  • The Connected variable makes no extra requests and ignores 429 and other unrelated errors.
  • The event, option and program associations are added only when needed, so existing installations get them too, and rebuilding the profiles still removes extra values from the selection.
  • The configurator now filters devices by their own cloud. Devices with no cloud are still listed.

… follow-ups

Upstream review of PR symcon#20:

- Since build 30 an expired rate-limit block stayed pending when ResetRateLimit()
  could not fetch a token. A new login then re-registered the stream through
  ForceRegisterServerEvents(), but nothing cleared the block: the stream ran while
  the cloud stayed at status 201 and the devices inactive, until some REST request
  happened to succeed. connectEventStream() now lifts an expired block as soon as
  the stream runs again (liftRateLimit), whoever registered it: reset timer,
  watchdog, new login or 401 recovery. A block that is still running is lifted by
  ResetRateLimit() after a successful request, as before.
- refreshDeviceState() cleared the RetryRefresh timer on every call, so a throttled
  value refresh dropped a pending initialization retry. The timer is now cleared
  only by a refresh that actually runs and does not leave the initialization due.
- The retry ran through the CONNECTED action and showed up as
  "Event:CONNECTED (deferred)" in the debug log; it has its own action
  RetryRefreshDeviceState now.
- getReadableName(): digits form a word of their own, a capital right after a digit
  belongs to it (IDos1BaseLevel -> "IDos 1 Base Level", HotAir3D -> "Hot Air 3D").
- Stale comment in handleExpiredTokenFromStream.

Tests: the 4 new tests fail before the fix (81 tests, 4 failures), 81 tests /
653 assertions green after.
bumaas added a commit to bumaas/HomeConnect that referenced this pull request Oct 8, 2026
… follow-ups

Upstream review of PR symcon#20:

- Since build 30 an expired rate-limit block stayed pending when ResetRateLimit()
  could not fetch a token. A new login then re-registered the stream through
  ForceRegisterServerEvents(), but nothing cleared the block: the stream ran while
  the cloud stayed at status 201 and the devices inactive, until some REST request
  happened to succeed. connectEventStream() now lifts an expired block as soon as
  the stream runs again (liftRateLimit), whoever registered it: reset timer,
  watchdog, new login or 401 recovery. A block that is still running is lifted by
  ResetRateLimit() after a successful request, as before.
- refreshDeviceState() cleared the RetryRefresh timer on every call, so a throttled
  value refresh dropped a pending initialization retry. The timer is now cleared
  only by a refresh that actually runs and does not leave the initialization due.
- The retry ran through the CONNECTED action and showed up as
  "Event:CONNECTED (deferred)" in the debug log; it has its own action
  RetryRefreshDeviceState now.
- getReadableName(): digits form a word of their own, a capital right after a digit
  belongs to it (IDos1BaseLevel -> "IDos 1 Base Level", HotAir3D -> "Hot Air 3D").
- Stale comment in handleExpiredTokenFromStream.

Tests: the 4 new tests fail before the fix (81 tests, 4 failures), 81 tests /
653 assertions green after.
…ofile

Upstream review of PR symcon#20, last point: an option or setting profile that already
exists as INTEGER while the API reports a Double made the variable an INTEGER too
(build 30), so a value like 4.5 °C was cut to 4. The lossy direction now gets a
float profile of its own (".f" suffix, as createStates does) and a float variable;
the existing profile is left alone. The other direction (FLOAT profile, Int
setting) is lossless and keeps following the profile.

Fixture: real capture of the oven HM778GMB1 (programs/available and the HotAir
options, Cooking.Oven.Option.SetpointTemperature is a Double).

Tests: the new test fails before the fix (82 tests, 1 failure), 82 tests /
663 assertions green after.
bumaas added a commit to bumaas/HomeConnect that referenced this pull request Oct 8, 2026
…ofile

Upstream review of PR symcon#20, last point: an option or setting profile that already
exists as INTEGER while the API reports a Double made the variable an INTEGER too
(build 30), so a value like 4.5 °C was cut to 4. The lossy direction now gets a
float profile of its own (".f" suffix, as createStates does) and a float variable;
the existing profile is left alone. The other direction (FLOAT profile, Int
setting) is lossless and keeps following the profile.

Fixture: real capture of the oven HM778GMB1 (programs/available and the HotAir
options, Cooking.Oven.Option.SetpointTemperature is a Double).

Tests: the new test fails before the fix (82 tests, 1 failure), 82 tests /
663 assertions green after.

@TillBrede TillBrede left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick follow-up. Build 31 addresses all points, approving.

One small thing for a follow-up: RetryRefresh is a repeating timer, and since build 31 it is only cleared inside the active-parent branch of refreshDeviceState(). If the cloud goes inactive while a retry is pending (e.g. a rate-limit block), the timer keeps firing every 1–30 s until the cloud is active again. That costs no API requests, but a script run and a debug line each time. Adding $this->SetTimerInterval('RetryRefresh', 0); before setInstanceStatus(IS_INACTIVE) would stop it; the status change back to IS_ACTIVE starts the initialization again anyway.

@TillBrede
TillBrede merged commit aaa0cb8 into symcon:master Oct 8, 2026
4 checks passed
@bumaas

bumaas commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review. Builds 31 and 32 (ee6ca57) are on the branch:

  • Pending block: connectEventStream() now lifts an expired block as soon as the stream runs again (liftRateLimit()), whichever path registered it: reset timer, watchdog, new login or 401 recovery. A block that is still running is lifted by ResetRateLimit() after a successful request, as before. Test as you suggested (testReRegistrationLiftsPendingBlock), plus the counterpart that a running block is kept.
  • Numeric profile type: the lossy direction only (existing INTEGER profile, API reports a Double) gets a float profile of its own with the .f suffix and a float variable; the existing profile is left alone. A FLOAT profile with an Int setting stays as it is. Fixture is a real capture of my oven, Cooking.Oven.Option.SetpointTemperature is a Double there.
  • RetryRefresh: the timer is only cleared by a refresh that actually runs and does not leave the initialization due. A throttled value refresh leaves it alone (testThrottledValueRefreshKeepsPendingRetry).
  • The retry has its own action RetryRefreshDeviceState and shows up as trigger: RetryRefresh in the debug log.
  • getReadableName(): IDos1BaseLevel gives "IDos 1 Base Level", HotAir3D gives "Hot Air 3D", Eco50 gives "Eco 50".
  • Comment in handleExpiredTokenFromStream fixed.

82 tests / 663 assertions green.

bumaas added a commit to bumaas/HomeConnect that referenced this pull request Oct 8, 2026
Follow-up from the review of PR symcon#20: RetryRefresh is a repeating timer. Since
build 31 it was only cleared inside the active-parent branch of
refreshDeviceState(), so if the cloud went inactive while a retry was pending
(e.g. a rate-limit block) the timer kept firing every 1-30 s until the cloud was
active again: no API request, but a script run and a debug line each time. An
inactive parent now stops the timer; the status change back to IS_ACTIVE starts
the initialization again anyway.

Tests: the new test fails before the fix (83 tests, 1 failure), 83 tests /
667 assertions green after.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants