Write the float32 unimplemented value as its bit pattern - #122
Conversation
SUNS_UNIMPL_FLOAT32 is 0x7fc00000, a bit pattern, but create_unimpl_value
passes it to f32_to_data, which packs it as a quantity:
struct.pack('>f', 2143289344.0) -> 4e ff 80 00
So a point recorded as unimplemented is serialised as 0x4EFF8000, which is the
perfectly ordinary number 2143289344.0. A reader cannot tell it from a
measurement, and a device file with an absent value no longer round-trips:
get_mb() writes that number and reading it back yields 2143289344.0 rather than
None.
data_to_f32 already maps the NaN pattern to None, so the reader has always been
correct and only the writer disagreed with it. Packing the bit pattern lets the
two meet, and makes float32 consistent with every other type in
create_unimpl_value, all of which already emit their sentinel's bits.
test_create_unimpl_value pinned the old bytes, so that assertion changes with
the behaviour. Two tests are added: one that the bytes are the sentinel pattern
and specifically not the quantity, and one that an unimplemented float32 now
round-trips back to None.
Kudrat9
left a comment
There was a problem hiding this comment.
Verified by cloning at 03aaa7f with the sunspec2/models submodule initialized and running CI-style pytest . per branch: master 234 passed, this branch 236, zero failures. Reverting the change while keeping the tests fails all three affected tests, including the amended test_create_unimpl_value.
The encoding is correct: 4 bytes, matching point_type_info[float32].len of 2 registers, and data_to_f32 maps the result to None. Read-side compatibility is preserved, which is worth stating explicitly in the description since it is the first question a reviewer will have: register images written by earlier versions still decode as unimplemented, because data_to_f32(b'N\xff\x80\x00') returns 2143289344.0 and is_impl_float32 rejects that value.
Should fix:
-
The description overstates the defect, and it will become the merge commit message. "A device file with an absent
float32no longer round-trips" does not hold at the library level. The old encoding decodes to exactly2143289344.0, which equalsSUNS_UNIMPL_FLOAT32, sois_impl_float32returnsFalseandset_mbsets the point toNone. Checked at the device level on both master and this change:master: unimpl float32 bytes: 4eff8000 -> data_to_f32: 2143289344.0 -> after set_mb, value = None this branch: unimpl float32 bytes: 7fc00000 -> data_to_f32: None -> after set_mb, value = NoneThe genuine defect is narrower and still worth fixing: the bytes on the wire are wrong for any spec-conformant external reader, which is exactly the case the description opens with. Suggest reframing around on-wire conformance and dropping the round-trip claim.
Questions:
is_impl_float32compares the decoded float against the integerSUNS_UNIMPL_FLOAT32(2143289344) rather than testing for NaN. After this change nothing reaches it with that value by way ofdata_to_f32, so the comparison becomes vestigial. Is folding a NaN check into this PR preferable, or is leaving it for a separate change intended?
|
No code change requested here, so nothing has been pushed to the branch. The description has been edited (maintainer edit): the round-trip sentence is narrowed to the encoding, and a marked note at the bottom records why, along with the read-compatibility point. Revert that block if you would rather phrase it yourself. The |
sunspec#121 and sunspec#122 landed on master. Both added tests to TestPoint at the same anchor as this branch, which is the only conflict; keep both sets, with the eui48 case from sunspec#121 first.
SUNS_UNIMPL_FLOAT32is0x7fc00000— a bit pattern, the canonical quiet NaN. Butcreate_unimpl_valuehands it tof32_to_data, which packs it as a quantity:0x4EFF8000isstruct.pack('>f', 2143289344.0). So a point recorded as unimplemented is serialised as the perfectly ordinary number 2143289344.0, which a reader cannot distinguish from a measurement.Consequence
The bytes on the wire are wrong for any reader that decodes them per the spec. Model 113 is a good example, since its temperatures are
float32and are frequently unimplemented:I met this comparing an independent decoder against this library across a set of captured devices: every unimplemented
float32came back as 2143289344.0 from the serialised registers whileget_dict()reportedNonefor the same point.The reader was always right
data_to_f32already maps the NaN pattern toNone:So this is the writer disagreeing with the reader, not an ambiguity about what the sentinel means. Packing the bit pattern lets the two meet.
Consistency
Every other type in
create_unimpl_valuealready emits its sentinel's bits —int16→80 00,uint16→ff ff,acc32→00 00 00 00, and so on.float32is the only one that encodes the sentinel as a value first. The change brings it into line:A changed assertion
test_create_unimpl_valuepinned the old bytes:Flagging that explicitly since it is an existing test changing rather than a new one passing. Two tests are added alongside it: that the bytes are the sentinel pattern and specifically not the quantity, and that an unimplemented
float32now round-trips back toNone. Both fail without the change.Test results
masterThree test modules are excluded from both runs (
test_modbus_client.py,test_modbus_modbus.py,test_tls_client.py). That is a fault in my environment, not this repository: I have an unrelated PyPI package namedserialshadowingpyserial, and it pulls infuture, which doesimport imp— removed in Python 3.12 — so those three fail at collection.Related, not changed here
create_unimpl_value('float64')raisesstruct.error:SUNS_UNIMPL_FLOAT64exists butfloat64has no entry inunimpl_value, soNonereachesf64_to_data. No bundled model usesfloat64, so I have left it alone rather than widen this change, but it looks like the same root cause and you may want it.Maintainer note (edited into this description): the sentence above originally read "a device file with an absent
float32no longer round-trips". That claim is too broad, so it has been narrowed to the encoding. The library's own round-trip was never broken:data_to_f32(b'N\xff\x80\x00')returns2143289344.0, which is exactlySUNS_UNIMPL_FLOAT32, sois_impl_float32rejects it andset_mbsets the point toNone. Checked at the device level on both sides of the change:The defect is confined to the encoding, which is still worth fixing for exactly the reason the description opens with: an external, spec-conformant reader sees a measurement where there is none.
The same mechanism means read compatibility is preserved by this change, which is the question a reviewer is most likely to ask: register images written by earlier versions still decode as unimplemented.
Revert this block if you would rather state it differently.