Skip to content

Stop the bool HDF5 type being closed twice when writing tables - #55

Merged
dimitrivlachos merged 3 commits into
mainfrom
fix-bool-type-double-close
Oct 8, 2026
Merged

dimitrivlachos merged 3 commits into
mainfrom
fix-bool-type-double-close

Conversation

@dimitrivlachos

Copy link
Copy Markdown
Collaborator

Writing a reflection table with a bool column prints H5Tclose(): not a datatype
at exit (#27). The file from that write is correct, so the message
looks like noise to suppress, but it is the tail end of a double close.
The other end is that a second bool column written in the same process
is dropped.

The type registry creates the bool enum type once and holds it in a
function-local static H5Type
, so it owns the id and closes it at exit.
Both write paths (file, group) wrapped the id from get_h5_native_type<T>()
in a second H5Type, which closed it as each write returned.
After the first write the registry was handing out a dead id,
so the next bool dataset failed to create, ReflectionTable::write
logged a warning and skipped the column, and at exit the registry
closed the dead id again. The other column types went through the
same wrap without consequence because H5T_NATIVE_INT and
friends are library built-ins, which HDF5 refuses to close, and
that refusal was raised inside the write path's error silencer.
The bool enum is the only type dx2 creates itself, so it was the
only one an extra close could destroy. fast-feedback-service writes
bool columns through this path, hence fixing it rather than silencing it.

In this PR the write paths borrow the id as a plain hid_t. To keep
it that way, the registry now hands out an H5TypeRef, which converts
to hid_t for HDF5 calls, and H5Cleanup has a deleted constructor
for it, so wrapping a registry type in an owning H5Type no longer
compiles.

Changes:

  • include/dx2/h5/h5write.hpp borrows the type id in both write paths
    instead of wrapping it in an owning H5Type.
  • include/dx2/h5/h5utils.hpp adds H5TypeRef and the deleted
    H5Cleanup(H5TypeRef) constructor.
  • include/dx2/h5/h5dispatch.hpp stores and returns H5TypeRef in
    the registry, so the id cannot be re-owned through either
    get_h5_native_type or get_supported_types.
  • tests/test_reflection_table.cxx adds two regression tests: the
    registry's bool type is still open after a write, and a bool column
    survives two writes in one process.
  • tests/test_reflection_table.cxx removes scratch files in the
    fixture's TearDown, so a test that fails partway through no longer
    leaves .h5 files in the working directory.

Closes #27

@jbeilstenedmands jbeilstenedmands left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried running the tests but get two failures:

The following tests FAILED:
         81 - ReflectionTableTest.WriteBooleanColumnLeavesRegistryTypeOpen (Failed) dx2tests
         82 - ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess (Failed) dx2tests

with errors like

[WARNING] No datasets found in group 'fast-feedback-service/dx2/tests/test_reflection_table.cxx:539: Failure
Value of: col_bool.has_value()
  Actual: false
Expected: true
Boolean column missing from "fast-feedback-service/dx2/build/tests/reflection_test_bool_second.h5"

[  FAILED  ] ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess (16 ms)
[----------] 1 test from ReflectionTableTest (16 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (16 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess

 1 FAILED TEST
HDF5-DIAG: Error detected in HDF5 (1.14.6) thread 1:
  #000: H5T.c line 2063 in H5Tclose(): not a datatype
    major: Invalid arguments to routine
    minor: Inappropriate type

do these tests pass for you?

@dimitrivlachos

dimitrivlachos commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator Author

Yes these all pass for me @jbeilstenedmands. That is very odd. Could you try a fresh build for me?

      Start 81: ReflectionTableTest.WriteBooleanColumnLeavesRegistryTypeOpen
81/92 Test #81: ReflectionTableTest.WriteBooleanColumnLeavesRegistryTypeOpen ............   Passed    0.02 sec
      Start 82: ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess
82/92 Test #82: ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess .................   Passed    0.03 sec

@jbeilstenedmands jbeilstenedmands left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I get build failures related to nlohmann_json on a clean build, if a clean build works for you then happy for this to be merged, given that the CI manages to build and pass the tests also.

@dimitrivlachos
dimitrivlachos merged commit ad5985b into main Oct 8, 2026
9 checks passed
@dimitrivlachos
dimitrivlachos deleted the fix-bool-type-double-close branch October 8, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HDF5 Error printed to console when saving reflection table containing bool data

2 participants