Repository navigation
Stop the bool HDF5 type being closed twice when writing tables - #55
Merged
Merged
Conversation
jbeilstenedmands
requested changes
Oct 7, 2026
jbeilstenedmands
left a comment
Contributor
There was a problem hiding this comment.
I tried running the tests but get two failures:
The following tests FAILED:
81 - ReflectionTableTest.WriteBooleanColumnLeavesRegistryTypeOpen (Failed) dx2tests
82 - ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess (Failed) dx2tests
with errors like
[WARNING] No datasets found in group 'fast-feedback-service/dx2/tests/test_reflection_table.cxx:539: Failure
Value of: col_bool.has_value()
Actual: false
Expected: true
Boolean column missing from "fast-feedback-service/dx2/build/tests/reflection_test_bool_second.h5"
[ FAILED ] ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess (16 ms)
[----------] 1 test from ReflectionTableTest (16 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (16 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ReflectionTableTest.WriteBooleanColumnTwiceInOneProcess
1 FAILED TEST
HDF5-DIAG: Error detected in HDF5 (1.14.6) thread 1:
#000: H5T.c line 2063 in H5Tclose(): not a datatype
major: Invalid arguments to routine
minor: Inappropriate type
do these tests pass for you?
Collaborator
Author
|
Yes these all pass for me @jbeilstenedmands. That is very odd. Could you try a fresh build for me? |
jbeilstenedmands
approved these changes
Oct 8, 2026
jbeilstenedmands
left a comment
Contributor
There was a problem hiding this comment.
Ok, I get build failures related to nlohmann_json on a clean build, if a clean build works for you then happy for this to be merged, given that the CI manages to build and pass the tests also.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Writing a reflection table with a bool column prints
H5Tclose(): not a datatypeat exit (#27). The file from that write is correct, so the message
looks like noise to suppress, but it is the tail end of a double close.
The other end is that a second bool column written in the same process
is dropped.
The type registry creates the bool enum type once and holds it in a
function-local
static H5Type, so it owns the id and closes it at exit.Both write paths (file, group) wrapped the id from
get_h5_native_type<T>()in a second
H5Type, which closed it as each write returned.After the first write the registry was handing out a dead id,
so the next bool dataset failed to create,
ReflectionTable::writelogged a warning and skipped the column, and at exit the registry
closed the dead id again. The other column types went through the
same wrap without consequence because
H5T_NATIVE_INTandfriends are library built-ins, which HDF5 refuses to close, and
that refusal was raised inside the write path's error silencer.
The bool enum is the only type dx2 creates itself, so it was the
only one an extra close could destroy. fast-feedback-service writes
bool columns through this path, hence fixing it rather than silencing it.
In this PR the write paths borrow the id as a plain
hid_t. To keepit that way, the registry now hands out an
H5TypeRef, which convertsto
hid_tfor HDF5 calls, andH5Cleanuphas a deleted constructorfor it, so wrapping a registry type in an owning
H5Typeno longercompiles.
Changes:
include/dx2/h5/h5write.hppborrows the type id in both write pathsinstead of wrapping it in an owning
H5Type.include/dx2/h5/h5utils.hppaddsH5TypeRefand the deletedH5Cleanup(H5TypeRef)constructor.include/dx2/h5/h5dispatch.hppstores and returnsH5TypeRefinthe registry, so the id cannot be re-owned through either
get_h5_native_typeorget_supported_types.tests/test_reflection_table.cxxadds two regression tests: theregistry's bool type is still open after a write, and a bool column
survives two writes in one process.
tests/test_reflection_table.cxxremoves scratch files in thefixture's
TearDown, so a test that fails partway through no longerleaves
.h5files in the working directory.Closes #27