Keep missing Event timestamps as None; make Event copyable and picklable - #1247
Merged
jhnwu3 merged 1 commit intoSep 27, 2026
Merged
Conversation
Two bugs in pyhealth.data.Event: 1. Event.__init__ replaced a missing timestamp with datetime.now(). Tables without a time column (MIMIC-III/IV `patients`, the eICU tables) have a null timestamp, so every Event built from them via get_events() got an invented, ever-changing time that disagreed with the DataFrame view (`return_df=True` shows null). Because MIMIC dates are shifted to the 2100s, the invented time also sorts before every real event, silently passing time-window filters. The timestamp is now None when missing. This is a behaviour change for code that relied on the now() default. 2. Event.__getattr__ read self.attr_dict. copy/pickle create the object without __init__ and probe for dunder methods before attr_dict exists, so pickle.loads, copy.copy and copy.deepcopy recursed until RecursionError. pickle.dumps succeeded, so a task sample holding an Event was written fine and crashed when read back. __getattr__ now rejects dunder names and reads attr_dict through __dict__. - tests/core/test_event.py: missing/explicit timestamps, from_dict and get_events agreeing with the DataFrame, pickle/copy/deepcopy round trips, attribute access, and an Event inside a sample dataset. - docs: document timeless events and copy/pickle support. - examples/event_timestamps_mimic3demo.py on the bundled demo data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jhnwu3
approved these changes
Sep 27, 2026
jhnwu3
left a comment
Collaborator
There was a problem hiding this comment.
That's a good catch. Thanks professor.
LGTM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This fixes two bugs in
pyhealth.data.Event.datetime.now().Eventcould not be copied or unpickled (RecursionError).1. Missing timestamps became "now"
Some tables have no time column, for example MIMIC-III/IV
patientsand all the eICU tables. Their rows have anulltimestamp, butEvent.__init__did:So every
Eventthatget_events()built from those tables got an invented time. On the bundled MIMIC-III demo, patient 10006:Why this matters:
null; theEventshows a date.e.timestamp <= admission.timestampwrongly includes demographic rows.Event.timestampfor this reason; see the comment intests/core/test_fhir_dataset.py.Fix: the timestamp stays
Nonewhen missing. I found nothing in the library that reads timestamps from these tables (the eICU tasks filter by stay ID). The full test suite passes unchanged, including the FHIR tests.2.
Eventcould not be copied or unpickledEvent.__getattr__readself.attr_dict.copyandpicklerebuild the object without calling__init__, then look up special methods such as__setstate__beforeattr_dictexists. That lookup lands back in__getattr__, which recurses forever:Because
dumpsworks, a task sample that contains anEvent(e.g. in a"raw"field) is written fine and then fails withRecursionErrorwhen PyHealth reads it back. I reproduced this withcreate_sample_dataset.Fix:
__getattr__now rejects dunder names and readsattr_dictviaself.__dict__. Attribute access (event.hadm_id,event["hadm_id"],"hadm_id" in event) is unchanged, and a missing field still raisesAttributeError.Changes
pyhealth/data/data.py: both fixes.timestampis now typeddatetime | None, and the docstring has>>>examples.tests/core/test_event.py(new, 8 tests):from_dictandget_eventsagree with the DataFrame and are stable across callsEventstored inside a sample datasetdocs/api/data/pyhealth.data.Event.rst: a new "Events without a time" section, with the migration note and copy/pickle support.examples/event_timestamps_mimic3demo.py: runs on the bundled demo data. It reads a timeless event, filters to timed events, and does a pickle round trip.Verification
RecursionError); all 8 pass after.Ran 1335 tests … OK (skipped=76).tools/check_pr_rules.py: all rules pass.Out of scope
hash(Event)raisesunhashable type: 'dict', because the frozen dataclass hashes theattr_dictdict. Nothing relies on hashing events.datasets/mimicextract.pybuildsEvents with the 1.x signature and noevent_type. That is already broken and is covered by PyHealth 2.0: several modules still target the 1.x dataset API #1201.🤖 Generated with Claude Code