Skip to content

Accept dict inputs for Mapping[...] parameters in input type validation - #1735

Open
breken-ai wants to merge 2 commits into
apache:mainfrom
breken-ai:fix/check-input-type-abstract-generics
Open

breken-ai wants to merge 2 commits into
apache:mainfrom
breken-ai:fix/check-input-type-abstract-generics

Conversation

@breken-ai

@breken-ai breken-ai commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Driver.execute rejects a plain dict input for a parameter annotated Mapping[str, int], and rejects an OrderedDict, defaultdict or Counter input for a parameter annotated dict[str, int]. The run fails before any node executes.

from collections.abc import Mapping

def total(weights: Mapping[str, int]) -> int:
    return sum(weights.values())

dr.execute(["total"], inputs={"weights": {"a": 1, "b": 2}})
# on main:
# ValueError: 1 errors encountered:
#   Error: Type requirement mismatch. Expected weights:collections.abc.Mapping[str, int] got {'a': 1, 'b': 2}:<class 'dict'> instead.

htypes.check_input_type accepts a value for a parameterized generic only when get_origin(node_type) == type(input_value). That is an exact type match, so a subclass of the origin (OrderedDict for dict) or an instance of an abstract origin (dict for collections.abc.Mapping) falls through every branch and returns False. The unparameterized forms already work (dict accepts an OrderedDict through isinstance), so only adding the type arguments breaks it. Sequence and set generics are unaffected because they have their own branches.

Changes

  • hamilton/htypes.py: add a last branch to check_input_type. A parameterized generic whose origin is a class now accepts any isinstance(input_value, origin). It sits after the existing sequence/tuple and set/iterable branches, so their element checks are unchanged (for example, Sequence[str] still rejects [1]).
  • tests/test_type_utils.py: Mapping, MutableMapping, and dict[...] given a dict, OrderedDict, defaultdict or Counter are accepted. Mapping[...] and dict[...] given a list or an int are still rejected.
  • tests/test_hamilton_driver.py: test_driver_accepts_dict_input_for_mapping_parameter runs the example above through Builder().build().execute(...).

How I tested this

  • On unmodified main (4f4c48fe): the 6 "accepts" cases and the driver test fail (7 failed). The 3 "rejects" cases pass on main and still pass with the fix.
  • With the fix, all 10 new tests pass.
  • pytest tests/test_type_utils.py tests/test_hamilton_driver.py tests/test_end_to_end.py tests/test_async_driver.py tests/test_graph.py tests/function_modifiers tests/test_function_modifiers.py tests/lifecycle tests/test_base.py tests/execution tests/io tests/caching tests/test_default_data_quality.py tests/test_node.py tests/test_ad_hoc_utils.py tests/test_parallel_graceful.py on Python 3.12: 1192 passed, 42 failed. The same 42 fail on main without this change (1182 passed): the test_graph.py display tests need graphviz, and the test_fingerprinting.py hash tests fail in my local env. None of them touch this code.
  • tests/test_type_utils.py and tests/lifecycle/test_default.py also pass on Python 3.10 and 3.14.
  • ruff check and ruff format --check (0.16.9, the pre-commit pin) are clean. git diff --check is clean.

Notes

This is the input-validation path used by the default adapter and by the Dask, Ray and Spark adapters (do_validate_input → htypes.check_input_type). A companion PR, #1736, fixes a separate crash in htypes.check_instance.

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output) — no new functions
  • Placeholder code is flagged / future TODOs are captured in comments — none added
  • Project documentation has been updated if adding/changing functionality — no documented behavior changes

AI disclosure (per the ASF Generative Tooling guidance): this change was written with an AI coding tool (Claude Code, Claude Opus 5.5) working through the breken-ai account. The tool found the bug, wrote the fix and the tests, and wrote this description. The commit carries a Generated-by: trailer. The diff is a small original change to existing Hamilton code and includes no third-party material. The red/green runs above are real and can be re-run from the diff. If you would rather not take AI-assisted contributions here, say so and I will close this.

🤖 Generated with Claude Code

check_input_type only accepted a value for a parameterized generic when
type(value) was exactly the generic's origin. A plain dict passed as an
input for a `Mapping[str, int]` parameter, or an OrderedDict / defaultdict
/ Counter passed for `dict[str, int]`, was rejected, so Driver.execute
raised "Type requirement mismatch" before running anything.

Fall back to isinstance(value, origin) for other generics, after the
existing sequence/set branches, so their element checks are unchanged.

Generated-by: Claude Code (Claude Opus 5.5)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@skrawcz skrawcz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review, validated against the current PR head with local reproductions and tests. The mapping fix and its focused coverage are good, but I found one backwards-compatibility blocker in the broad generic fallback. Details are inline.

Comment thread hamilton/htypes.py
typing_inspect.is_generic_type(node_type)
and inspect.isclass(typing_inspect.get_origin(node_type))
and isinstance(input_value, typing_inspect.get_origin(node_type))
):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review comment (validated with local reproductions): inspect.isclass(origin) does not guarantee that isinstance(input_value, origin) is legal. A parameterized protocol whose origin is not decorated with @runtime_checkable raises TypeError here instead of returning a boolean. For example, both check_input_type(Parallelizable[int], [1]) and check_input_type(Collect[int], [1]) now raise; the base revision returned False, and a user-defined generic Protocol behaves the same way. Could this fallback safely handle origins that reject runtime checks (or be narrowed to the intended mapping-like origins), with a regression test for a non-runtime-checkable protocol?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thank you. You're right: inspect.isclass(origin) doesn't make isinstance legal for a Protocol origin without @runtime_checkable.

Fixed in 41e7a89: the fallback now catches the TypeError and returns False, the same result as before this PR. I kept the fallback general rather than narrowing it to mapping origins, so dict[str, int] given an OrderedDict/Counter still passes.

Regression test: test_check_input_type_generic_non_runtime_checkable_protocol is parametrized over Parallelizable[int], Collect[int] and a user-defined generic Protocol, each called with [1].

  • Without the guard: collect and user-protocol fail with TypeError: Instance and class checks can only be used with @runtime_checkable protocols. parallelizable also fails.
  • With the guard: 3/3 pass.

tests/test_type_utils.py, test_hamilton_driver.py, test_base.py and function_modifiers pass: 699 passed, 1 skipped. ruff check and ruff format --check are clean.

The isinstance(value, origin) fallback added for Mapping[...] inputs raised
TypeError for origins that are Protocols without @runtime_checkable, e.g.
Parallelizable[int], Collect[int] or a user-defined generic Protocol. The
previous behavior was to return False; restore it by catching the TypeError.

Generated-by: Claude Code (Claude Opus 5.5)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants