Updates for numpy v2 - #19
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request modernizes the project’s Python dependency stack (notably moving to NumPy v2) and adds two uv-runnable scripts to run the Prototype MTC model with Sharrow enabled, improving ease of execution and reproducibility via a lockfile.
Changes:
- Updated core dependencies to newer versions (including
numpy>=2,<3, newerpandera, and a newer ActivitySim via a git source override). - Added
scripts/run-small-sharrow.pyandscripts/run-large-sharrow.pyfor Sharrow-enabled runs (the large script also downloads/unpacks the full dataset). - Updated environment defaults/ignores (Python version set to 3.10; ignore
data_full/in git).
Reviewed changes
Copilot reviewed 4 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Locks updated dependency set (NumPy v2 ecosystem, ActivitySim from git, Sharrow 2.15.0, etc.) for reproducible installs. |
pyproject.toml |
Updates declared requirements (NumPy v2, newer Pandera) and configures ActivitySim to come from the git main branch via tool.uv.sources. |
scripts/run-small-sharrow.py |
New uv script to run a small-sample Sharrow-enabled model run. |
scripts/run-large-sharrow.py |
New uv script to download/unpack full data and run a large-sample Sharrow-enabled model run. |
.python-version |
Sets default local Python version to 3.10 (within project’s >=3.10,<3.12 constraint). |
.gitignore |
Ensures data_full/ is ignored by git. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+14
to
+18
| Run the MTC example model with sharrow enabled, on the small test sample. | ||
|
|
||
| The metadata in the header above allows this script to be run with `uv` without | ||
| needing to set up a separate virtual environment or install dependencies manually. | ||
| """ |
| import platformdirs | ||
| import activitysim.abm # register components # noqa: F401 | ||
| from activitysim.core import workflow | ||
| from activitysim.examples.external import download_external_example, download_asset |
Comment on lines
+57
to
+61
| # do not allow full data to enter Git repo | ||
| working_dir.joinpath(".gitignore").write_text("**\n") | ||
|
|
||
| out_dir = Path(str(__file__).replace(".py", "-output")) | ||
| out_dir.mkdir(exist_ok=True) |
Comment on lines
+51
to
+55
| sha256 = "b402506a61055e2d38621416dd9a5c7e3cf7517c0a9ae5869f6d760c03284ef3", | ||
| link = Path(platformdirs.user_cache_dir(appname="ActivitySim")) / "External-Data", | ||
| base_path=str(working_dir), | ||
| unpack = "data_full", | ||
| ) |
Configure MTC runs with shared data, explicit chunking, and adjustable sample sizes and worker counts. Ignore local benchmark outputs and the data cache link.
Enable explicit chunking for supported memory-intensive components and document usage, observed memory peaks, and limitations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR modernizes the MTC example for NumPy 2 and adds reproducible Sharrow runs using the repository's shared
pyproject.tomlanduv.lock.activitysim>=1.6,numpy>=2,<3, andpandera>=0.30. The common lock selects ActivitySim 1.6.0 and Sharrow 2.16.2.uv run --locked scripts/run-small-sharrow.pyoruv run --locked scripts/run-large-sharrow.py. The large script downloads the full dataset and runs a 500,000-household sample. Neither script uses a separate script environment or script lockfile, and the large script preserves the repository's.gitignore..gitignore, and overlay inheritance in single-process and multiprocessing configurations.Validation: all 10 tests passed locally against both ActivitySim 1.6.0 and ActivitySim main at
8ca6741c83e44e70fbfe718cb91066dece989805, including the full model checkpoint comparisons. Script smoke tests avoid downloading or simulating the large dataset.