Resolve the NXP profiler paths on use, not at import - #22608
Open
shoumikhin wants to merge 1 commit into
Open
Conversation
A small config file in the NXP test support code refuses to load on any machine that
does not have NXP's hardware tools installed. That matters because it is not only
used by the hardware tests. Shared test helpers import it, and the wheel ships those
helpers, so importing one of them from an installed wheel failed outright.
Two things caused it, both running the moment the file loads rather than when a value
is needed. It imports the eIQ Neutron SDK (the vendor toolkit that talks to the
hardware), which is not installed on an ordinary machine. It also builds a path from
`shutil.which("nsys")`, and `which` returns None when the profiler is missing, so
`pathlib.Path` then raises TypeError.
The four values that need the toolchain are now functions, resolved when something
asks for them:
nsys_path() the profiler binary
nsys_config_path() its ini file
nsys_firmware_path() the Neutron firmware, which needs the SDK
neutron_test_path() the built executor runner
PROJECT_DIR and OUTPUTS_DIR stay as they were, since neither needs the toolchain.
Both callers now call these instead of reading a constant. One of them only wanted to
know whether the profiler exists, not where it is, so that check asks `shutil` directly
and still tolerates a missing profiler rather than failing.
Nothing changes on a machine that has the toolchain: the same paths are computed
from the same inputs, with the same assertions, only later. On a machine without it,
the failure moves from import time to the first call that actually needs a path, and
says which piece is missing.
### Test plan
Installed the wheel into a clean environment with no vendor toolkit and no profiler,
then imported the file. Before this change it failed with ModuleNotFoundError. After
it, the import succeeds and the project directory still resolves. Each of the four
functions still raises there, naming what is absent.
Confirmed no module level statement in config.py needs the toolchain any more, by
walking the parsed file and looking for one.
Note the shared helpers still do not import from a wheel, for a different reason
that this change does not touch: backends/nxp/backend/neutron_converter_manager.py
requires the SDK deliberately, in production code rather than in test support.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22608
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 2300275 with merge base 7dc8641 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A small config file in the NXP test support code refuses to load on any machine that
does not have NXP's hardware tools installed. That matters because it is not only
used by the hardware tests. Shared test helpers import it, and the wheel ships those
helpers, so importing one of them from an installed wheel failed outright.
Two things caused it, both running the moment the file loads rather than when a value
is needed. It imports the eIQ Neutron SDK (the vendor toolkit that talks to the
hardware), which is not installed on an ordinary machine. It also builds a path from
shutil.which("nsys"), andwhichreturns None when the profiler is missing, sopathlib.Paththen raises TypeError.The four values that need the toolchain are now functions, resolved when something
asks for them:
nsys_path() the profiler binary
nsys_config_path() its ini file
nsys_firmware_path() the Neutron firmware, which needs the SDK
neutron_test_path() the built executor runner
PROJECT_DIR and OUTPUTS_DIR stay as they were, since neither needs the toolchain.
Both callers now call these instead of reading a constant. One of them only wanted to
know whether the profiler exists, not where it is, so that check asks
shutildirectlyand still tolerates a missing profiler rather than failing.
Nothing changes on a machine that has the toolchain: the same paths are computed
from the same inputs, with the same assertions, only later. On a machine without it,
the failure moves from import time to the first call that actually needs a path, and
says which piece is missing.
Test plan
Installed the wheel into a clean environment with no vendor toolkit and no profiler,
then imported the file. Before this change it failed with ModuleNotFoundError. After
it, the import succeeds and the project directory still resolves. Each of the four
functions still raises there, naming what is absent.
Confirmed no module level statement in config.py needs the toolchain any more, by
walking the parsed file and looking for one.
Note the shared helpers still do not import from a wheel, for a different reason
that this change does not touch: backends/nxp/backend/neutron_converter_manager.py
requires the SDK deliberately, in production code rather than in test support.
cc @robert-kalmar @JakeStevens @digantdesai @rascani