Arm backend: Stop re-creating Ethos-U driver objects per inference on Linux

[vacu9708]

Problem

On the Cortex-A path, the same driver setup is redone on every inference.

sequenceDiagram
    participant ET as EthosUBackend
    participant CA as Cortex-A platform
    participant K as Kernel driver
    ET->>CA: init()
    loop every execute()
        ET->>CA: execute(inputs)
        CA->>K: create network, weight, intermediate, IFM/OFM buffers<br/>(copies command stream and weights)
        CA->>K: write inputs
        CA->>K: run inference
        K-->>CA: outputs
        CA->>K: free all of them
        Note over K: allocated and freed on every inference
        CA-->>ET: outputs
    end

Loading

Changes

sequenceDiagram
    participant ET as EthosUBackend
    participant CA as Cortex-A platform
    participant K as Kernel driver
    ET->>CA: init()
    CA->>K: create network, weight, intermediate, IFM/OFM buffers<br/>(copies command stream and weights)
    Note over K: allocated once, freed in destroy()
    loop every execute()
        ET->>CA: execute(inputs)
        CA->>K: write inputs
        CA->>K: run inference
        K-->>CA: outputs
        CA-->>ET: outputs
    end
    ET->>CA: destroy()
    CA->>K: free all of them

Loading

This commit creates the driver objects once in platform_init() and keeps them in PlatformState for the lifetime of the loaded method.

invoke_linux_driver() now only writes the inputs, runs the inference, and reads the outputs.

Piece	Change
PlatformState	Holds the driver objects: network, weight buffer, intermediate buffer, IFM and OFM buffers
invoke_linux_driver()	Setup removed; takes const PlatformState& and reuses its objects on each inference
create_driver_objects()	The setup part moved out of invoke_linux_driver()

Measurement

These results were measured on the Corstone-1000 A320 FVP (Ethos-U85) using the CPU time report from #22596.
Additionally, I would like to ask if PR #22596, which I used to benchmark this PR, is mergeable as-is.

Model	execute() CPU time per inference, before -> after	Saved
ResNet8	2.60 -> 0.93 ms	64%
DS-CNN	1.99 -> 0.93 ms	53%
MobileNetV1 0.25	4.35 -> 1.01 ms	77%
DeepAutoEncoder	3.97 -> 0.93 ms	77%

• Outputs byte-identical and NPU cycle counter unchanged for all four models.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22599

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 15 Awaiting Approval

As of commit 820f84a with merge base c9eee94 ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build documentation (gh)
• Build Presets (gh)
• Cadence Build & Test (gh)
• Lint (gh)
• periodic (gh)
• pull (gh)
• Test ARM Backend (gh)
• Test CoreML Backend (gh)
• Test Cortex-M Backend (gh)
• Test NXP Backend (gh)
• Test QNN Backend (gh)
• Test QNN Windows MSVC build (gh)
• Test Vulkan Backend (gh)
• Test WebGPU Backend (gh)
• Test XNNPACK Backend (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[pytorch-bot]

The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:

• ciflow/trunk

Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.