Add NumJVM 0.6.0 fusion and opt-in CPU workers

.github/workflows/oracle-tests.yml

@@ -44,9 +44,11 @@ jobs:
           python numjvm-tests-oracle/generate_cases.py --check
           python numjvm-tests-oracle/generate_matmul_cases.py --check
           python numjvm-tests-oracle/generate_contraction_cases.py --check
+          python numjvm-tests-oracle/generate_parallel_cases.py --check
           python numjvm-tests-oracle/check_m3_1_contract.py
           python -m unittest discover -s numjvm-bench -p test_compare_expression.py
           python -m unittest discover -s numjvm-bench -p test_run_050_reference.py
+          python -m unittest discover -s numjvm-bench -p test_run_060_reference.py
 
       - name: Java tests, including the NumPy oracle replay
         run: ./gradlew test --stacktrace

README.md

@@ -14,10 +14,9 @@
 Numerical computing for the Java Virtual Machine: N-dimensional arrays with NumPy semantics, built on the Java 25
 Foreign Function & Memory API. No Python, NumPy or native libraries are needed at run time.
 
-> **Status: 0.5.0-alpha implementation candidate (M3.1).** Experimental expression
-> execution and real contractions extend the CPU APIs. Reference measurements show
-> substantial AUTO regressions in some cases; see [0.5.0 status](docs/milestones/M3-1-status.md).
-> Performance gates remain open and the API remains experimental.
+> **Status: 0.6.0-alpha implementation candidate.** Optional CPU workers and
+> generated fusion improvements are experimental. Reference performance and
+> scaling remain unverified; see [0.6.0 status](docs/milestones/M4-0.6-status.md).
 
 ## Quick start
 
@@ -144,6 +143,15 @@ Reference results show substantial AUTO slowdowns and allocation in some cases;
 see the [results](docs/benchmarks/M3-1-reference-results.md) and
 [reference protocol](docs/benchmarks/M3-1-protocol.md).
 
+## Opt-in parallel CPU execution
+
+`NJ.parallel(workers)` provides a reusable bounded worker context for selected
+contiguous float32/64 ufuncs, expressions and full-array sum. Existing calls
+remain single-threaded. Confined scopes and unsupported layouts fall back to
+the calling thread. Parallel floating-point sum uses a deterministic chunk
+order, which can produce different low bits from sequential sum. See the
+[parallel execution guide](docs/design/parallel-cpu.md).
+
 ## Building
 
 Requires JDK 25. The Gradle wrapper downloads Gradle, and the build can provision a JDK 25 toolchain if none is

ROADMAP.md

@@ -8,6 +8,7 @@
 | M2 | Optional CPU acceleration and portable matmul — **0.3.0-alpha** | Implementation candidate; release evidence pending |
 | M3        | JIT and fusion: tracing, IR, ClassFile API code generation — **0.4.0-alpha** | Merged; reference-performance evidence pending |
 | M3.1      | Fusion performance and real dot/vecdot — **0.5.0-alpha** | Implementation candidate; reference issues and open gates |
+| M3.2      | Fusion allocation and opt-in CPU workers — **0.6.0-alpha** | Implementation candidate; reference evidence pending |
 | M4        | Ecosystem: Kotlin and Scala APIs, Arrow, DLPack, notebooks, Maven Central           | Planned                                               |
 | M5        | Advanced computing: GPU and experimental backends                                   | Planned                                               |
 | M6        | Production readiness: API freeze, compatibility, 1.0                                | Planned                                               |
@@ -32,9 +33,9 @@ No multiworker scaling target applies to M2.
 ## Proposed post-M3 sequence
 
 0.5.0 adds real dot/vecdot and fusion improvements; see its
-[status and remaining gates](docs/milestones/M3-1-status.md). The proposed [0.6.0 plan](docs/milestones/M4-0.6-plan.md) prioritizes the
-observed fusion regressions, then worker pools and deterministic parallel
-reductions after an ownership design.
+[status and remaining gates](docs/milestones/M3-1-status.md). The [0.6.0 plan](docs/milestones/M4-0.6-plan.md) addresses the
+observed fusion regressions and introduces opt-in workers and deterministic
+reductions. Its reference scaling and performance gates remain open.
 Consider a separately scoped 0.7.0 for portable solve/Cholesky and optional LAPACK
 before M4 ecosystem integration. Outstanding M2 and M3 reference-performance
 evidence remains open; no publication is implied by merging implementation code.

docs/README.md

@@ -29,3 +29,5 @@
 
 - [0.5.0 reference results](benchmarks/M3-1-reference-results.md)
 - [Proposed 0.6.0 milestone](milestones/M4-0.6-plan.md)
+- [Opt-in parallel CPU execution](design/parallel-cpu.md)
+- [0.6.0 reference protocol](benchmarks/M4-reference-protocol.md)

docs/adr/ADR-010-parallel-execution.md

@@ -0,0 +1,40 @@
+# ADR-010: opt-in parallel CPU execution
+
+Status: proposed for 0.6.0.
+
+A reusable, explicitly closed execution context owns a bounded pool; no global
+pool or implicit change to existing APIs is introduced. Calls remain synchronous.
+The caller selects the worker limit; fixed, bounded chunking and deterministic
+combination order do not depend on completion order. Small, unsupported or
+worker-inaccessible arrays use the existing sequential implementation.
+
+## Ownership and safety
+
+Validate scope liveness and access on the calling thread before scheduling.
+Only submit work for segments accessible to all target workers. A confined
+`NJ.scope()` cannot be accessed by a pool worker: execute the original operation
+on the calling thread, without copying or changing result ownership. Shared and
+automatic memory may be eligible. Never assume caller accessibility implies
+worker accessibility. Follow the existing output validation and overlap rules;
+use parallel writes only for provably disjoint regions, with identical views
+allowed only when each element is read before that element is written.
+
+A concurrent shared-scope close may cause an FFM access failure. It is not
+transactional: output may be partially written. Await every submitted worker on
+success, error or interruption before returning; restore interruption status
+and propagate a documented exception. No worker may access arrays after return.
+Closing the context must prevent new operations, wait for active operations,
+and terminate the pool. Do not hold locks in a way that deadlocks nested calls;
+worker-originated nested work must run sequentially or be rejected explicitly.
+The caller must not mutate inputs or close their scope concurrently unless it
+accepts the documented failure/partial-output behavior.
+
+## Numeric and native interaction
+
+Parallel sum uses fixed logical chunks and a fixed merge order independent of
+worker count and scheduling. Reproducibility is scoped to the same algorithm,
+dtype, backend and Java environment; it does not promise identical results to
+sequential accumulation. Document NaNs, infinities and signed zero according to
+the implemented arithmetic. Do not launch BLAS from NumJVM worker tasks or
+modify process-wide native thread settings. Performance thresholds are
+provisional until the reference campaign. No multiworker speedup is guaranteed.

docs/adr/README.md

@@ -15,3 +15,4 @@ marked "Superseded by ADR-NNN" rather than deleted.
 - [ADR-007: explicit expression execution](ADR-007-expression-execution.md) (proposed)
 - [ADR-008: expression strategy and optional fused vectors](ADR-008-expression-strategy.md) (proposed)
 - [ADR-009: real dot and vecdot contractions](ADR-009-contractions.md) (proposed)
+- [ADR-010: opt-in parallel CPU execution](ADR-010-parallel-execution.md) (proposed)

docs/benchmarks/M3-1-protocol.md

@@ -43,3 +43,10 @@ allocating methods only against other allocating methods.
 The proposed 1.5× large fused-output target, 1.8× allocating regression guard,
 and small/medium AUTO budgets remain open pending valid measurements. If a gate
 fails, report it and retain the failure; do not silently lower the target.
+
+The 0.6.0 benchmark module adds `ContractionBenchmark` for float32/64 vector
+`dot` and `vecdot`, allocating and preallocated outputs. Preserve its JMH logs
+as well as JSON: setup reports the observed backend and SIMD provider. Run the
+existing `ExpressionLifecycleBenchmark` and `expressionCompileProbe` separately
+for trace, first-call, compilation and warm costs; compilation-probe results
+are single-process diagnostics, not a substitute for JMH distributions.

docs/benchmarks/M3-1-reference-results.md

@@ -43,7 +43,7 @@ that run the recorded status also listed two untracked JSON files. For 1,048,576
 elements, F-order CHAIN eager versus AUTO was 391.303 versus 3,664.833 µs for
 float32, and 3,002.260 versus 7,548.619 µs for float64. The earlier apparent
 F-order AUTO advantage therefore does not hold in these corrected cases. The
-benchmark patch has not yet been applied to this repository.
+benchmark correction is applied on the 0.6.0 branch.
 
 ## JFR and compilation
 
@@ -81,3 +81,12 @@ compilation. The separate lifecycle JMH suite also ran.
 
 See the [protocol](M3-1-protocol.md) and the proposed
 [0.6.0 plan](../milestones/M4-0.6-plan.md).
+
+## 0.6.0 starting baseline
+
+The 0.6.0 work starts from the merged 0.5.0 implementation and oracle cache
+fix. The reference timings above were measured at `4aff717` on the user's
+machine, not at the later merge: source provenance is part of the measurement.
+The clean follow-up was a focused investigation, not a complete replacement for
+the earlier dirty full campaign. Keep those datasets distinct when comparing
+future work. No reference-machine measurements have yet been made for 0.6.0.

docs/benchmarks/M4-fusion-investigation.md

@@ -0,0 +1,51 @@
+# Fused vector investigation
+
+The supplied JDK 25 recordings reproduce the 0.5.0 allocation issue on the
+reference host: float32, 1,048,576 elements, JMH GC profiler: AUTO ADD
+16,785,621 B/op and CHAIN 33,568,252 B/op versus eager approximately 1,089
+and 3,267 B/op. JFR allocation samples point to `FloatFusedKernel.run` vector
+loads, including `float[]` and `Float256Vector` objects. Samples are not counts;
+see [reference results](M3-1-reference-results.md) for provenance and limits.
+
+The loop loads vectors into a `FloatVector[]` and sends that array through the
+`FloatExpression.evaluate` interface at every vector-sized iteration. This
+can cause objects to escape scalar replacement, but the profile does not prove
+why HotSpot allocated them. Generate or otherwise specialize a whole traversal
+loop and measure both allocation and speed before concluding that a change
+fixes the cause. Confirm float64 and mixed independently. Small AUTO overhead
+occurs even when eager is selected and needs a separate investigation.
+
+`FusedAllocationBenchmark` provides a focused JMH regression matrix. Run with
+`-prof gc` and inspect `gc.alloc.rate.norm` as well as elapsed time, recording
+provider and `NUMJVM_AUTO_DECISION` from its setup. For example, pass
+`-PjmhArgs='FusedAllocationBenchmark -p size=1048576 -p layout=C -p types=FLOAT32 -p chain=ADD,CHAIN -prof gc'`
+to `:numjvm-bench:jmh` with `-PsimdBench` and the required vector JVM options.
+
+An exploratory shared-machine JDK 25 run (one fork, three one-second warmups
+and measurements, 1,048,576 float32 C-order elements) compared the old array
+based path with a generated whole-run loop. AUTO ADD allocation fell from about
+12.6 MB/op to 8.1 kB/op and CHAIN from 25.2 MB/op to 13.7 kB/op. These are
+separate runs on a shared machine, not reference-machine acceptance results.
+The generated loop keeps vector intermediates in locals, preserving a separate
+scalar-tail path; mixed graphs are handled by a separately generated typed loop. Repeat the full matrix on the reference
+machine before closing performance gates.
+
+A short exploratory JDK 25 smoke run also exercised C- and F-order float64
+and mixed CHAIN cases with the generated loops and GC profiler. Mixed
+float32-to-float64 transitions remain explicit so float32 nodes round before
+promotion. These smoke tests do not replace the reference campaign.
+
+For a single unary or binary ufunc with array inputs, the eager scheduler needs
+no temporary arrays. AUTO now checks the small-input eager path before building
+a generated traversal, and that single-ufunc case avoids opening a temporary
+scope. More complex small graphs still incur scheduler and scope costs; the
+reference-machine small-input budget has not yet been remeasured.
+
+AUTO selects eager for a single direct array ufunc with an available SIMD
+provider. Such an operation has no intermediate array for fusion to eliminate;
+the clean 0.5.0 ADD comparison also favored eager. A decision is reported as
+`single ufunc`. Scalar-provider dispatch and multi-operation thresholds remain
+unchanged. Since the kernels changed after the reference run, numerical
+threshold calibration requires new reference measurements; do not infer it
+from the older measurements. The allocation benchmark's ADD case now measures
+the selected eager path, while CHAIN exercises eligible fused execution.

docs/benchmarks/M4-performance-status.md

@@ -0,0 +1,31 @@
+# 0.6.0 fusion performance status
+
+The original reference measurements remain documented in
+[0.5.0 results](M3-1-reference-results.md). They preceded the generated-loop
+changes and cannot establish the performance of this revision. The user's
+Ryzen 9 5950X reference machine is not available in this environment.
+
+A one-fork JDK 25 shared-machine smoke comparison of C-order float32 ADD and
+CHAIN with 1,048,576 elements found substantially lower AUTO allocation after
+generating the whole vector traversal loop; see the
+[investigation](M4-fusion-investigation.md). Float64, mixed, C and F smoke
+runs also completed. These short, separate runs can identify gross regressions,
+but do not satisfy three-fork reference gates or establish stable speedups.
+AUTO now selects eager for direct single-ufunc expressions with a SIMD provider;
+it must be evaluated as selected, not relabeled as a fused ADD measurement.
+
+| Gate | Status |
+|---|---|
+| 1.5× large fused-output target | Open: rerun eligible multi-operation cases on the reference host. |
+| 1.8× allocating regression guard | Open: rerun the specified allocating cases on the reference host. |
+| Small and medium AUTO budgets | Open: the implementation changed after the old reference measurements. |
+| Real dot/vecdot performance | Open: run the new contraction benchmark with backend provenance. |
+| 0.6.0 worker scaling | Open: requires the parallel implementation and reference campaign. |
+
+Do not lower or waive the earlier targets. Record clean revision, patch status,
+JDK, SIMD/native provider and thread settings with raw JSON and logs.
+
+The [0.6.0 scaling protocol](M4-reference-protocol.md) is ready, but the
+reference campaign is outstanding. Crossover thresholds and multiworker
+speedups are not established. A short shared-machine benchmark verified the
+harness executes; it does not provide reference performance evidence.

docs/benchmarks/M4-reference-protocol.md

@@ -0,0 +1,38 @@
+# 0.6.0 reference performance protocol
+
+The reference campaign requires the user's Ryzen 9 5950X and has not been run
+in this environment. Do not treat shared-machine smoke checks as scaling or
+release evidence. Run scalar and SIMD campaigns separately on an idle machine,
+with JDK 25, Python 3 and a clean checkout of the actual candidate revision.
+Record the output of `git status --short` and retain the entire result directory.
+Each command requires a new empty output directory without spaces.
+
+```sh
+OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 python3 numjvm-bench/run_060_reference.py \
+  --mode scalar --workers 1,2,4,8,16 --output /tmp/numjvm-060-scalar
+OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 python3 numjvm-bench/run_060_reference.py \
+  --mode simd --workers 1,2,4,8,16 --output /tmp/numjvm-060-simd
+```
+
+The runner records revision, checkout status, JVM, CPU, memory, governor, SMT,
+thread settings and the exact JMH command. It uses one JMH caller thread,
+three forks, three one-second warmups and five one-second measurements. The
+benchmark's own worker count is the `workers` parameter. Each mode runs 120
+parameter/method combinations and can take an hour or more. Use `--dry-run`
+to inspect the command. A separate optional 32-worker campaign requires a
+third output directory and `--workers 32`.
+
+Preserve `manifest.json`, `jmh.log` and `results.json`. The log includes
+`NUMJVM_PARALLEL_DECISION`, provider and worker count from setup; verify the
+intended paths were actually selected. `JAVA_TOOL_OPTIONS` is recorded, so
+inspect it for private information before sharing. Native BLAS thread settings
+are not changed by NumJVM; report any external overrides. Compare the parallel
+and sequential methods only at matching parameters and report error bars,
+small-array overhead, crossover and memory-bandwidth saturation. A single
+JMH caller is distinct from the requested NumJVM worker count.
+
+The current shared-machine smoke run used one fork, no warmup and 100 ms
+measurements solely to check that ADD, CHAIN and SUM benchmarks execute at one
+and two workers. Its timings are not suitable for performance conclusions.
+The 0.5.0 fusion, allocating and AUTO targets remain open pending the
+reference reruns listed in [performance status](M4-performance-status.md).