Perf/verifier per air work - #989
Closed
jotabulacios wants to merge 3 commits into
Closed
jotabulacios wants to merge 3 commits into
jotabulacios wants to merge 3 commits into
Conversation
…pying them. An AirWithBuses held the interaction list twice — once in auxiliary_trace_build_data and again inside the LogUpLayout built from it — and Clone copied both, so every clone duplicated the same data twice over. The in-VM verifier clones one AIR per table per epoch, which turns that into real work: profiling the recursion guest put clone+drop of Vec<BusInteraction> at 9.2% of its cycles without computing anything. Both now sit behind an Arc, so a clone bumps two refcounts. This is the pattern the struct already uses one field up for the captured constraint IR, and for the same reason. Nothing else changes: every use of these two fields is a read, so Arc's Deref covers them, and no construction site had to move. Measured on the recursion guest verifying a real mainnet block (30 epochs), same input blob on both sides: 1,546,877,064 -> 1,485,030,675 cycles, -4.00%. The guest's keccak-permutation count is identical on both sides (2,009,340), so the verifier did the same work.
…onstraint. evaluate_zerofier computed that inverse itself, but the caller evaluates every constraint of an AIR at the same z and trace_length, so the value is identical for all of them — the loop was redoing one extension-field pow and one extension-field inv per constraint, hundreds of times per table, always with the same result. The constraint-dependent half (the end-exemptions product) is the only part that had to be inside. zerofier_base_inv now computes the shared half and the callers hoist it out of the loop. Both call sites had the identical loop, so both get it: the verifier's OOD transition evaluation and the prover's. Measured on the recursion guest verifying a real mainnet block (30 epochs), same input blob on both sides: 1,485,030,675 -> 1,311,599,244 cycles, -11.21%. Combined with the preceding commit that is -15.21% off main, or 235,277,820 fewer cycles — and since the outer trace carries one CPU row per guest cycle, the same number of rows off whatever has to prove that verifier. Identical keccak-permutation count on both sides (2,009,340). The win is specific to the in-VM verifier. Verifying the same 1.6 GB continuation proof natively shows no measurable change: three interleaved runs per side averaged 27.173s before and 27.378s after, a +0.75% difference against a within-arm spread of up to 2.08%. That is expected — in the VM every instruction costs one cycle and there is no SIMD, so a cubic-extension inversion (an addition chain of dozens of multiplications) dominates in a way it never does natively, where the 27 seconds go to deserializing 1.6 GB, Merkle paths and FRI. The prover side is not measured either and is expected neutral for the same reason: there the zerofier is evaluated once per proof, not per row.
Collaborator
Author
|
/bench-verify |
|
⏳ Benchmark started on the bench server. Two verifier arms (monolithic + continuations over an ethrex 20-tx block), then the recursion-guest cycle comparison, which adds guest builds on top — longer on a cold runner. The bench server is occupied until it finishes. |
Verifier benchmark —
|
| Metric | main | PR | Δ |
|---|---|---|---|
| Verify time (ABBA, 20 pairs) | 2.514s | 2.518s | +0.18% ⚪ |
| Proof size (exact, 1 reading) | 102.33 MiB | 102.33 MiB | +0.00% ⚪ |
pairs: 20 mean A (PR): 2.518s mean B (main): 2.514s
[parametric] paired-t mean +0.18% sd 0.76% se 0.17%
95% CI: [-0.18%, +0.54%] (t df=19 = 2.093)
[robust] median +0.30% Wilcoxon W+=135 W-=75 p(exact)=0.2774 (z=+1.10)
run-to-run jitter: A CV 0.44% B CV 0.58% (lower = steadier)
within-session drift: -0.32% over the run, 1st->2nd half -0.23%
⚪ INCONCLUSIVE — effect not separable from 0 at n=20 (point estimate ~+0.30%). Add pairs to resolve.
ethrex 20-tx block · continuations, epoch 2^20 (3 epochs) · blowup=2, 219 queries
| Metric | main | PR | Δ |
|---|---|---|---|
| Verify time (ABBA, 8 pairs) | 3.154s | 3.175s | +0.67% 🔴 |
| Proof size (exact, 1 reading) | 177.07 MiB | 177.07 MiB | +0.00% ⚪ |
pairs: 8 mean A (PR): 3.175s mean B (main): 3.154s
[parametric] paired-t mean +0.67% sd 0.72% se 0.25%
95% CI: [+0.08%, +1.27%] (t df=7 = 2.365)
[robust] median +0.57% Wilcoxon W+=27 W-=1 p(exact)=0.0312 (z=+2.11)
run-to-run jitter: A CV 0.62% B CV 0.16% (lower = steadier)
within-session drift: +0.31% over the run, 1st->2nd half +0.21%
🔴 REAL REGRESSION — PR verifies ~0.67% slower (paired-t and Wilcoxon agree).
Verify-time rows only: drift-free interleaved A/B/B/A, with paired-t and exact Wilcoxon — trust the verdict when the two agree. Proof sizes are single exact readings (no averaging). - = PR faster.
Recursion guest cycles — verifier running INSIDE the VM (main vs PR)
empty program · monolithic · blowup=2, 1 query (diagnostic — NOT a real verifier cost)
Single exact reading per ref — no ABBA: guest cycles are deterministic for a fixed
(guest ELF, input blob), so there is no machine drift to cancel.
| Metric | main | PR | Δ |
|---|---|---|---|
| Guest cycles | 32.1M | 25.6M | -6.6M (-20.40%) |
| Keccak calls | 3029 | 3029 | 0 |
baseline origin/main 99d7567afe guest=recursion-min.elf
PR 9f2dcc7d4e0f49dcb5eebea77ba2c0c6fe7678a4 9f2dcc7d4e guest=recursion-min.elf
note: cycles reproduce to ~±100k (build codegen + proof nondeterminism);
treat sub-100k deltas as noise, not signal.
raw (exact integer counts)
ref_b_sha=99d7567afec95c672e78065bc2bacb4416fb7577 ref_b_elf=recursion-min.elf ref_b_cycles=32137866 ref_b_keccak=3029 ref_b_execute_wall_s=1
ref_a_sha=9f2dcc7d4e0f49dcb5eebea77ba2c0c6fe7678a4 ref_a_elf=recursion-min.elf ref_a_cycles=25582694 ref_a_keccak=3029 ref_a_execute_wall_s=0
delta_cycles=-6555172 delta_keccak=0
ethrex 20-tx block · continuations, epoch 2^21 (2 epochs) · blowup=2, 219 queries (128-bit)
Single exact reading per ref — no ABBA: guest cycles are deterministic for a fixed
(guest ELF, input blob), so there is no machine drift to cancel.
| Metric | main | PR | Δ |
|---|---|---|---|
| Guest cycles | 1910.0M | 1877.4M | -32.7M (-1.71%) |
| Keccak calls | 3296811 | 3250507 | -46304 |
baseline origin/main 99d7567afe guest=recursion-cont-blowup2.elf
PR 9f2dcc7d4e0f49dcb5eebea77ba2c0c6fe7678a4 9f2dcc7d4e guest=recursion-cont-blowup2.elf
note: cycles reproduce to ~±100k (build codegen + proof nondeterminism);
treat sub-100k deltas as noise, not signal.
raw (exact integer counts)
ref_b_sha=99d7567afec95c672e78065bc2bacb4416fb7577 ref_b_elf=recursion-cont-blowup2.elf ref_b_cycles=1910023462 ref_b_keccak=3296811 ref_b_execute_wall_s=31
ref_a_sha=9f2dcc7d4e0f49dcb5eebea77ba2c0c6fe7678a4 ref_a_elf=recursion-cont-blowup2.elf ref_a_cycles=1877350891 ref_a_keccak=3250507 ref_a_execute_wall_s=31
delta_cycles=-32672571 delta_keccak=-46304
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.