Skip to content
4 changes: 3 additions & 1 deletion docs/hermes-base-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,9 @@ v98 的 shape 索引和 offset 一样只用于定位(delta 可能重排 shape

为什么不能按 dump 的整段文本比:Hermes 的缓冲区构建器会**重叠/去重**序列化后的字面量——一个字面量的最后一个值字节可以同时是下一个字面量的 tag 字节(模糊测试实测:`61 52 | cd 09 b3 05 11`,前一段以 `[String 82]` 结尾,后一条指令的 offset 正指向 `52`)。顺序解析整段缓冲区(hermesc 的 dump 就是这么打印的)从这里开始失步,之后的条目全是噪声;delta 构建的 id 宽度不同,重叠位置也不同,于是两段"噪声"在某处不一致就被判为差异。2026-09-10 的 20 轮冒烟模糊测试里 3 次误杀全部源于此,改按指令比较后全部等价。无法读二进制缓冲区(文件结构不识别)时两侧一起比较整段文本,仅辅助诊断;即使文本相等也返回 `dump-failed` 并回退 plain,不能以丢失 offset/count 的文本确认等价。结果里 `literals: 'buffer'` 标明这一点。

`normalizeDisassemblyLine` 只折叠表示层差异:按指令解析后的字面量地址、已知宽度后缀、引号外的列对齐空白、switch 表的物理偏移(含经典 `SwitchImm`)、debug 偏移。字符串内部的连续空格、跳转目标标签、寄存器均保留。未知 string ID、无法解码的字面量、未知 buffer 操作数形态直接失败;两侧都无法解析也不等价。
`normalizeDisassemblyLine` 只折叠表示层差异:按指令解析后的字面量地址、已知宽度后缀、引号外的列对齐空白、switch 表的物理偏移(含经典 `SwitchImm`)、debug 偏移、`DefineOwnById*` 的字符串操作数。字符串内部的连续空格、跳转目标标签、寄存器均保留。

**`DefineOwnById*` 的字符串操作数为什么只能丢给 raw**:Hermes 的 `BytecodeList.def` 只给 `DefineOwnByIdLong` 标了字符串操作数,短形式没标。于是同一条指令有两种 pretty 打印:普通编译 id 小、走短形式,打印**裸 id**;base 编译继承了 base 的字符串表,id 溢出 16 位后改用 Long 形式,打印**文本**,而 pretty 的文本又按显示预算截断(`equivalenceCheckPropertyName` 打成 `"equivalenceCheckP"...`)。两种表示互相还原不了——这个预算对非 ASCII 还会静默截断且不加 `...`(实测 `ab中` 打成 `"ab"`、`abcdefghijklmnop中` 打成 `"abcdefghi"`),所以还原出的完整名字永远等不上打印出来的名字。归一化因此把该操作数整个折成 `<str>`,属性名交给 raw 核对按二进制字符串表全量比较(`hermes-raw.ts` 的 `STRING_OPERANDS` 正是为此显式补了 `DefineOwnById`)。2026-09-22 之前这里会把任何超过显示预算的属性名判成差异,线上因此误杀过 base。未知 string ID、无法解码的字面量、未知 buffer 操作数形态直接失败;两侧都无法解析也不等价。

**原始操作数核对**:`hermes-raw.ts` 从 raw dump 读取指令起点和操作数类型,并检查操作数与 HBC 字节一致、指令覆盖完整函数体。字符串从 small/overflow string table 与 string storage 按完整 ASCII/UTF-16 code unit 解码;BigInt、正则和 double 读取真实字节(保留 `-0` 和尾部精度);函数引用保留索引,与顺序对齐的函数表共同检查,同名函数不能互换。地址映射为目标指令序号;整数和字符串 switch 从二进制恢复 case 值与目的地。函数运行时 flags 和参数/寄存器等字段也参与比较,剔除的仅是物理地址、debug presence 与 compact/overflow 表示。

Expand Down
52 changes: 52 additions & 0 deletions docs/hermes-review-follow-up.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# PR #85 review follow-up

## Changes

- `CacheNewObject` now resolves its shape index into the ordered, fully decoded
property keys in the pretty pass. Both registers and the per-function cache
index remain part of the comparison. Unknown shapes, undecodable keys and
unsupported operands fail closed. With no binary resolver, the shape is only
folded for diagnostics; the existing binary-data and raw-audit gates still
forbid text-only equivalence.
- Fuzz success requires a positive integer round count, all requested positive
comparisons, zero compilation/comparison failures, at least one effective
planted difference, and no missed or unbuildable negatives. Optimized-away
differences do not count. The last round also attempts a negative, so runs
shorter than ten rounds are not structurally unable to exercise rejection.
- New regressions cover shape-index relocation, key changes, key order,
register/cache preservation, malformed/missing references, text-only fallback,
and fuzz runs with zero or incomplete useful coverage.
- Enforcing that gate exposed a pre-existing generator defect in the HBC 96
CI run (seed 96, round 41): the quoted-string regex matched a gap after an
escaped quote and inserted `~` before a quoted property name. String mutation
and negative planting now use real string-token boundaries from the already
installed Babel parser and JSON-encode replacements. The minimized regression
also checks comments, regexps, templates, escapes and UTF-16 source offsets.
The failure gate remains strict; the generator is fixed rather than skipping
the failing round or changing the seed.

## Scope and evidence

The CacheNewObject regression uses synthetic instruction bytes and binary literal
sections to exercise the production pretty and raw normalizers. It does not claim
that a source-level fixture was compiled into CacheNewObject by a real hermesc.
The existing pinned HBC 96/98 compiler CI remains required before merging.

Unknown-opcode semantic coverage remains a separate follow-up, not a verified
current-compiler false-acceptance bug. Before accepting additional compiler
snapshots, audit their complete opcode/operand schemas and referenced sections.
Do not remove the whole pretty pass: information such as exception-handler tables
is not currently covered by the raw instruction normalizer alone.

Local validation in the authoring environment is limited to Node.js execution of
transpiled helpers and synthetic fixtures; Bun and hermesc are not installed.
The PR description records the subsequent CI state separately.

## CodeRabbit argument-validation follow-up

Only an omitted `--rounds` flag selects the default of 200. A flag without a
value, an empty value, or a following option is rejected with exit code 2 before
creating the output directory or invoking the compiler. Four CLI regressions
use an executable compiler fixture with a call marker to verify those side
effects do not occur; the valid-count control verifies the marker does work.
Argument-reading, compilation and run-entry functions now document their contracts.
96 changes: 73 additions & 23 deletions scripts/fuzz-hermes-base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@
* `normalizeDisassemblyLine` + a unit test) or a real hermesc delta-mode bug
* (report upstream; record it in docs/hermes-base-verification.md §3).
*
* Every tenth round also plants a one-literal change into the delta build and
* asserts the check still catches it, so a rule that folds too much shows up
* here as well.
* Every tenth round and the final round also plant a one-literal change into
* the delta build and assert the check still catches it, so a rule that folds
* too much shows up here as well. Success requires all requested comparisons,
* zero compilation failures and at least one effective planted difference.
*
* HERMESC=<path> bun scripts/fuzz-hermes-base.ts [--rounds N] [--seed S]
* [--out DIR] [--verbose]
Expand All @@ -20,26 +21,36 @@
* cases (both sources, all three HBC files) are kept under --out
* (default: a fresh temp dir, printed at the end); passing cases are deleted.
* Exit code: 0 when every round was equivalent and every planted change was
* caught, 1 otherwise.
* caught with useful coverage, 1 otherwise; invalid arguments exit 2.
*/
import { spawnSync } from 'node:child_process';
import fs from 'fs-extra';
import os from 'os';
import path from 'path';

import { compareHermesBytecode } from '../src/utils/hermes-base';
import { fuzzStringLiterals } from './hermes-fuzz-literals';
import { hermesFuzzSucceeded } from './hermes-fuzz-result';

// ---------------------------------------------------------------------------
// arguments
// ---------------------------------------------------------------------------

/** Read a flag's next token; undefined alone does not imply flag omission. */
function argValue(name: string): string | undefined {
const index = process.argv.indexOf(`--${name}`);
if (index < 0) return undefined;
return process.argv[index + 1];
}

const ROUNDS = Number(argValue('rounds') ?? 200);
// Only an omitted flag uses the default; a missing value becomes NaN.
const ROUNDS = Number(
process.argv.includes('--rounds') ? argValue('rounds') : 200,
);
if (!Number.isSafeInteger(ROUNDS) || ROUNDS <= 0) {
Comment thread
coderabbitai[bot] marked this conversation as resolved.
console.error('--rounds must be a positive safe integer');
process.exit(2);
}
const SEED = Number(argValue('seed') ?? Date.now() % 2 ** 31);
const VERBOSE = process.argv.includes('--verbose');
const OUT_DIR =
Expand Down Expand Up @@ -471,13 +482,17 @@ class Gen {
break;
}
case 1: {
// change one string literal (a real literal: quote, body without
// an unescaped quote or backslash, same quote — never the gap
// between two literals)
const literals = [...text.matchAll(/(["'])([^"'\\\n]{1,40})\1/g)];
// A regex can match the gap after an escaped closing quote. Use
// parser offsets and encode the replacement as one complete token.
const literals = fuzzStringLiterals(text).filter(
({ value }) => value.length > 0 && value.length <= 40,
);
if (literals.length > 0) {
const m = this.rng.pick(literals);
text = `${text.slice(0, m.index)}${m[1]}${m[2]}~${m[1]}${text.slice((m.index ?? 0) + m[0].length)}`;
const literal = this.rng.pick(literals);
text =
text.slice(0, literal.start) +
JSON.stringify(`${literal.value}~`) +
text.slice(literal.end);
}
break;
}
Expand Down Expand Up @@ -510,14 +525,17 @@ class Gen {
/** identical to `source` except one string literal value — must be caught */
/** `marker`: the new string, to tell whether it survived the optimizer */
plantDifference(source: string): { source: string; marker: string } | null {
const literals = [...source.matchAll(/(["'])([A-Za-z]{3,20})\1/g)];
const literals = fuzzStringLiterals(source).filter(({ value }) =>
/^[A-Za-z]{3,20}$/.test(value),
);
if (literals.length === 0) return null;
const target = this.rng.pick(literals);
const before = source.slice(0, target.index);
const after = source.slice((target.index ?? 0) + target[0].length);
const marker = `${target[2]}Z`;
const marker = `${target.value}Z`;
return {
source: `${before}${target[1]}${marker}${target[1]}${after}`,
source:
source.slice(0, target.start) +
JSON.stringify(marker) +
source.slice(target.end),
marker,
};
}
Expand All @@ -527,6 +545,7 @@ class Gen {
// compile + compare
// ---------------------------------------------------------------------------

/** Compile a generated source, returning diagnostics on compiler failure. */
function compile(
input: string,
out: string,
Expand Down Expand Up @@ -558,6 +577,7 @@ interface Finding {
count: number;
}

/** Run seeded comparisons and exit successfully only with effective coverage. */
async function main() {
const rng = new Rng(SEED);
const gen = new Gen(rng);
Expand All @@ -574,6 +594,7 @@ async function main() {
let planted = 0;
let plantedMissed = 0;
let plantedFolded = 0;
let plantedCompileErrors = 0;
const started = Date.now();

for (let round = 0; round < ROUNDS; round++) {
Expand Down Expand Up @@ -634,21 +655,33 @@ async function main() {
console.log(`round ${round}: dump failed — ${outcome.detail}`);
}

// detection check: a planted one-literal change must be rejected
if (round % 10 === 9) {
// Include the last round so even a short run attempts a negative case.
if (round % 10 === 9 || round === ROUNDS - 1) {
const wrong = gen.plantDifference(next);
if (wrong) {
const wrongJs = path.join(dir, 'wrong.js');
const wrongHbc = path.join(dir, 'wrong.delta.hbc');
fs.writeFileSync(wrongJs, wrong.source);
if (!compile(wrongJs, wrongHbc, [`-base-bytecode=${baseHbc}`])) {
const wrongError = compile(wrongJs, wrongHbc, [
`-base-bytecode=${baseHbc}`,
]);
if (wrongError) {
plantedCompileErrors++;
keep = true;
fs.writeFileSync(
path.join(dir, 'planted-compile-error.txt'),
wrongError,
);
console.log(`round ${round}: planted compile error (kept in ${dir})`);
} else {
// The literal may sit in code the optimizer removes or folds
// (`!'x'`, an unreachable switch case — Static Hermes folds far more
// than classic hermesc). Then both builds are really equivalent and
// the round tests nothing; the string storage tells, independently
// of the check under test (ASCII strings are stored as is).
if (!fs.readFileSync(wrongHbc).includes(wrong.marker)) {
plantedFolded++;
keep = true;
if (VERBOSE) {
console.log(
`round ${round}: planted "${wrong.marker}" optimized away`,
Expand All @@ -674,24 +707,32 @@ async function main() {
}
}
}
} else {
// Keep the input to diagnose a run with no effective negative cases.
keep = true;
}
}

if (!keep) fs.removeSync(dir);
}

const seconds = ((Date.now() - started) / 1000).toFixed(1);
const different = [...findings.values()].reduce((n, f) => n + f.count, 0);
console.log('');
console.log(`rounds: ${ROUNDS} in ${seconds}s (seed ${SEED})`);
console.log(`equivalent: ${equivalent}`);
console.log(
`different: ${[...findings.values()].reduce((n, f) => n + f.count, 0)} (${findings.size} unique)`,
);
console.log(`different: ${different} (${findings.size} unique)`);
console.log(`dump failed: ${dumpFailed}`);
console.log(`compile errors (generator): ${compileErrors}`);
console.log(`planted compile errors: ${plantedCompileErrors}`);
console.log(
`planted differences: ${planted}, missed: ${plantedMissed} (${plantedFolded} more optimized away, not counted)`,
);
if (planted === 0) {
console.error(
'No effective planted difference was checked; coverage is insufficient.',
);
}
if (findings.size > 0) {
console.log('');
console.log('unique differences (first occurrence, reproduction dir):');
Expand All @@ -700,7 +741,16 @@ async function main() {
console.log(` ${f.detail}`);
}
}
const ok = findings.size === 0 && dumpFailed === 0 && plantedMissed === 0;
const ok = hermesFuzzSucceeded({
rounds: ROUNDS,
equivalent,
different,
dumpFailed,
compileErrors,
planted,
plantedMissed,
plantedCompileErrors,
});
if (!ok) console.log(`\nfailing cases kept under ${OUT_DIR}`);
else if (!argValue('out')) fs.removeSync(OUT_DIR);
process.exit(ok ? 0 : 1);
Expand Down
28 changes: 28 additions & 0 deletions scripts/hermes-fuzz-literals.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { parse } from '@babel/parser';

export interface FuzzStringLiteral {
start: number;
end: number;
value: string;
}

/** Locate actual JS strings, never the gap between two closing/opening quotes. */
export function fuzzStringLiterals(source: string): FuzzStringLiteral[] {
const { tokens } = parse(source, {
sourceType: 'script',
tokens: true,
errorRecovery: true,
});
if (!tokens) throw new Error('Parser did not return string-token data');
const literals: FuzzStringLiteral[] = [];
for (const token of tokens) {
if (
typeof token.type === 'object' &&
token.type.label === 'string' &&
typeof token.value === 'string'
) {
literals.push({ start: token.start, end: token.end, value: token.value });
}
}
return literals;
}
26 changes: 26 additions & 0 deletions scripts/hermes-fuzz-result.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
export interface HermesFuzzSummary {
rounds: number;
equivalent: number;
different: number;
dumpFailed: number;
compileErrors: number;
planted: number;
plantedMissed: number;
plantedCompileErrors: number;
}

/** A green run must contain all requested comparisons and an effective negative. */
export function hermesFuzzSucceeded(summary: HermesFuzzSummary): boolean {
return (
Number.isSafeInteger(summary.rounds) &&
summary.rounds > 0 &&
summary.equivalent === summary.rounds &&
summary.different === 0 &&
summary.dumpFailed === 0 &&
summary.compileErrors === 0 &&
Number.isSafeInteger(summary.planted) &&
summary.planted > 0 &&
summary.plantedMissed === 0 &&
summary.plantedCompileErrors === 0
);
}
2 changes: 1 addition & 1 deletion src/locales/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ export default {
accountUnknown: 'Unknown',
accountNoExpiry: 'No expiry specified',
accountRenewalWarning:
'[Warning] Your Pushy paid plan expires in {{days}} days. Renew soon to avoid service disruption: {{url}}',
'[Notice] Your Pushy paid plan expires in {{days}} days. Renew soon to avoid service disruption: {{url}}',
accountLookupFailed:
'[Warning] Unable to retrieve account information. Continuing with the command.',
addedToGitignore: 'Added {{line}} to .gitignore',
Expand Down
2 changes: 1 addition & 1 deletion src/locales/zh.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ export default {
accountUnknown: '未知',
accountNoExpiry: '未设置有效期',
accountRenewalWarning:
'[警告] 您的 Pushy 付费套餐将在 {{days}} 天内到期,请及时续费,以免影响服务:{{url}}',
'[注意] 您的 Pushy 付费套餐将在 {{days}} 天内到期,请及时续费,以免影响服务:{{url}}',
accountLookupFailed: '[警告] 暂时无法获取账号信息,将继续执行命令。',
addedToGitignore: '已将 {{line}} 添加到 .gitignore',
androidCrunchPngsWarning:
Expand Down
Loading