[chengxinshengglj-png]

English | 中文

---

中文

问题

在 DSH 入口下，概念图（gm_nodes / gm_edges）永远不会被自动抽取写入。

检查项	结果
dsh.ts 是否 import upsertEdge	没有
dsh.ts 里写节点的调用点	仅 1 处 —— 手动工具 gm_record
index.ts 里写边的调用点	仅 1 处 —— gm_record，且只在传 relatedSkill 时建一条 SOLVED_BY
src/extractor/extract.ts 的提示词	写死「必须且只能包含 summary、outcome、triples 三个顶层字段」

所以问题不是“忘了写”，而是契约里根本没有概念的位置。本机实测（1,159 条轮次记忆、2,983 条导航三元组）：

• 119 个节点、77 条边全部创建于 2026-09-09 之前；此后新增的 23 个节点全是孤点；
• nodesForTurnMemories() 能取到至少一个节点的轮次记忆占比 0%；
• 因此 assembleContext() 每轮渲染出的知识图谱段都是空的，提示词里只有 <memory_capsules> 和 <navigation_graph>。

做法

把「概念」提升为抽取契约的一等字段，并在 extractOnce 落库。

src/extractor/contract.ts —— 新增两个可选块：

entities:  { name (1..24), type: EVENT|SKILL|TASK, description }
relations: { from, to, type: <既有 7 种边类型>, instruction }

from / to 必须指回同批 entities 的 name；additionalProperties: false 保持不变。

src/store/store.ts —— 新增 applyConceptExtraction()：

• 已存在的概念走 confirm，不是替换。默认 upsert 在替换时会 DELETE 掉该节点原有的 gm_node_sources，等于摘掉别的轮次的证据；复现一个概念应当是「提高置信度 + 增加来源」；
• 边按 (from, to, type) 去重，重复观察只更新 instruction；
• 端点未在同批声明的边直接丢弃 —— 指向未声明概念的链接不构成证据；
• 类型集合在落库前再校验一次（原因见下）。

src/extractor/extract.ts —— 三个必需字段不变，提示词补充两个可选块与一个带概念的示例。没有引入任何宿主侧的猜测、改写或语义推断，schema 仍是唯一闸门。

dsh.ts —— 在 replaceNavigationTriples 之后调用。

为什么是可选而不是必需

• 现有两个入口共用 Extractor，字段可选则旧 payload 行为完全不变；
• 一轮对话本来就可能没有可复用概念（“继续”“好的”），强制非空只会诱导模型编造；
• 因此这是纯增量改动，不构成 breaking change。

一个顺带发现

TypeBox 的 Value.Check 不校验 enum。

const S = Type.Object({ type: Type.String({ enum: ["EVENT","SKILL","TASK"] }) });
Value.Check(S, { type: "CONCEPT" });   // → true

minLength、maxLength、additionalProperties 都生效，唯独 enum 放行。这意味着现有契约里的 outcome 枚举在运行时也没有被拦住。

本 PR 在持久化层自行校验类型集合，没有擅自改 outcome 的语义 —— 是否要在 assertGraphExtractionContract 里补一次显式枚举断言，由你判断。

第二部分：召回命中率（第二个提交）

补上概念层之后，本机又实测了一周的真实会话，发现召回有一个独立的缺陷：短追问会整块消失。

recall() 按 recallMaxNodes（本机配 3）取候选，dsh.ts 随后用 filterDshRecallMemories 丢掉「同会话且当前可见」的条目。如果这 3 条恰好全来自当前会话，过滤后为 0，dsh.ts 直接 return decision —— 这一轮一条记忆都不注入。

实测（同一会话、同一查询）：

查询	旧行为	加候选池后
体系7审计好了吗？	3 条候选，0 条跨会话 → 过滤后 0 条	15 条候选，9 条跨会话 → 过滤后 9 条
客必得 账号 关联 任务	3 条	15 条

三处改动：

1. recall(query, { overFetch }) —— 把「检索宽度」与「渲染宽度」解耦。默认 1 完全保持原有行为；DSH 路径传 5，过滤后再 slice(0, recallMaxNodes)。向量路径本来就是全表扫描、词法是一次 LIKE，扩大候选池没有可测量的成本（实测 98102 ms vs 120123 ms）。

2. 词法臂改为按词探测 —— 原来要求整句作为子串出现。中文提问天然是句子，客必得 账号 关联 任务 永远不会原样出现在任何 summary 里，所以这一臂长期是空的。现在按标点/空白切词、剥掉疑问外壳（好了吗/是多少/吗…）后逐个 LIKE，并按命中数排序。

精度用多数命中守住：两词查询要求两词都命中。这条规则来自实测——发布端口，配置文件 在宽松门槛下召回了 5 条，没有一条提到端口，只共享了「配置文件」，属于噪声。

3. 新近度作为有界平局裁决 —— 30 天半衰期、上限 0.15，低于一个 reciprocal-rank 步长（1/1 − 1/2 = 0.5）。所以它只能重排近似并列的候选，不会盖过 embedding 给出的相似度顺序。

第三部分：精排（可选）与巩固原语（第三个提交）

精排：双塔嵌入把 query 和文档分开编码，看不到词级交互。本机实测，cross-encoder 给一条真匹配打 0.9974，次名只有 0.3435 —— 这个排序 embedding 分数根本没给出。

Recaller.setRerankFn() 接受一个可选的 RerankFn；DSH 路径在配置了 rerankEndpoint 时构造 HTTP 客户端。

为什么不默认开：本机 CPU cross-encoder 约 140 ms/条，8 条候选会给一轮增加 1.2 s（而召回本身只要 150 ms）。rerankTopN（默认 6）限制成本，rerankTimeoutMs（默认 1500）限制最坏情况。所有失败路径 —— 没配端点、超时、非 200、坏 payload、条数不符、抛异常 —— 都原样返回 embedding 顺序。只精排头部，尾部保持原序，因此精排"不同意"也删不掉候选。

接线方式（best-effort、opt-in、精排坏了只损失精排）参考 slow-stack/mneme（MIT）。

巩固原语：src/consolidate/cluster.ts 改编自 mneme 的 dream/clustering.js（MIT，文件头已署名）：余弦相似度、k-means（k-means++ 播种 + 空簇修复）、同 key 冲突候选检测。独立模块、有测试、不碰数据库，目前没有任何调用方。

为什么没搬整个 autoDream：先量过。1,172 条轮次记忆 / 35 个会话里，同会话相似度 >0.90 的只有 14 对，跨会话 >0.93 的 0 对；summary 中位 93 字，没有空行、确认行或填充行。这个插件产出的语料本身就具体且不重复，巩固几乎找不到可合并项，却要每批付一次 LLM 调用。原语留下是因为冲突检测那部分不能靠猜，值得先备好。

测试

新增 test/concept-extraction.test.ts（6 例）+ test/recall-pool.test.ts（6 例）+ test/rerank.test.ts（7 例）+ test/consolidate-cluster.test.ts（9 例）+ test/extraction-contract.test.ts 增补 2 例：

• 概念落库并挂上本轮消息来源（召回可达的前提）；
• 复现概念时保留既有内容、validatedCount 递增、来源从 2 条变 4 条而不是被重置；
• 端点未声明的边被丢弃；
• 同一条边重复观察不产生重复行；
• 自指边被忽略；
• 无概念的轮次是 no-op；
• 非法 kind 被跳过；
• 默认 overFetch 严格等于原行为；overFetch: 3 返回更宽的池；
• 多词查询能命中，不再要求整句；
• 剥掉疑问外壳后仍能命中；
• 两词查询只共享一个词时返回空（锁住上面那个噪声案例）；
• 未配精排时顺序与 embedding 完全一致；精排给出分数时头部按分数重排；
• 精排返回 null 或抛错时回落到原顺序；rerankTopN 只影响头部条数；
• HTTP 客户端按 index 回填而非依赖返回顺序；payload 条数不符时返回 null；
• 聚类：相似度退化输入返回 0 而非 NaN；k-means 能分开两个分离簇、k>n 时收敛成一簇；冲突检测只配对同 key 项。

全量：27 个文件 / 181 个测试通过（原 23 / 149）。

影响面

项	变化
每轮抽取输出	多 35 行 entity/relation（约 100200 token）。llmMaxTokens 默认 4096，实测远未触顶
数据库	每轮新增 14 个节点、03 条边
召回	<knowledge_graph> 段开始有内容，受既有 recallMaxNodes 限制
召回宽度	默认不变（overFetch: 1）；仅 DSH 路径请求 5 倍池并在过滤后截断，检索耗时无可测变化
词法召回	从「整句子串」放宽到「按词探测 + 多数命中」；旧行为下长期为空的臂现在能出结果
精排	默认关闭；配了 rerankEndpoint 才启用，每轮 +0.4~1.2 s（取决于 rerankTopN），任何失败都回落原序
兼容性	无变化：字段可选，recall() 的第二个参数可选，overFetch 缺省即原行为

不包含

• 不做历史回填：只让新轮次进图；存量需要一次性脚本，我刻意没放进插件；
• 不清理图里已有的历史节点；
• 不改默认阈值：semanticScoreThreshold 与 recallMaxNodes 的默认值保持上游原样，本机的 0.60 属于本地配置，不是本 PR 的一部分。

---

English

Problem

On the DSH entry point the concept graph is never written by extraction.

check	result
dsh.ts imports upsertEdge?	no
node writers in dsh.ts	1 — the manual gm_record tool
edge writers in index.ts	1 — gm_record, and only a SOLVED_BY when relatedSkill is passed
src/extractor/extract.ts prompt	pins the payload to summary, outcome, triples only

So it is not that a write was forgotten — there is no shape in which a concept could arrive. Measured on a database with 1,159 turn memories and 2,983 navigation triples:

• every one of the 119 nodes and 77 edges predates 2026-09-09; the 23 nodes added since are all isolated;
• nodesForTurnMemories() resolves at least one node for 0 % of turn memories;
• assembleContext() therefore renders an empty knowledge-graph section on every turn — the prompt only ever carries <memory_capsules> and <navigation_graph>.

What this does

Promote concepts to first-class fields on the existing contract and persist them in extractOnce.

src/extractor/contract.ts — two optional blocks:

entities:  { name (1..24), type: EVENT|SKILL|TASK, description }
relations: { from, to, type: <the seven declared edge kinds>, instruction }

from / to must name a concept declared in the same payload; additionalProperties: false is kept.

src/store/store.ts — new applyConceptExtraction():

• a concept that already exists is confirmed, not replaced. The default upsert path deletes the node's existing gm_node_sources rows on replacement, which would detach the evidence of earlier turns; re-observing a concept should raise confidence and add provenance only;
• edges are keyed by (from, to, kind), so re-observing updates the instruction instead of duplicating the row;
• a relation whose endpoints were not declared is dropped — a link to an undeclared concept is not evidence;
• kind values are re-checked before insert (see below).

src/extractor/extract.ts — the three required fields are unchanged; the prompt documents the two optional blocks and carries one worked example. No host-side guessing, rewriting or semantic inference is added: the schema stays the only gate.

dsh.ts — calls the helper right after replaceNavigationTriples.

Why optional rather than required

• existing payloads keep working unchanged, for both entry points that share Extractor;
• a turn legitimately may declare no reusable concept ("continue", "ok"), and forcing the field would invite invented entities;
• the change is therefore purely additive, not a breaking change.

A side finding worth its own fix

TypeBox's Value.Check does not enforce enum.

const S = Type.Object({ type: Type.String({ enum: ["EVENT","SKILL","TASK"] }) });
Value.Check(S, { type: "CONCEPT" });   // → true

minLength, maxLength and additionalProperties are enforced; enum is not. This means assertGraphExtractionContract does not currently reject an out-of-enum outcome either.

This PR re-checks the kind sets at persistence time instead of touching outcome semantics. Whether to add an explicit enum assertion to the contract is your call — it would change existing behaviour, so I did not.

Part two: recall hit rate (second commit)

With the concept layer in place I measured another week of real sessions and found an independent defect: short follow-ups lost the whole block.

recall() returns recallMaxNodes candidates (3 here); dsh.ts then drops anything already visible in the live session. When all three hits come from that session, the filter leaves nothing and dsh.ts returns without injecting — zero memory for that turn.

Measured on the same session and query:

query	before	after
体系7审计好了吗？	3 candidates, 0 cross-session → 0 after filtering	15 candidates, 9 cross-session → 9 after filtering
客必得 账号 关联 任务	3	15

Three changes:

1. recall(query, { overFetch }) decouples retrieval width from render width. Default 1 reproduces the old behaviour exactly; the DSH path asks for 5 and slices back after filtering. The vector path is already a full scan and the lexical path is a single LIKE, so a wider pool is not measurable in wall time (98–102 ms vs 120–123 ms here).

2. The lexical arm probes terms instead of requiring the sentence. It used to demand the entire query as a substring. Chinese questions are sentences, so 客必得 账号 关联 任务 never appeared verbatim in any summary and this arm had been returning nothing. It now splits on punctuation and whitespace, strips question shells (好了吗, 是多少, 吗, …), and ranks by how many terms hit.

Precision is defended by a majority-hit rule: a two-term query must match both. That rule came from measurement — under a looser threshold 发布端口，配置文件 returned five summaries that never mentioned the port, matching only on 配置文件.

3. Recency joins the fusion as a bounded tie-breaker — 30-day half life, 0.15 ceiling, deliberately below one reciprocal-rank step (1/1 − 1/2 = 0.5). It can reorder near-ties toward current work without overriding the similarity order the embedding provider produced.

Part three: optional reranking, plus consolidation primitives (third commit)

Reranking. A bi-encoder encodes query and document separately, so it cannot see term-level interaction. Measured here, a cross-encoder scored a true match at 0.9974 while the runner-up sat at 0.3435 — an ordering the embedding scores never produced.

Recaller.setRerankFn() accepts an optional RerankFn; the DSH path builds an HTTP client when rerankEndpoint is configured.

Why it is off by default: the local CPU cross-encoder costs ~140 ms per document, so eight candidates add 1.2 s to a turn whose recall otherwise takes 150 ms. rerankTopN (default 6) bounds the cost and rerankTimeoutMs (default 1500) bounds the worst case. Every failure path — no endpoint, timeout, non-200, malformed payload, mismatched result count, thrown error — returns the embedding order unchanged. Only the head is scored and the tail keeps its order, so a disagreeing reranker can never erase a candidate.

The integration shape (best-effort, opt-in, a broken reranker may only lose the rerank) follows slow-stack/mneme (MIT).

Consolidation primitives. src/consolidate/cluster.ts adapts mneme's dream/clustering.js (MIT, attribution in the file header): cosine similarity, k-means with k-means++ seeding and empty-cluster repair, and same-key conflict candidate detection. Standalone, tested, no database access, and no caller yet.

Why not the whole autoDream: it was measured first. Across 1,172 turn memories in 35 sessions, only 14 same-session pairs exceed 0.90 similarity and no cross-session pair exceeds 0.93; summaries have a median length of 93 characters with no empty, confirmatory, or filler rows. This plugin's output is already specific and non-duplicative, so consolidation would find almost nothing to merge while paying an LLM call per batch. The primitives stay because conflict detection is the part that must not be guessed at.

Tests

New test/concept-extraction.test.ts (6 cases), test/recall-pool.test.ts (6), test/rerank.test.ts (7), test/consolidate-cluster.test.ts (9), plus 2 in test/extraction-contract.test.ts:

• concepts are stored and linked to the turn's message rows (the precondition for recall to resolve them);
• a recurring concept keeps its established content, increments validatedCount, and grows provenance 2 → 4 instead of resetting it;
• a relation with an undeclared endpoint is dropped;
• re-observing the same relation does not duplicate the edge;
• a self-referencing relation is ignored;
• a turn that declares nothing is a no-op;
• out-of-enum kinds are skipped;
• the default overFetch matches the previous behaviour exactly, and 3 returns a wider pool;
• a multi-term query matches without the exact sentence;
• a question shell is stripped before probing;
• a two-term query that shares only one term returns nothing (locks the noise case);
• without a reranker the order matches the embedding order; a scored head reorders by score;
• a null or throwing reranker falls back to the original order; rerankTopN bounds the scored head;
• the HTTP client maps by index rather than trusting response order, and returns null on a short payload;
• clustering: degenerate vectors score 0 rather than NaN; k-means separates two disjoint blobs and collapses when k > n; conflict detection only pairs same-key items.

Full suite: 27 files / 181 tests pass, up from 23 / 149.

Impact

	change
extraction output	+3..5 entity/relation lines (~100..200 tokens). llmMaxTokens defaults to 4096 and is far from binding in practice
database	+1..4 nodes and 0..3 edges per completed turn
recall	the knowledge-graph section starts carrying content, bounded by the existing recallMaxNodes
recall pool	unchanged by default (overFetch: 1); only the DSH path asks for 5x and truncates after filtering, with no measurable retrieval cost
lexical recall	from whole-sentence substring to term probing with a majority-hit rule; the arm that was permanently empty now returns results
reranking	off by default; enabled only when rerankEndpoint is set, costing 0.4–1.2 s per turn depending on rerankTopN, with fallback to the embedding order on any failure
compatibility	none: fields are optional, the second recall() argument is optional, and overFetch defaults to the previous behaviour

Out of scope

• no historical backfill — this only makes new turns durable; existing rows would need a one-off script, which I deliberately kept out of the plugin;
• no cleanup of heuristic nodes already present in a graph;
• no default threshold change — semanticScoreThreshold and recallMaxNodes keep their upstream defaults; the 0.60 in my local profile is configuration, not part of this PR.

Reproduce

git clone https://github.com/adoresever/graph-memory && cd graph-memory
git checkout 5841b19
npm install
git apply 0001-feat-concept-graph.patch
npm run build:dsh
npm test        # 159 passed