feat: 超级大更新

[SherkeyXD]

1. 合并来自上游的更改 https://github.com/PaddlePaddle/FastDeploy/tree/release/1.1.0
2. 更新了 CI 流程，现在有自动构建了
3. 更新 onnxruntime 版本至 1.29.0
4. 添加 ppocr v5 与 v6 支持
5. 重构目录结构

[sourcery-ai]

嘿——我发现了 5 个问题

AI Agent 提示词

请处理本次代码审查中的评论：

## 单独评论

### 评论 1
<location path="llm/server/server/engine/infer.py" line_range="345-346" />
<code_context>
+                    task["stop_seqs_len"].append(0)
+                self.share_inputs['stop_seqs_len'][:] = np.array(
+                                                        task["stop_seqs_len"], dtype="int32")
+                self.share_inputs['stop_seqs'][:stop_seqs_num, :len(task['stop_seqs'][0])] = np.array(
+                                                        task["stop_seqs"], dtype="int64")
+
+            if self.is_speculate_decoding:
</code_context>
<issue_to_address>
**问题 (bug_risk)：** 不包含停止序列的普通请求会以 `task["stop_seqs"] == []` 进入 `dy_input_preprocess`，因此在推理开始前，`len(task["stop_seqs"][0])` 会抛出 `IndexError`。对于此类请求，`process_request` 始终会调用 `update_stop_seq`，而该调用会创建空列表。

**触发条件：** 请求省略 `stop_sequences` 或提供空列表时。

**建议修复：** 当 `stop_seqs_num == 0` 时跳过张量赋值，同时显式清空共享的停止序列张量。
</issue_to_address>

### 评论 2
<location path="llm/server/server/engine/infer.py" line_range="268-295" />
<code_context>
         self.share_inputs['free_list_len'] = paddle.full(
                             shape=[1], fill_value=self.free_list_len, dtype="int32")

+        self.share_inputs['stop_seqs_len'] = paddle.full(shape=[self.max_stop_seqs_num,],
+                                            fill_value=0,
+                                            dtype="int32")
</code_context>
<issue_to_address>
**问题 (broader_impact)：** 停止序列张量是全局共享输入，但循环会为批次中的每个任务重复覆盖它们，而不是按 `idx` 存储每个任务的值。因此，批处理请求会使用最后处理任务的停止序列；当下一个任务的序列数量更少时，之前任务留下的行也会保持陈旧数据。

**触发条件：** 同一批次中调度了两个或更多具有不同停止序列的请求时。

**建议修复：** 使停止序列输入按批次索引组织，并使用 `idx` 写入；或者确保每个请求都使用其自身的停止序列数据进行评估。
</issue_to_address>

### 评论 3
<location path="llm/server/server/engine/infer.py" line_range="341-346" />
<code_context>

+            if "stop_seqs_len" in task:
+                stop_seqs_num = len(task["stop_seqs_len"])
+                for i in range(stop_seqs_num, self.max_stop_seqs_num):
+                    task["stop_seqs_len"].append(0)
+                self.share_inputs['stop_seqs_len'][:] = np.array(
+                                                        task["stop_seqs_len"], dtype="int32")
+                self.share_inputs['stop_seqs'][:stop_seqs_num, :len(task['stop_seqs'][0])] = np.array(
+                                                        task["stop_seqs"], dtype="int64")
+
+            if self.is_speculate_decoding:
</code_context>
<issue_to_address>
**问题 (bug_risk)：** 包含超过 `MAX_STOP_SEQS_NUM` 个序列的请求既未进行验证，也未被截断；随后代码会将长度超过固定 `[max_stop_seqs_num]` 张量的数组进行赋值，从而引发形状/广播错误。同样，长度超过 `STOP_SEQS_MAX_LEN` 的序列也会超出赋值时固定的第二维。

**触发条件：** 客户端发送的停止序列数量超过 `MAX_STOP_SEQS_NUM`，或分词后的停止序列长度超过 `STOP_SEQS_MAX_LEN` 时。

**建议修复：** 在构造固定大小的张量之前，对这两个限制进行验证，并拒绝或截断超出限制的输入。
</issue_to_address>

### 评论 4
<location path="llm/server/server/engine/infer.py" line_range="73-82" />
<code_context>
         self.cache_kvs = {}
         self.init_inputs()

+        if self.is_speculate_decoding:
+            logger.info(f'Using speculate decoding, method: {self.speculate_config.speculate_method}.')
+            if self.speculate_config.speculate_method == "inference_with_reference":
+                self.proposer = InferenceWithReferenceProposer(
+                    self.speculate_config.speculate_max_draft_token_num,
+                    self.speculate_config.speculate_max_ngram_size,
+                    self.args.max_batch_size,
+                    self.args.max_seq_len)
+        else:
+            self.proposer = None
</code_context>
<issue_to_address>
**问题 (bug_risk)：** 不受支持的 `speculate_method` 仍会使 `is_speculate_decoding` 为 true，但只有在 `inference_with_reference` 情况下才会为 `self.proposer` 赋值。运行循环随后会访问 `self.proposer` 并抛出 `AttributeError`，而推测执行路径此时已经启用。

**触发条件：** 模型配置包含除 `None` 或 `inference_with_reference` 之外的推测方法时。

**建议修复：** 记录错误后拒绝不受支持的方法；或者在条件判断之前初始化 `self.proposer = None`，并禁用推测解码。

```suggestion
        self.proposer = None
        if self.is_speculate_decoding:
            logger.info(f'Using speculate decoding, method: {self.speculate_config.speculate_method}.')
            if self.speculate_config.speculate_method == "inference_with_reference":
                self.proposer = InferenceWithReferenceProposer(
                    self.speculate_config.speculate_max_draft_token_num,
                    self.speculate_config.speculate_max_ngram_size,
                    self.args.max_batch_size,
                    self.args.max_seq_len)
            else:
                self.is_speculate_decoding = False
```
</issue_to_address>

### 评论 5
<location path="llm/server/server/engine/token_processor.py" line_range="154" />
<code_context>
+        for token_id in token_ids:
</code_context>
<issue_to_address>
**问题 (bug_risk)：** 当一个推测步骤包含多个已接受的 token，且最后一个已接受的 token 是 EOS 时，`_get_single_result` 一看到 EOS 就会清空 `result["token_ids"]`，从而丢弃该步骤中之前所有非 EOS token。调用方已经在内部统计并追加了这些 token，因此客户端永远无法在流式响应中收到它们。

**触发条件：** 推测解码在一个步骤中接受多个 token，且接受的序列以 EOS token 结尾时。

**建议修复：** 将非 EOS 的已接受 token 追加到结果中，并在遇到 EOS 时停止处理，而不要清除已经为响应累积的 token。

```suggestion

```
</issue_to_address>

---

Sourcery 对开源项目免费——如果您喜欢我们的审查，请考虑分享它们 ✨

• X
• Mastodon
• LinkedIn
• Facebook

Original comment in English

Hey - I've found 5 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="llm/server/server/engine/infer.py" line_range="345-346" />
<code_context>
+                    task["stop_seqs_len"].append(0)
+                self.share_inputs['stop_seqs_len'][:] = np.array(
+                                                        task["stop_seqs_len"], dtype="int32")
+                self.share_inputs['stop_seqs'][:stop_seqs_num, :len(task['stop_seqs'][0])] = np.array(
+                                                        task["stop_seqs"], dtype="int64")
+
+            if self.is_speculate_decoding:
</code_context>
<issue_to_address>
**issue (bug_risk):** A normal request without stop sequences reaches `dy_input_preprocess` with `task["stop_seqs"] == []`, so `len(task["stop_seqs"][0])` raises `IndexError` before inference starts. `process_request` always calls `update_stop_seq` for such requests, which creates the empty lists.

**Triggers:** When a request omits `stop_sequences` or supplies an empty list.

**Suggested fix:** Skip the tensor assignment when `stop_seqs_num == 0`, while explicitly clearing the shared stop-sequence tensors.
</issue_to_address>

### Comment 2
<location path="llm/server/server/engine/infer.py" line_range="268-295" />
<code_context>
         self.share_inputs['free_list_len'] = paddle.full(
                             shape=[1], fill_value=self.free_list_len, dtype="int32")

+        self.share_inputs['stop_seqs_len'] = paddle.full(shape=[self.max_stop_seqs_num,],
+                                            fill_value=0,
+                                            dtype="int32")
</code_context>
<issue_to_address>
**issue (broader_impact):** The stop-sequence tensors are global shared inputs, but the loop overwrites them once for every task in a batch instead of storing values per `idx`. Consequently, batched requests use the stop sequences from the last task processed, and rows from an earlier task remain stale when the next task has fewer sequences.

**Triggers:** When two or more requests with different stop sequences are scheduled in the same batch.

**Suggested fix:** Make the stop-sequence inputs batch-indexed and write them using `idx`, or otherwise ensure each request is evaluated with its own stop-sequence data.
</issue_to_address>

### Comment 3
<location path="llm/server/server/engine/infer.py" line_range="341-346" />
<code_context>

+            if "stop_seqs_len" in task:
+                stop_seqs_num = len(task["stop_seqs_len"])
+                for i in range(stop_seqs_num, self.max_stop_seqs_num):
+                    task["stop_seqs_len"].append(0)
+                self.share_inputs['stop_seqs_len'][:] = np.array(
+                                                        task["stop_seqs_len"], dtype="int32")
+                self.share_inputs['stop_seqs'][:stop_seqs_num, :len(task['stop_seqs'][0])] = np.array(
+                                                        task["stop_seqs"], dtype="int64")
+
+            if self.is_speculate_decoding:
</code_context>
<issue_to_address>
**issue (bug_risk):** Requests containing more than `MAX_STOP_SEQS_NUM` sequences are not validated or truncated; the code then assigns an array longer than the fixed `[max_stop_seqs_num]` tensor and raises a shape/broadcasting error. A sequence longer than `STOP_SEQS_MAX_LEN` likewise exceeds the fixed second dimension during assignment.

**Triggers:** When a client sends more stop sequences than `MAX_STOP_SEQS_NUM` or a tokenized stop sequence longer than `STOP_SEQS_MAX_LEN`.

**Suggested fix:** Validate and reject or truncate both limits before constructing the fixed-size tensors.
</issue_to_address>

### Comment 4
<location path="llm/server/server/engine/infer.py" line_range="73-82" />
<code_context>
         self.cache_kvs = {}
         self.init_inputs()

+        if self.is_speculate_decoding:
+            logger.info(f'Using speculate decoding, method: {self.speculate_config.speculate_method}.')
+            if self.speculate_config.speculate_method == "inference_with_reference":
+                self.proposer = InferenceWithReferenceProposer(
+                    self.speculate_config.speculate_max_draft_token_num,
+                    self.speculate_config.speculate_max_ngram_size,
+                    self.args.max_batch_size,
+                    self.args.max_seq_len)
+        else:
+            self.proposer = None
</code_context>
<issue_to_address>
**issue (bug_risk):** An unsupported `speculate_method` still makes `is_speculate_decoding` true, but `self.proposer` is only assigned for `inference_with_reference`. The run loop later evaluates `self.proposer`, raising `AttributeError`, while the speculative execution path has already been enabled.

**Triggers:** When the model configuration contains a speculative method other than `None` or `inference_with_reference`.

**Suggested fix:** Reject unsupported methods after logging the error, or initialize `self.proposer = None` before the conditional and disable speculative decoding.

```suggestion
        self.proposer = None
        if self.is_speculate_decoding:
            logger.info(f'Using speculate decoding, method: {self.speculate_config.speculate_method}.')
            if self.speculate_config.speculate_method == "inference_with_reference":
                self.proposer = InferenceWithReferenceProposer(
                    self.speculate_config.speculate_max_draft_token_num,
                    self.speculate_config.speculate_max_ngram_size,
                    self.args.max_batch_size,
                    self.args.max_seq_len)
            else:
                self.is_speculate_decoding = False
```
</issue_to_address>

### Comment 5
<location path="llm/server/server/engine/token_processor.py" line_range="154" />
<code_context>
+        for token_id in token_ids:
</code_context>
<issue_to_address>
**issue (bug_risk):** When a speculative step contains several accepted tokens and the final accepted token is EOS, `_get_single_result` clears `result["token_ids"]` as soon as it sees EOS, discarding all preceding non-EOS tokens from that step. The caller has already counted and appended those tokens internally, so the client never receives them in the streamed response.

**Triggers:** When speculative decoding accepts multiple tokens in one step and the accepted sequence ends with an EOS token.

**Suggested fix:** Append non-EOS accepted tokens to the result and stop processing at EOS without clearing tokens already accumulated for the response.

```suggestion

```
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[sourcery-ai]

New security issues found

[SherkeyXD]

Related: MaaAssistantArknights/MaaDeps#45

[SherkeyXD]

Ready to merge
Deps CI: https://github.com/MaaAssistantArknights/MaaDeps/actions/runs/34429096481?pr=45

[MistEO]

🐂🍺