fix(provider): record standalone calls and attribute fallback usage

[RhoninSeiei]

Standalone SDK calls, plugin tool-loop agents and scheduled agents can return without appearing in provider statistics. Record these paths consistently, including errors and cancellations, and include them in Dashboard aggregates.

Attribute usage to the provider that performed each request, including successful tool rounds and schema-repair requests before a later fallback. Settle accumulated usage on provider switches without assigning earlier tokens to the final provider or counting them twice. Request-local ownership avoids duplicate provider and wrapper records.

Persist public token fields, retain usage attached to exceptions, and normalize unfinished execution times after fallback timing adjustments. With the current asynchronous runner initialization, early hook exits and image-preparation failures do not read an uninitialized runner. Cron records usage in its finalizer and treats an ERROR runner as a failed job.

Merged upstream master at 8b82d03 while preserving the original PR history. This PR remains independent of the OAuth provider PR. Validation: 283 related tests passed in an isolated Linux container, covering provider attribution, schema repair, errors, cancellation and unreset runners; Ruff lint and formatting checks passed.

Summary by Sourcery

Record provider usage consistently across standalone calls and agent executions while preserving accurate attribution, failure handling, and Dashboard aggregates.

New Features:

• Record statistics for standalone provider SDK calls, tool-loop agents, and scheduled agents, including successful, failed, and cancelled executions.
• Attribute token usage and persisted statistics to the provider that handled each request, including fallback attempts and additional tool/schema-repair rounds.
• Include provider-level records in Dashboard token aggregates and success-rate calculations.

Bug Fixes:

• Prevent provider usage from being duplicated or attributed to the final fallback provider after provider switches.
• Preserve token usage from provider exceptions and normalize execution timing after fallback adjustments.
• Avoid accessing uninitialized runners during early pipeline exits and image-preparation failures.
• Treat cron runners ending in an ERROR state as failed jobs while still persisting their statistics.

Enhancements:

• Centralize provider and agent statistics recording with request-local ownership and public token-field normalization.

Tests:

• Add coverage for provider attribution across fallbacks, tool rounds, schema repair, errors, cancellation, cron execution, standalone calls, and runner initialization edge cases.

[sourcery-ai]

Hey - I've found 2 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/provider/stats.py" line_range="82-91" />
<code_context>
+) -> None:
+    """Persist stats for one direct provider request."""
+    try:
+        usage = response.usage if response and response.usage else TokenUsage()
+        await db.insert_provider_stat(
+            umo=umo,
+            conversation_id=conversation_id,
+            provider_id=_provider_id(provider),
+            provider_model=provider.get_model(),
+            status=_response_status(response),
+            stats={
+                "token_usage": usage.__dict__.copy(),
+                "start_time": start_time,
+                "end_time": end_time,
+                "time_to_first_token": 0.0,
+            },
+            agent_type=agent_type,
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Clarify and stabilize the shape of stored token usage stats

Here `usage.__dict__.copy()` is persisted directly, so any future changes to `TokenUsage` (new internal fields, non-serializable attrs) will silently alter the stored schema or break serialization. To keep the stored shape stable and explicit, either expose and use a `to_dict()` on `TokenUsage`, or construct the dict from the specific fields you intend to persist (input/output/total tokens, etc.).

Suggested implementation:

```python
            stats={
                "token_usage": usage.to_dict(),
                "start_time": start_time,
                "end_time": end_time,
                "time_to_first_token": 0.0,
            },

```

You’ll need to implement a `to_dict()` method on the `TokenUsage` class (wherever it is defined) that returns a stable, explicit shape, e.g.:

- Only include the fields you intend to persist (for example: `input_tokens`, `output_tokens`, `total_tokens`, `cached_tokens`, etc.).
- Avoid non-serializable attributes or internal fields that might change over time.

For example:

```python
@dataclass
class TokenUsage:
    prompt_tokens: int = 0
    completion_tokens: int = 0
    total_tokens: int = 0

    def to_dict(self) -> dict[str, int]:
        return {
            "prompt_tokens": self.prompt_tokens,
            "completion_tokens": self.completion_tokens,
            "total_tokens": self.total_tokens,
        }
```

Adjust the exact fields to match your current `TokenUsage` definition and what you want to persist long-term.
</issue_to_address>

### Comment 2
<location path="tests/unit/test_star_context.py" line_range="150-159" />
<code_context>
         assert config.provider_settings is provider_settings
         assert config.provider_settings["fallback_chat_models"] == ["fallback-provider"]

+    @pytest.mark.asyncio
+    async def test_woke_main_agent_persists_one_aggregated_provider_stat(
+        self,
</code_context>
<issue_to_address>
**suggestion (testing):** Add tests for error scenarios in `llm_generate` to validate `ProviderStat.status` and usage when the provider fails or returns an error response.

Since stats are recorded in a `finally` block and status is derived from the LLM response, it would be good to cover non-happy paths too. Please add at least:

1. A case where `StatsProvider.text_chat` raises (e.g. `RuntimeError`), asserting via `pytest.raises` that:
   - the exception is propagated,
   - a `ProviderStat` is still persisted,
   - it has `status == "error"` and zero (or default) token usage.

2. A case where `StatsProvider.text_chat` returns an `LLMResponse` with `role="err"`, asserting that:
   - `llm_generate` returns this response,
   - the `ProviderStat` has `status == "error"` while preserving the usage values.

This will help ensure stats remain correct for failure and error-response scenarios, which downstream reporting depends on.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[RhoninSeiei]

本次自动检查此前统一失败于 anthropic==1.0.0 将内部 HTTP 模块从 httpx 更名为 httpx2。本分支已同步上游 #9769 的同等兼容修复，并保留 Anthropic 0.x 支持；重新触发的 Unit Tests、Dashboard Build、格式检查、CodeQL 和多平台 Smoke Test 均已通过。#9769 合并后，这部分公共差异会从本 PR 中消失。