Skip to content

feat:add compaction and log token cost - #191

Merged
Kavya Sree Kaitepalli (KavyaSree2610) merged 6 commits into
mainfrom
kkaitepalli/openai-token-logging-compaction
Sep 29, 2026
Merged

Kavya Sree Kaitepalli (KavyaSree2610) merged 6 commits into
mainfrom
kkaitepalli/openai-token-logging-compaction

Conversation

@KavyaSree2610

@KavyaSree2610 Kavya Sree Kaitepalli (KavyaSree2610) commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Token usage logging — OpenAIApi and AzureOpenAIApi now log input/output/total token counts returned by the Responses API after every call, warning when usage is absent.
  • Server-side compaction — Both clients pass context_management=[{"type": "compaction", ...}] with a configurable compact_threshold (default 200k tokens). When the server compacts, the client rebuilds its local history around the returned compaction item and logs the event. Requires openai==3.17.0; adds a --debug-http CLI flag for inspecting the underlying httpx2 client.
  • Cherry-pick — "Add Per-Instance Feedback Generation" commit for auto_memory/eval/swebenchverified.py.
    (ec4ffae)
  • Fix twine check — The release failed with ImportError: cannot import name 'errors' from 'packaging', because pip install -e . downgraded twine 7.0's deps (packaging>=26.1, rich>=14.3.3). Bumped packaging to 26.3 and rich to 15.0.0, and moved the build/twine install after the project install.

@KavyaSree2610 Kavya Sree Kaitepalli (KavyaSree2610) changed the title Kkaitepalli/OpenAI token logging compaction feat:add compaction and log token cost Sep 28, 2026
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Sep 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.52%. Comparing base (713b551) to head (f886f48).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #191      +/-   ##
==========================================
+ Coverage   68.52%   71.52%   +2.99%     
==========================================
  Files          44       44              
  Lines        2821     2946     +125     
==========================================
+ Hits         1933     2107     +174     
+ Misses        888      839      -49     
Flag Coverage Δ
ghcp 30.65% <13.72%> (-0.51%) ⬇️
integration 40.90% <56.86%> (+0.38%) ⬆️
ollama_local 32.51% <13.72%> (-0.28%) ⬇️
slow-browser 17.31% <10.78%> (-0.20%) ⬇️
slow-other 37.98% <31.37%> (-0.70%) ⬇️
unit 67.75% <100.00%> (+4.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/microbots/auto_memory/cli.py 100.00% <100.00%> (ø)
src/microbots/auto_memory/eval/swebenchverified.py 85.20% <100.00%> (+2.24%) ⬆️
src/microbots/llm/azure_openai_api.py 100.00% <100.00%> (ø)
src/microbots/llm/openai_api.py 100.00% <100.00%> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@KavyaSree2610

Copy link
Copy Markdown
Collaborator Author

Codecov Report

❌ Patch coverage is 86.27451% with 14 lines in your changes missing coverage. Please review. ✅ Project coverage is 71.04%. Comparing base (713b551) to head (e69707a). ⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/microbots/auto_memory/eval/swebenchverified.py 81.57% 7 Missing ⚠️
src/microbots/auto_memory/cli.py 40.00% 3 Missing ⚠️
src/microbots/llm/azure_openai_api.py 93.10% 2 Missing ⚠️
src/microbots/llm/openai_api.py 93.33% 2 Missing ⚠️
❌ Your patch status has failed because the patch coverage (86.27%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
Impacted file tree graph

@@            Coverage Diff             @@
##             main     #191      +/-   ##
==========================================
+ Coverage   68.52%   71.04%   +2.52%     
==========================================
  Files          44       44              
  Lines        2821     2946     +125     
==========================================
+ Hits         1933     2093     +160     
+ Misses        888      853      -35     

Flag Coverage Δ
ghcp 30.65% <13.72%> (-0.51%) ⬇️
integration 40.90% <56.86%> (+0.38%) ⬆️
ollama_local 32.51% <13.72%> (-0.28%) ⬇️
slow-browser 17.31% <10.78%> (-0.20%) ⬇️
slow-other 38.28% <31.37%> (-0.39%) ⬇️
unit 67.27% <86.27%> (+3.64%) ⬆️
Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/microbots/llm/azure_openai_api.py 97.43% <93.10%> (-2.57%) ⬇️
src/microbots/llm/openai_api.py 96.66% <93.33%> (-3.34%) ⬇️
src/microbots/auto_memory/cli.py 91.42% <40.00%> (-8.58%) ⬇️
src/microbots/auto_memory/eval/swebenchverified.py 81.63% <81.57%> (-1.33%) ⬇️
... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

Copilot add tests to make the patch coverage 100%

…aths

Co-authored-by: KavyaSree2610 <92566732+KavyaSree2610@users.noreply.github.com>

Copilot AI commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

[Codecov](https://app.codecov.io/gh/microsoft/microbots/pull/191?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microso...

Added tests covering the uncovered patch lines in a34a9da — patch coverage is now 100% locally:

src/microbots/auto_memory/cli.py                        35      0 100.00%
src/microbots/auto_memory/eval/swebenchverified.py     196     29  85.20%   86-93, 117-128, 152-161, 194-217, 384-390
src/microbots/llm/azure_openai_api.py                   78      0 100.00%
src/microbots/llm/openai_api.py                         60      0 100.00%

(the remaining swebenchverified.py misses are pre-existing lines outside this patch)

New tests:

  • test_ask_warns_when_token_usage_is_missing for both OpenAIApi and AzureOpenAIApi
  • test_parse_args_debug_http_defaults_to_false / test_main_enables_http_debug_logging_when_requested for the --debug-http flag
  • _generate_instance_feedback: success, bot-unavailable (exception) and bot-failure paths
  • _combine_result_feedback: empty-input short circuit and the bot-returned-failure fallback

@KavyaSree2610
Kavya Sree Kaitepalli (KavyaSree2610) merged commit a544976 into main Sep 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants