Skip to content

fix(detector): parse labeled AI score - #44

Open
mikemikimike wants to merge 2 commits into
ferrix-lab:mainfrom
mikemikimike:fix/parse-ai-score
Open

mikemikimike wants to merge 2 commits into
ferrix-lab:mainfrom
mikemikimike:fix/parse-ai-score

Conversation

@mikemikimike

Copy link
Copy Markdown

Description

Fixes the AI/LLM probability score parser so unrelated numbers in generated explanations cannot override the model's score.

  • Parse scores only from labeled AI_SCORE, Score, or Probability lines.
  • Keep the neutral default score of 5 when a valid labeled score is missing or out of range.
  • Require Gemini responses to end with a dedicated AI_SCORE line.
  • Add regression coverage for score fractions, unrelated numbers, and invalid scores.

Issue: #22

Fixes #22


Type of Change

  • Bug fix (non-breaking change fixing an issue)
  • Tests (adding missing tests or fixing existing tests)

Validation

  • pytest tests/ (6 passed)
  • ruff check .
  • python -m py_compile setup.py insight/*.py
  • python -m build --sdist --wheel

Checklist

  • My code follows the style guidelines of this project (PEP 8 / ruff).
  • I have performed a self-review of my own code.
  • I have added tests that prove my fix is effective.
  • All new and existing tests pass locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(detector): AI/LLM probability score heuristic produces inverted and random scores

1 participant