Skip to content

rules backtest: word an empty-side profit factor as insights does (#988) - #998

Merged
eaitbrahim merged 3 commits into
mainfrom
fix/988-backtest-empty-side
Oct 13, 2026
Merged

eaitbrahim merged 3 commits into
mainfrom
fix/988-backtest-empty-side

Conversation

@eaitbrahim

@eaitbrahim eaitbrahim commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Closes #988

What

keel rules backtest used to print stats.profit_factor raw. That meant Infinity for an all-wins sample, and a 0 that looks measured for an all-scratch sample (0/0) or for a sample with no closed trade. Since #986, keel insights summary and the Rules page word these cases instead. The backtest line now uses the same words:

sample engine value (unchanged) line before line now
all wins (incl. a zero-pnl win) Infinity / 0 profit_factor=Infinity / 0 profit_factor=n/a (no losses yet)
all scratches 0 profit_factor=0 profit_factor=n/a (no wins or losses yet)
no closed trade 0 win_rate=0.00% ... profit_factor=0 expectancy_px=0 max_drawdown_px=0 win_rate=n/a ... profit_factor=n/a expectancy_px=n/a max_drawdown_px=n/a (bare n/a for every figure, as render_summary prints at n=0)
all losses 0 profit_factor=0 profit_factor=0 (a measurement, unchanged)
both sides ratio ratio ratio (unchanged)

How

  • keel/commands/insights.py: _profit_factor_reason(n_trades, avg_win, avg_loss) is now the single rule. It finds the empty side and drops "no wins yet", because an all-losses 0 is measured. RuleTrackRecord.profit_factor_reason calls it, and the new profit_factor_reason_of(stats) applies it to a BacktestResult. Each side is decided by counting trades through _side_average, as build_rule_track_record does.
  • keel/commands/rules.py: at n=0 every unmeasured figure prints bare n/a. Otherwise the line prints undefined_reading(reason) when there is a reason, and the engine's own value when there isn't.

This changes the display only. BacktestResult, the gates and the console's returned stats are unchanged, and tests pin that. rules backtest has no --json.

Tests (TDD)

All new tests are in tests/commands/test_rules_services.py. They drive the real run_rule_backtest and replace only the engine run:

  • Parametrized profit-factor test: cases are all wins, zero-pnl win, all scratches, no trades, all losses and mixed. It asserts the exact parsed field and that the engine value is untouched.
  • n=0 test: asserts every figure on the line.
  • Parity test: for each empty-side shape, the rendered field equals undefined_reading(RuleTrackRecord.profit_factor_reason) for the same stats.

Six mutations were each killed by these tests: raw line, no n=0 guard, zero-average sides, no win_rate guard, no px guard, and no no-wins filter.

Follow-up filed: #1000. The keel simulate edge table's R profit factor still prints inf / 0.000; its docstring calls that deliberate, so it was left out of scope.

🤖 Generated with Claude Code

The `keel rules backtest` line printed the engine's profit_factor raw: `Infinity` for
all wins and a measured-looking `0` for all scratches or no closed trade. It now prints
`undefined_reading` over the reason #986 gave the track record -- `n/a (no losses yet)`,
`n/a (no wins or losses yet)`, `n/a (no trades yet)` -- through one shared
`insights.profit_factor_reason_of(stats)`, whose sides are decided by counting trades
like `build_rule_track_record`. An all-losses 0 is a measurement and keeps its number.
The returned BacktestResult and the gates are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@eaitbrahim eaitbrahim left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 review (#988).

Checked: the helper reuses _empty_side, _side_average and undefined_reading, and adds no wording of its own. RuleTrackRecord.profit_factor_reason now goes through the same _profit_factor_reason, so it cannot drift. Open trades are fine, because _side_average counts only win/loss outcomes and n_trades counts only closed trades. A win with pnl 0 reads n/a (no losses yet) over an engine 0, which matches insights. backtest_resolved always returns summarize output, so a hand-built BacktestResult (trades=[], n>0, which would read as no wins or losses yet) cannot reach this line. Mutation check: reverting the f-string to {stats.profit_factor} fails 3 of the 5 parametrized cases (all-wins, all-scratch, no-trades). Other surfaces: the web Rules page reads record.profit_factor_reason, the strategy console renders this same line, and research, trials and walk-forward print no profit factor.

[minor] Outside this diff: the simulate report's edge table (keel/sim/report.py:1008, _r_cell) still prints inf for an all-wins profit_factor_r and 0.000 for an all-scratch R sample. _r_cell's docstring says inf is deliberate, and the field is in R rather than money, so this is follow-up material and does not block this PR. It is the last surface that prints the engine's empty-side convention raw.

Comment thread keel/commands/rules.py Outdated
Comment thread tests/commands/test_rules_services.py Outdated
…ves the line

The rules backtest line prints bare n/a for win_rate, profit_factor,
expectancy_px and max_drawdown_px when no trade closed, as insights summary
does. The parity test now runs all-wins, all-scratch and a zero-pnl win
through the real line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@eaitbrahim eaitbrahim left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 2. Both round-1 fixes are real. At n=0 the line prints bare n/a for win_rate, profit_factor, expectancy_px and max_drawdown_px, matching render_summary. The parity test now drives _backtest_line_over. I ran six mutants on a scratch copy and each one failed at least one new test: raw profit factor, no n=0 guard, deciding sides by avg or None instead of counting, no n=0 guard on win_rate, no n=0 guard on max_drawdown_px, and dropping the no-wins filter. The tests assert parsed fields, not substrings.

[minor] The PR body is stale after 52f0f99:

  • The table's "no closed trade" row still says the line now prints profit_factor=n/a (no trades yet). It prints bare profit_factor=n/a.
  • The body never says that win_rate, expectancy_px and max_drawdown_px also become n/a at n=0. That is a visible change to the line.
  • The Tests section lists five cases. It omits the zero-pnl-win case and the separate n=0 every-figure test.
  • It still describes the parity test as comparing the backtest reason with the record's reason. It now compares the rendered line.

The reviewer and the squash commit read this body, so please update it.

Comment thread keel/commands/insights.py Outdated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@eaitbrahim eaitbrahim left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 3: no findings

@eaitbrahim
eaitbrahim merged commit f269f81 into main Oct 13, 2026
4 checks passed
@eaitbrahim
eaitbrahim deleted the fix/988-backtest-empty-side branch October 13, 2026 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rules backtest: profit_factor prints Infinity / 0 for an empty side, unlike insights summary since #986

1 participant