Bayesian Search

[andrewdalpino]

A companion to Grid Search. Uses Optuna-style TPE to iteratively propose the next set of hyper-parameters over n trials.

[Copilot]

Copilot review overview

🟡 Changes recommended

There are correctness/performance issues in candidate generation and search-space sizing that can lead to wasted trials and severe memory usage on large spaces.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 3 · 2

Open (5)

• Validate hyper-parameter values are serializable during construction · New
• Avoid materializing the full Cartesian product to compute search space size · New
• Avoid returning an already evaluated candidate after resampling fails · New
• Correct base() documentation to show the non-null Estimator return type · New
• Import Euclidean and Manhattan in the Bayesian Search example · New

What changed in this PR

Introduces a new BayesianSearch meta-estimator (TPE-style sampler) as a companion to GridSearch, including PHPUnit coverage and documentation updates to expose the new tuning option in the docs navigation.

Changes:

• Added Rubix\ML\BayesianSearch implementing sequential Bayesian hyper-parameter optimization with a startup random-search phase and result reporting (scores(), results(), best()).
• Added end-to-end PHPUnit tests validating training, best-trial selection, result ordering, and argument validation behavior.
• Added documentation pages/links describing Bayesian Search and comparing it to Grid Search.

File	Description
src/​BayesianSearch.php	New Bayesian hyper-parameter search meta-estimator using TPE-style sampling.
tests/​Base/​BayesianSearchTest.php	New PHPUnit coverage for BayesianSearch behavior and validation.
docs/​bayesian-search.md	New documentation page for Bayesian Search usage and API.
docs/​hyper-parameter-tuning.md	Added Bayesian Search section and example.
mkdocs.yml	Added Bayesian Search to docs navigation under Meta Estimators.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.