Installing Dependencies

Make sure you have Python and Poetry installed (Poetry is a Python package manager).

If you don't have Poetry installed:

sudo apt update -y
sudo apt install -y pipx
pipx install poetry

# Optional
poetry config virtualenvs.in-project true

From the repo root:

poetry install

to install all required dependencies.

Input data

Forecast CSVs live under local/private_input_data/ (gitignored). For Summer 2026 the simulation script expects:

• local/private_input_data/pro_forecasts_2026_summer.csv
• local/private_input_data/bot_forecasts_2026_summer.csv

You can:

• If you have access, download prior analysis outputs (and sometimes inputs) from this Google Drive folder, or
• Request access to analysis data from the instructions shared here which will show you how to use the Metaculus Data Needs Form

Running the analysis

For the current (Summer 2026) tournament:

poetry run python aib_analysis/run_summer_simulations.py

That writes JSON artifacts to local/summer_2026_simulations/.

Check logs/latest_info.log for warnings (annulled questions, weight mismatches, unmatched questions, missing control bots, etc.).

For older seasons (Q1–Q4), use aib_analysis/run_q1_q2_simulations.py and edit the __main__ paths/folders as needed.

Viewing results

poetry run streamlit run aib_analysis/front_end.py

In the UI, set the tournaments folder to the simulation output path (e.g. local/summer_2026_simulations). Restart Streamlit or use Clear cache in the ⋮ menu after changing underlying JSON or loading/scoring code.

To get a markdown results summary (leaderboards, Pro vs bot head-to-head, per-type and team-size breakdowns, charts) from the same JSON outputs:

poetry run python aib_analysis/utils/summarize_season_results.py --simulations-folder local/summer_2026_simulations --season-name "Summer 2026"

That writes results_summary.md, CSV leaderboards, and PNG charts to local/summer_2026_simulations/results_summary/.

Pros vs bots write-up

To draft the season's results post from the same simulation outputs:

poetry run python -m aib_analysis.pro_vs_bot_writeup.run --season spring_2026 --html

It reads local/<season>_simulations/ and writes local/<season>_pro_vs_bot_writeup/: writeup.md (precommitted result sentences plus [===Fill me===] blocks with reminders), writeup_preview.md (the same text with charts embedded, for viewers that won't load local images), PNG charts, CSV leaderboards, and pro_name_key.csv (maps Pro aliases to usernames; keep it local). Add a new season, and anything the data can't provide, in aib_analysis/pro_vs_bot_writeup/season_config.py.

Structure

The project is built around SimulatedTournament (see data_models.py / simulated_tournament.py). It is initialized from Forecast objects and derives users, scores, etc. Most analyses are filtered, aggregated, or combined tournaments built from those forecasts.