Conjecture and refutation in Craftax

An agent that learns an explicit theory of its environment, a set of probabilistic rules, by designing and running its own experiments, without any reward. This repository has the code, the run scripts and the results of its experiments on Craftax-Classic (the JAX reimplementation of Crafter), the full Craftax game, and Crafter-OO (for the comparison with OneLife).

The method

For each kind of change it can observe, the agent keeps a short list of probabilistic rules. Each rule is an action and a conjunction of conditions, with the probability of the change when they hold. The conditions are observed features, plus counters and block classes the learner invents.

• Conjecture. On a schedule, and whenever a theory is refuted, rules are grown by sequential covering over all stored experience. A condition is kept only if it saves more bits in coding the observed outcomes than it costs to write down.
• Refutation. After every step, the rules pay −log₂ of the probability they gave what happened, against the entropy their own probabilities promised. When an effect's rules pay significantly more than they promised (a one-sided three-sigma test), they are refuted and refitted at once on all experience.
• Experiments. The agent plans through its current rules. It tests situations it has not tried, or where the theory is known to fail, and chains backward through the rules to reach the states those tests need.

No game knowledge is built in. craftax_wm/tests/test_no_game_knowledge.py checks that no entity of either game is named in the learner or the planner. An ablation that shuffles and renames every symbol scores about the same.

docs/craftax_active_learner_design.md describes the method as implemented, with pointers into the code. craftax_wm/RESULTS.md has every result with its numbers, including those not in the figures.

Results

Mean of seeds 1, 42 and 100; every learner is scored on the steps it chose itself.

• Prediction (Craftax-Classic test split, changed components predicted right one step ahead): 0.735 at 300k steps. The same rule learner acting at random reaches 0.659, a neural learner driven by ensemble disagreement 0.555, and DreamerV3 Plan2Explore 0.458. On the full game: 0.636 at 300k.
• Crafter score of its own episodes, without the reward: 8.9% at 300k and 11.2% at 1M. On the same symbolic observation, learners trained on the game's reward score less: PPO, PPO-RNN and PPO-RND 3.4–3.8% at 300k and 5.0–5.9% at 1M, and DreamerV3 2.6% at 300k. The reward-free explorers (RND, ICM, E3B, DreamerV3 Plan2Explore) stay at 0.7–1.4% at 300k.
• Crafter-OO, OneLife's evaluation (all 38 scenarios): mean reciprocal rank 0.471 at 100k steps. OneLife scores 0.466 from 455 steps plus an LLM's prior knowledge of Crafter.

Every learner at every budget: craftax_wm/figures/headline_table.md.

Layout

path	what
wm_core/	the environment-agnostic core: code lengths, the sequential-covering rule learner, the refutation test, the failure map
craftax_wm/rules_wm.py	the rule learner: conjecture, refutation, refits, latent counters, invented block classes
craftax_wm/rules_agent.py	the planner that designs experiments with the learned rules
craftax_wm/mob_model.py, replay.py	rules for creatures; the bounded, outcome-stratified replay
craftax_wm/common.py, schema_full.py	the observation schemas of Craftax-Classic and the full game
craftax_wm/harness.py, metrics.py	one interaction harness for every active learner, and one evaluator for every model
craftax_wm/plan2explore.py, train_nn_wm.py	the neural active learner and its random control; the offline neural reference
craftax_wm/baselines/	wrappers that run the published baselines unchanged: PPO, PPO-RNN, PPO-RND, ICM and E3B from the Craftax baselines; DreamerV3 and its Plan2Explore
craftax_wm/crafter_oo_env.py, onelife_eval.py	Crafter-OO as an environment, and OneLife's own evaluation of the learner
craftax_wm/ppo.py, collect.py	PPO policies used only to make test data, and the val/test splits
craftax_wm/mechanisms.py	mechanism recovery against ground truth written from each game's source (evaluation only)
craftax_wm/run_*.sh	the run scripts, all resumable
craftax_wm/results/	the result files of every run: per budget, the prediction scores, the Crafter score and achievements; the learned rule sets (*_rules.txt)
craftax_wm/figures/	the figures and the headline table, made from results/
craftax_wm/tests/, wm_core/tests/	the no-game-knowledge guard, schema round trips, chaining, the core's tests

Setup

Run everything from the repository root. The scripts use the environment craftax_wm_venv/ there.

python3 -m venv craftax_wm_venv
craftax_wm_venv/bin/pip install -r requirements.txt
craftax_wm_venv/bin/pip install "jax[cuda12]==0.10.2"     # on a GPU machine
craftax_wm_venv/bin/python -m pytest -q craftax_wm/tests wm_core/tests

DreamerV3 and OneLife need other versions of their dependencies. Their scripts build their own environments under craftax_wm/vendor/ on first use.

Quick start (CPU, about a minute)

Make a small test split from random play, then let the learner run for 3,000 self-chosen steps:

craftax_wm_venv/bin/python -m craftax_wm.collect --out quickstart/data --splits test --scale 0.05
craftax_wm_venv/bin/python -m craftax_wm.rules_wm --policy plan --seed 1 --budgets 1000,3000 \
  --test quickstart/data/test --out quickstart/results
cat quickstart/results/rules_plan_s1_rules.txt     # the rules it learned

quickstart/results/rules_plan_s1.json holds the scores at each budget.

Figures and tables from the shipped results

craftax_wm_venv/bin/python -m craftax_wm.plot_headline        # headline.png, headline_crafter*, crafter_published
craftax_wm_venv/bin/python -m craftax_wm.headline_table       # headline_table.{md,png}
craftax_wm_venv/bin/python -m craftax_wm.plot_ablations       # ablations*.{png,svg,pdf}
craftax_wm_venv/bin/python -m craftax_wm.summarize_ablations craftax_wm/results/ablations
(cd craftax_wm/figures && pdflatex approach.tex)               # the method figure

Rerunning the experiments

The scripts skip every budget whose results already exist, so move the shipped results aside first:

mv craftax_wm/results craftax_wm/results_paper

Every script is resumable: after an interruption, run it again. Each writes its results as each budget is scored. Runs use seeds 1, 42 and 100.

1. Test data (GPU). The val/test splits are not shipped. Each script trains PPO with the game's reward, only to get policies that play further than random. It then collects the splits from random play and from each saved policy. run_gpu.sh also trains the offline neural reference.

PPO_STEPS=3e8 bash craftax_wm/run_gpu.sh        # Craftax-Classic -> craftax_wm/data/
bash craftax_wm/run_full_gpu.sh                 # the full game -> craftax_wm/data_full/, plus its DreamerV3 and RND runs

Our Classic splits came from the policies at 10M, 50M, 201M and 293M steps (the PPO run was stopped at 293M). PPO_STEPS=3e8 gives the nearest schedule. Different policies give slightly different test data, so rerun scores can differ a little from the shipped ones.

2. The experiments. Each row lists the script and where its results land:

result	script	results in craftax_wm/results/
our learner and planner, Classic and the full game, to 300k (CPU)	bash craftax_wm/run_generic.sh	core2/, full/core2/
the same on Classic to 1M	GAMES=classic BUDGETS=10000,30000,100000,300000,500000,1000000 bash craftax_wm/run_generic.sh	core2/
the same rule learner acting at random (CPU)	bash craftax_wm/run2_rules.sh, bash craftax_wm/run_full_rules.sh	run2/, full/rules/
Crafter-OO and OneLife's evaluation (CPU; fetches OneLife, Crafter-OO and OneLife's released data)	bash craftax_wm/run_onelife.sh, then bash craftax_wm/run_onelife_generic.sh	crafter_oo/
ablations (CPU, val split)	CONFIGS="..." bash craftax_wm/run_ablations.sh, with the configurations listed in the script	ablations/<config>/
neural active learner and its random control (GPU)	bash craftax_wm/run2_gpu.sh; the stronger variant: bash craftax_wm/run2b_gpu.sh	run2/, run2b/
DreamerV3 Plan2Explore and its random control (GPU)	bash craftax_wm/run_dreamer_gpu.sh; at XL size: bash craftax_wm/run_dreamer_xl.sh	baselines/, baselines_xl/
RND without reward (GPU)	bash craftax_wm/run_rnd_gpu.sh	baselines/
ICM and E3B without reward (GPU)	REWARD=0 ALGOS="ppo_icm ppo_e3b" bash craftax_wm/run_reward_baselines.sh	baselines/
PPO, PPO-RNN and PPO-RND trained on the reward (GPU)	bash craftax_wm/run_reward_baselines.sh	baselines_reward/
DreamerV3 trained on the reward (GPU; after run_dreamer_gpu.sh has built its environment)	medium, per seed: craftax_wm/vendor/dv3_venv/bin/python -m craftax_wm.baselines.dreamer_p2e --policy reward --seed 1 --out craftax_wm/results/baselines_reward; XL, from its symbolic input and from images in its own Crafter setup: bash craftax_wm/run_dreamer_calibration.sh	baselines_reward/, baselines_reward_xl/
the Crafter paper's own runs, scored at our budgets	craftax_wm_venv/bin/python -m craftax_wm.published_crafter	published/

The rule-learner runs each take one CPU core and up to about 3 GB of memory. run_generic.sh, run_ablations.sh and run_onelife_generic.sh start several runs at once; set the number with PAR. For long jobs, bash craftax_wm/job.sh start NAME 'COMMAND' runs a command in a named screen session, logging to craftax_wm/artifacts/NAME.log. It also restarts unfinished jobs after a reboot; for that, its first use adds an @reboot line to your crontab.

Third-party code

Nothing third-party is redistributed except the Crafter sprites in craftax_wm/figures/icons/ (MIT, see the LICENSE there). The scripts fetch everything else at pinned versions:

• the official Craftax baselines (Matthews et al. 2024);
• DreamerV3 1.5.0 from PyPI (Hafner et al.);
• Crafter's published scores (Hafner 2021);
• Crafter-OO and OneLife with OneLife's released data (Khan et al. 2026).

Licence

MIT, see LICENSE.