tests: Make the MPI test timeout configurable

[ggorman]

The parallel marker launches each MPI test as mpiexec --oversubscribe --timeout 300 .... The cap is there to stop a hung test blocking CI, but it is hard-coded, and on the DevitoPRO CI runners (pytest -n auto, eight workers with OMP_NUM_THREADS=4 on one cpuset) the heaviest four-rank tests already run that long: test_abox.py::TestIsotropicAcoustic::test_backward_mpi[4] took 271 s in a passing job and 302 s in the next one, which mpiexec aborted (devitopro run 38969858835, main). Both versions of the test spend their time waiting for CPU, not computing; the serial variant takes 6 to 8 s.

Changes:

• The timeout is read from DEVITO_MPI_TEST_TIMEOUT, defaulting to 600 s. A hang is still caught within ten minutes; a slow runner can raise it without touching the code.

Verification on main d7f53a3:

• DEVITO_MPI_TEST_TIMEOUT=3 pytest "tests/test_mpi.py::TestDistributor::test_partitioning": both ranks report Timeout: 3 seconds and the test fails, as intended.
• With the default: 2 passed.

Why the tests are slow in the first place: the ranks and the OpenMP threads of every worker spin-wait while oversubscribed. Reproduced on DevitoPRO main with tests/test_abox.py and tests/test_fft.py -k "TestIsotropicAcoustic or stream_out", eight xdist workers with OMP_NUM_THREADS=4 pinned to 16 CPUs (a shared, loaded host, so relative numbers only):

	default	OMP_WAIT_POLICY=passive	passive + OMPI_MCA_mpi_yield_when_idle=1
subset wall time	1214 s	788 s	736 s
slowest abox MPI test	179 s	60 s	52 s
test_fft_stream_out median	13.1 s	6.7 s	8.0 s

The DevitoPRO CI environment change is a separate PR there; this one keeps a hang from waiting longer than ten minutes while giving legitimately slow runners room.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.23%. Comparing base (d7f53a3) to head (a759739).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3073   +/-   ##
=======================================
  Coverage   84.23%   84.23%           
=======================================
  Files         258      258           
  Lines       57070    57071    +1     
  Branches     4874     4874           
=======================================
+ Hits        48074    48075    +1     
  Misses       8169     8169           
  Partials      827      827           

Flag	Coverage Δ
pytest-gpu-aomp-amdgpuX	68.51% <100.00%> (+0.01%)	⬆️
pytest-gpu-gcc-	79.00% <100.00%> (+<0.01%)	⬆️
pytest-gpu-icx-	78.92% <0.00%> (+<0.01%)	⬆️
pytest-gpu-nvc-nvidiaX	69.08% <100.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[JDBetteridge]

Creating DEVITO_MPI_TEST_TIMEOUT is no better than adding a global variable with a magic value and further complicates running the test suite. A better approach would be to make it an optional kwarg to the marker and only update it for tests that need it.