Skip to content

compiler: Write the complex arithmetic header atomically - #3071

Open
ggorman wants to merge 1 commit into
mainfrom
fix-complex-header-race
Open

ggorman wants to merge 1 commit into
mainfrom
fix-complex-header-race

Conversation

@ggorman

@ggorman ggorman commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

_complex_includes writes the extra complex arithmetic definitions a C++ language needs (def-complex) to complex_arith.h in the JIT cache directory, with open(..., 'w'), on every lowering of an Operator that uses a complex dtype. The directory is shared by every process compiling as the same user, and 'w' truncates the file before writing it, so with pytest -n auto one worker can truncate the header while another's compiler is reading it. The compile then fails on the first complex expression, e.g. no match for 'operator*' (operand types are 'int' and 'const std::complex<float>'); in DevitoPRO's CUDA backend the same race shows as user-defined literal operator not found for 1.0_if.

Changes:

  • The header is keyed on its content (complex_arith_<sha1>.h), written to a private temporary file and renamed into place, and skipped when it already exists. A reader therefore sees either no file or a complete one, and two languages with different definitions no longer overwrite each other.
  • test_complex_arith_header checks that two Operators share one header whose content is exactly def-complex.

Verification on main d7f53a3:

  • 32 processes each JIT-compiling 30 distinct complex-valued language='CXX' Operators from one cache directory: main fails 7 of 960 compiles (twice, 7 each) with the error above; this patch 0 of 960, with a single complex_arith_<sha1>.h in the directory and no leftover temporaries.
  • tests/test_dtypes.py: 47 passed, 5 skipped.

🤖 Generated with Claude Code

@ggorman ggorman added bug-py compiler CI continuous integration labels Oct 17, 2026
@ggorman
ggorman requested a review from JDBetteridge October 17, 2026 11:34
@codecov

codecov Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.23%. Comparing base (d7f53a3) to head (0f675e4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3071      +/-   ##
==========================================
- Coverage   84.23%   84.23%   -0.01%     
==========================================
  Files         258      258              
  Lines       57070    57089      +19     
  Branches     4874     4876       +2     
==========================================
+ Hits        48074    48087      +13     
- Misses       8169     8173       +4     
- Partials      827      829       +2     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX 68.51% <25.00%> (+0.01%) ⬆️
pytest-gpu-gcc- 79.01% <100.00%> (+<0.01%) ⬆️
pytest-gpu-icx- 78.93% <100.00%> (+<0.01%) ⬆️
pytest-gpu-nvc-nvidiaX 69.09% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-py CI continuous integration compiler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant