Repository navigation
Use scipy_openblas with Fortran 77 code in a Python wheel #254
Description
tl;dr: to use scipy_openblas32 with Fortran 77 code, do I need my own preprocessor mapping from "normal" to scipy_openblas symbols, e.g. -DDGETRF=SCIPY_DGETRF ?
I'm a maintainer of Slycot, which is a thin Python library around the Fortran 77 library SLICOT. One of our long-standing issues is to provide binary wheels on PyPI for Linux, Windows, and macos.
I've made a model of the problem at https://github.com/roryyorke/f77ex , which is a Python project with Fortran 77 code calling DGETRF. It's scikit-core-build-based, and successfully makes a manylinux binary wheel with bundled OpenBLAS.
To get this to work I needed to add compile-time flags -cpp -DDGETRF=SCIPY_DGETRF.
My questions:
- Am I doing this the hard way? Can I copy a solution from some other project? I have looked at Scipy's build code, but found it hard to follow.
- If a "normal"->"scipy_openblas" mapping is needed would this preprocessor option be best, or should I rather rename symbols in the object files using
objcopy? Or is there something else?
Any other advice, especially around getting this to work on Windows, would be gratefully received.
Activity
mattip commentedon Feb 21, 2026
We don’t currently have a good story for users outside SciPy and NumPy. I guess we could provide a scipy-openblas.h header to properly do the magic mapping like SciPy and NumPy.
Give me a week or so to work on something.
rgommers commentedon Feb 21, 2026
The goal here is to use the scipy-openblas wheels from F77 code right? Then a regular header won't work, that'd be more suitable for C/C++.
If so: the trick we used in SciPy was to codegen new
.ffiles which map the regular symbol names to the mangled ones (e.g.,dgemm->scipy_dgemm_including all the upper/lowercase and trailing underscore yes/no flavors. It's actually fairly elegant. The original code was introduced in scipy/scipy#11302, later rewritten and extended several times for ILP64, handling arbitrary suffixes, etc.I don't have an example of the codegen'd files at hand, but if the above sounds right, I could try and dig it up. From there, writing a Python script to do the codegen specifically for scipy-openblas would probably be pretty clean, it'll end up much simpler than the SciPy machinery.
roryyorke commentedon Feb 22, 2026
Thank you both.
Yes.
My untested assumption was that for C one would use "original" openblas headers (I guess there's risk of version mismatch though?), and the pkgconfig or CMake files supplied with the wheel, which have flags like
-DBLAS_SYMBOL_PREFIX=scipy_, and-DBLAS_SYMBOL_SUFFIX=64_for scipy-openblas64.That PR helps, thank you. For I'm focusing on getting a proof of concept to work, and it looks like I could use the approach in the PR to generate two checked-in files (
scipy_openblas_blas.incandscipy_openblas_lapack.inc) and feed them to gfortran's-includeoption when compiling the Fortran77 code.I'll try that and give feedback.
rgommers commentedon Feb 22, 2026
If you only use a few functions in slycot, and only need the exact mapping to
scipy-openblas32symbol names, you might even just write that.incfile manually.roryyorke commentedon Mar 1, 2026
Summary
Using information from Scipy, I was able to use cibuildwheel to create Linux and Windows Python 3.13 wheels embedding the scipy-openblas libraries. I first did this using preprocessing, and then using post-build objcopy symbol redefinition. The wheels pass the Slycot unit tests.
preprocessor: https://github.com/roryyorke/Slycot/releases/tag/cibuildwheel-1
objcopy: https://github.com/roryyorke/Slycot/releases/tag/cibuildwheel-2
Context
SLICOT is just under 270klines of Fortran 77 code, and uses about 200 of the 1783 symbols I generated OpenBlas remappings for.
Mappings
I took the build info from Scipy 1.17.1, specifically the Cython binding files and the exclusions, and combined all the information into one set of symbols mappings.
While testing I found SLICOT also uses dlamc3 which is not exposed in Scipy: from reading its docs this function forces writes to memory, maybe something like C's volatile keyword. It is provided by OpenBLAS, and I added it to the symbol mappings.
I removed the
xerblasymbol mapping: in Slycot we want this to call our own, no-op xerbla routine.Preprocessing approach
gfortran doesn't accept
-include, but does have@, with which one can feed it an options file containing '-DDGETRF=SCIPY_DGETFRF` and so on for all 1783 symbols.I needed to add compile option
-ffixed-line-length-132as the preprocessing step increased some line lengths beyond the (80 character?) limit, which caused compilation errors.After reading some warnings about applying the C preprocessor to Fortran code in the gcc manual, I decided to try post-build objcopy manipulation.
objcopy approach
I'm using CMake; I got it to assemble the SLICOT code into a static (".a") library, and applied an
objcopycommand to this library with--redefine-syms.--redefine-syms' is fed a file containing the 1783 symbol mappings asdgetrf_ scipy_dgetrf_`, etc.Concluding
My opinion is that the objcopy approach is less risky and is marginally simpler to add to the scikit-build-core (i.e., CMake) based approach I'm working on.
Thanks again!
roryyorke commentedon Mar 22, 2026
Final note for any future readers: GNU objcopy doesn't work on macos (only tried ARM); objconv doesn't work on macos ARM; and I didn't try llvm-objcopy.
I've switched back to the preprocessor approach, and have managed to produce "repaired" wheels on Linux, Windows, and MacOS with cibuildwheel.
mattip commentedon Mar 22, 2026
Glad you solved it. It would be nice to link to your work or add a howto to the README here.
roryyorke commentedon May 1, 2026
I've put a trivial example project here: https://github.com/roryyorke/f77ex . The write-up is long - sorry!
The PR for Slycot incorportaing this is python-control/Slycot#252