Skip to content

Real libraries #79

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
saidctb merged 6 commits into from
Sep 28, 2026
Merged

Real libraries #79

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b74b279
codex: restructure the Reference BLAS example guide
saidctb Sep 28, 2026
e3ffd01
codex: restructure the LAPACK example guide
saidctb Sep 28, 2026
3326edc
codex: restructure the MINPACK example guide
saidctb Sep 28, 2026
b9bcc99
codex: compare MINPACK's SciPy-exposed solvers with SciPy
saidctb Sep 28, 2026
2c87e4c
codex: restructure the FFTPACK example guide
saidctb Sep 28, 2026
1d0ea2f
codex: restructure the BSPLINE-FORTRAN example guide
saidctb Sep 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ release tags add a leading `v` to the package version.

## Unreleased

- Improve the Reference BLAS example guide
- Improve the LAPACK example guide
- Improve the MINPACK example guide
- Compare the MINPACK example's eight SciPy-exposed solvers with SciPy
- Improve the FFTPACK example guide
- Improve the BSPLINE-FORTRAN example guide
- Improve the Open MPI `mpi_f08` tutorial
- Improve the PRIMA example guide
- The README and the documentation homepage now open with a short terminal demo.
Expand Down
285 changes: 149 additions & 136 deletions docs/user/examples/fortran/blas-wrapper.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,94 +9,94 @@ publication: reviewed

# Build and Validate the Reference BLAS with PRIK

This example builds the official Reference BLAS sources as two importable
Python extension modules:
This example wraps all 155 routines of the Reference BLAS twice, once with
PRIK and once with NumPy's f2py, and checks both against independent
mathematical results. BLAS is compiled once into a shared library that both
wrappers link, so a difference between them comes from the wrapper, never
from the numerics.

- one generated by **PRIK**
- one generated by **NumPy’s f2py**
### What you get

It compares both wrappers with independent mathematical results across all 155
callable Reference BLAS routines.

### What this example shows

- Build PRIK and f2py wrappers against the same compiled BLAS library.
- Call vector and matrix routines with NumPy arrays.
- Compare numerical results and the Python interfaces produced by each tool.

You should already be comfortable with NumPy arrays, basic packaging, and building Fortran extensions.
- Two extension modules, `prik_reference_blas` and `f2py_reference_blas`, each
exposing the same 155 routines: Level 1 vector, Level 2 matrix-vector, and
Level 3 matrix-matrix operations, in real and complex precisions, including
packed, banded, symmetric, Hermitian, and triangular storage.
- A named test for every routine that checks PRIK and f2py against an
independent formula and against each other.

---

## Versions used

| Component | Version / source |
| --- | --- |
| PRIK | current repository checkout |
| Reference BLAS | snapshot shipped in Netlib LAPACK 3.12.1 |
| Python | 3.12 in the dedicated CI job |
| NumPy / f2py | NumPy 2.5.1 |
| Meson | 1.11.2 |
| Ninja | 1.13.0 |
| Fortran compiler | GNU Fortran 13 in CI; a compatible `gfortran` works locally |
## Quick start

> **Note:** f2py is part of NumPy.
> On Python 3.12 it uses the Meson backend, which is why Meson and Ninja are required.
From a PRIK checkout with PRIK installed, GNU Fortran on `PATH`, and the pinned
NumPy, Meson, and Ninja (see [Set up a clean environment](#set-up-a-clean-environment)):

## Tested platforms
```bash
source examples/fortran/blas/build_all.sh
python3 -m pytest -q examples/fortran/blas/tests
```

The Real Libraries Portability workflow builds and runs this example with
Python 3.12 on:
The first command builds both wrappers and puts them on `PYTHONPATH` for this
shell; use `source`, not `bash`, so that setting survives. The second runs the
complete comparison.

| Operating system | Architectures | Native toolchain |
| --- | --- | --- |
| Linux | x86-64, ARM64 | GNU Fortran 13 + GCC 13 |
| macOS | Intel, ARM64 | GNU Fortran 13 + GNU GCC 13 |
After this, both modules import in the same shell:

The ordinary numerical suite runs on all four targets. The maintainer
full-surface audit also runs on Linux x86-64.
```python
import numpy as np
import f2py_reference_blas
import prik_reference_blas

For everyday use of this example, prefer the checked-in sources in
`examples/fortran/blas/native/`.
(See the [Source provenance](#source-provenance) section at the end if you want to verify the upstream archive yourself.)
x = np.array([2.0, -4.0, 1.0])
y = np.array([3.0, 5.0, -2.0])
prik_reference_blas.daxpy(np.int32(3), np.float64(-1.5), x, np.int32(1), y, np.int32(1))
print(y) # [ 0. 11. -3.5]
```

---

## 1. Prepare the repository and toolchain
## Key files

Clone PRIK, create a virtual environment, and install the same Python build
tools used by the dedicated CI job:
Everything lives under
[`examples/fortran/blas/`](../../../../examples/fortran/blas/):

```bash
git clone https://github.com/PyNumLab/prik.git
cd prik
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install -e ".[qa]" \
"numpy==2.5.1" "meson==1.11.2" "ninja==1.13.0"
```
| File | What it does |
| --- | --- |
| [`native/`](../../../../examples/fortran/blas/native/) | The 155 Reference BLAS sources from Netlib LAPACK 3.12.1; the build downloads nothing. |
| [`build_prik.sh`](../../../../examples/fortran/blas/build_prik.sh) | Compiles BLAS into one shared library and builds the PRIK wrapper against it. |
| [`blas.pyf`](../../../../examples/fortran/blas/blas.pyf) | The reviewed f2py signature file for all 155 routines. |
| [`build_f2py.sh`](../../../../examples/fortran/blas/build_f2py.sh) | Builds the f2py wrapper from `blas.pyf`, linked to the same library. |
| [`build_all.sh`](../../../../examples/fortran/blas/build_all.sh) | Runs both build scripts and adds both modules to `PYTHONPATH`. |
| [`routine_inventory.py`](../../../../examples/fortran/blas/routine_inventory.py) | The authoritative list of routines, grouped by BLAS level and kind. |
| [`tests/`](../../../../examples/fortran/blas/tests/) | One test file per routine family, plus [`test_routine_coverage.py`](../../../../examples/fortran/blas/tests/test_routine_coverage.py), which fails if a routine is missing from the sources, inventory, exports, or tests. |
| [`tests/helpers.py`](../../../../examples/fortran/blas/tests/helpers.py) | The small comparison helpers used by every test. |

Install GNU Fortran separately. On Ubuntu:
---

```bash
sudo apt-get update
sudo apt-get install --yes gfortran
gfortran --version
```
## How the build works

All remaining commands run from the repository root with the virtual
environment active.
BLAS is compiled once. Both wrappers link that one shared library:

The runnable material is self-contained in the repository's
[`examples/` directory](../../../../examples/). After PRIK and the listed tools
are installed, you can copy that directory alone.
```text
155 BLAS sources ──compile once──> libprik_full_blas
│
PRIK API, read from the sources ─────────┼──> prik_reference_blas
f2py API, from blas.pyf ─────────────────┴──> f2py_reference_blas
```

---
1. `examples.native_library` compiles the 155 sources into
`libprik_full_blas` and prints its path.
2. PRIK reads the same sources to generate its Python API, compiles no BLAS
source itself (`--no-compile-input-sources`), and links the library.
3. f2py builds its wrapper from the reviewed `blas.pyf` and links the same
library.

## 2. Compile BLAS once and build the PRIK wrapper
Both wrappers are built with `-O0`, so the comparison checks correctness
rather than optimization-dependent results. This example is not a performance
benchmark. `build_all.sh` runs these two scripts; each can also be reused on
its own.

Run the first build script from the repository root:
**The PRIK build**, from `build_prik.sh`:

<!-- prik-doc-source: examples/fortran/blas/build_prik.sh -->
```bash
Expand All @@ -122,20 +122,7 @@ python -m prik "$EXAMPLE_WORKSPACE/examples/fortran/blas/native" \
--wrapper-c-flags="-O0 -g0"
```

`examples.native_library` compiles all 155 implementations and returns the
resulting shared-library path. PRIK reads the same source directory to build
the Python API, skips native implementation compilation, and links that
library.

`-O0` keeps the PRIK and f2py correctness builds equivalent and avoids making
optimization-dependent claims. This example focuses on correctness, not
performance.

---

## 3. Build f2py against the same native library

Run the same f2py build script exercised by the test suite:
**The f2py build**, from `build_f2py.sh`:

<!-- prik-doc-source: examples/fortran/blas/build_f2py.sh -->
```bash
Expand All @@ -161,54 +148,28 @@ python -m numpy.f2py -c \
--opt=-O0
```

The committed [`blas.pyf`](../../../../examples/fortran/blas/blas.pyf) defines the f2py
interface. f2py compiles only its wrapper and links it to
`BLAS_SHARED_LIBRARY`, so both wrappers exercise the same compiled BLAS
implementations.

A few routines expose scalar writebacks differently through the two wrappers;
the comparison below shows those return-value differences explicitly.

Import both modules:

```python
import os
import sys

build_root = os.environ["BLAS_BUILD_ROOT"]
sys.path.insert(0, f"{build_root}/prik")
sys.path.insert(0, f"{build_root}/f2py")

import f2py_reference_blas
import prik_reference_blas
```

### Important wrapper difference

PRIK deliberately follows the native scalar contract:

- For a subroutine such as `DAXPY`, arrays are mutated in place and PRIK returns the visible input scalars because they are treated as inout.
- The f2py comparison module also mutates the output array but returns `None`.
- Function routines such as `DDOT` return their numerical result through both wrappers.
Everything is written to the temporary `BLAS_BUILD_ROOT` directory, not to the
repository.

---

## 4. Run the complete test suite
## PRIK and f2py differences

Build both wrappers and run the 155-routine suite:
Both wrappers mutate output arrays in place and compute identical results.
They differ in what a call returns:

```bash
source examples/fortran/blas/build_all.sh
python3 -m pytest -q examples/fortran/blas/tests
```
| Routine | PRIK returns | f2py returns |
| --- | --- | --- |
| A subroutine such as `daxpy` | The visible scalar arguments, which it treats as in-out: `(n, alpha, incx, incy)` | `None` |
| A function such as `ddot` | The result and the visible scalars: `(result, n, incx, incy)` | The result alone |
| The 6 rotation routines, whose scalars have no Fortran `intent` | The scalar writebacks directly | Typed NumPy 0-D arrays, because `blas.pyf` records those scalars as `intent(inout)` |

The tests cover vector, matrix, packed, banded, symmetric, Hermitian, and
triangular operations. Each routine is called with representative inputs and
checked against an independent mathematical result.
PRIK follows the native argument list and scalar contract; the f2py signature
file is the reviewed comparison interface.

---

## 5. See how results are validated
## How results are validated

Each comparison checks three relationships:

Expand All @@ -223,10 +184,8 @@ leading dimensions, and unused storage where they are part of a routine's
contract. The independent formula or residual remains the primary numerical
reference.

The two examples below come directly from the runnable suite and use its small
NumPy comparison helpers.

#### Test helper conventions
The two examples below come directly from the runnable suite and use its
helpers:

- `assert_allclose_for_dtype` compares floating-point results with a tolerance
matched to their NumPy dtype. Its optional `operation_size` is a
Expand Down Expand Up @@ -259,8 +218,8 @@ def test_daxpy(prik_blas, f2py_blas):
assert_storage_unchanged(f2py_x, x)
```

Both wrappers must mutate `y` to the expected value.
The input-only array `x` must remain unchanged.
Both wrappers must mutate `y` to the expected value. The input-only array `x`
must remain unchanged.

### DDOT – scalar function result

Expand Down Expand Up @@ -288,31 +247,85 @@ def test_ddot(prik_blas, f2py_blas):

---

## 6. Run focused examples
## Run the tests

After building the wrappers, run a family or one routine:
Run the complete suite, one family, one routine, or every test that mentions a
routine name:

```bash
python3 -m pytest -q examples/fortran/blas/tests
python3 -m pytest -q examples/fortran/blas/tests/test_level1_real.py
python3 -m pytest -q examples/fortran/blas/tests/test_level1_real.py::test_daxpy
python3 -m pytest -q examples/fortran/blas/tests -k dgemm
```

- Complete Level-1 examples → [`test_level1_real.py`](../../../../examples/fortran/blas/tests/test_level1_real.py)
- Matrix / packed / banded / symmetric / Hermitian / triangular examples → files under [`examples/fortran/blas/tests/`](../../../../examples/fortran/blas/tests/)
- Public routine list → [`routine_inventory.py`](../../../../examples/fortran/blas/routine_inventory.py)
- Routine coverage check → [`test_routine_coverage.py`](../../../../examples/fortran/blas/tests/test_routine_coverage.py)

For the copyable build scripts, test commands, and source provenance, see the
[`examples/fortran/blas` project README](../../../../examples/fortran/blas/README.md).
The tests cover vector, matrix, packed, banded, symmetric, Hermitian, and
triangular operations, each called with representative inputs and checked
against an independent result.

---

## Set up a clean environment

Clone PRIK, create a virtual environment, and install the same Python build
tools used by the dedicated CI job:

```bash
git clone https://github.com/PyNumLab/prik.git
cd prik
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install -e ".[qa]" \
"numpy==2.5.1" "meson==1.11.2" "ninja==1.13.0"
```

Install GNU Fortran separately. On Ubuntu:

```bash
sudo apt-get update
sudo apt-get install --yes gfortran
gfortran --version
```

Run the example's commands from the repository root with the virtual
environment active.

## Versions used

| Component | Version / source |
| --- | --- |
| PRIK | current repository checkout |
| Reference BLAS | snapshot shipped in Netlib LAPACK 3.12.1 |
| Python | 3.12 in the dedicated CI job |
| NumPy / f2py | NumPy 2.5.1 |
| Meson | 1.11.2 |
| Ninja | 1.13.0 |
| Fortran compiler | GNU Fortran 13 in CI; a compatible `gfortran` works locally |

f2py is part of NumPy. On Python 3.12 it uses the Meson backend, which is why
Meson and Ninja are required.

## Tested platforms

The Real Libraries Portability workflow builds and runs this example with
Python 3.12 on:

| Operating system | Architectures | Native toolchain |
| --- | --- | --- |
| Linux | x86-64, ARM64 | GNU Fortran 13 + GCC 13 |
| macOS | Intel, ARM64 | GNU Fortran 13 + GNU GCC 13 |

The numerical suite runs on all four targets. A maintainer full-surface audit
also runs on Linux x86-64.

## Troubleshooting

- Confirm that `gfortran`, `meson` and `ninja` are on your `PATH`.
- On Python 3.12+, do **not** force the old distutils backend of f2py. Use the
pinned Meson and Ninja setup shown above.
- Confirm that `gfortran`, `meson`, and `ninja` are on your `PATH`.
- On Python 3.12 or later, do not force f2py's old distutils backend. Use the
pinned Meson and Ninja shown above.
- Use `source examples/fortran/blas/build_all.sh`; running it with `bash`
starts a child shell, so the exported `PYTHONPATH` is lost.
- Run a single failing test with more detail and keep the build directory:

```bash
Expand Down
Loading
Loading