Skip to content

Add explicit asynchronous submit/poll offload API - #34

Open
byrnedj wants to merge 1 commit into
intel:mainfrom
byrnedj:async-api-upstream
Open

byrnedj wants to merge 1 commit into
intel:mainfrom
byrnedj:async-api-upstream

Conversation

@byrnedj

@byrnedj byrnedj commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

This adds a public dto.h with an explicit submit/poll API so applications can overlap DSA data movement with their own CPU work:

dto_submit_memcpy / dto_submit_memset / dto_submit_memcpy_crc /
dto_submit_crc -> dto_async_poll -> dto_async_crc_val
dto_batch_op_new / dto_submit_batch_copy / dto_batch_poll (up to 64 copies)

The operation state is caller-allocated (opaque dto_async_op) so it can be polled from any thread. Every submit takes a flags argument: DTO_SUBMIT_CC directs output toward the CPU cache when the device supports it, and DTO_SUBMIT_BOF makes the device block on page faults instead of aborting (requires a work queue configured with block-on-fault). Without BOF a faulting operation surfaces as DTO_ASYNC_FAILED and the caller redoes it on the CPU; submission returns DTO_ASYNC_FALLBACK when DSA is unavailable or the request is out of range, with nothing copied. CRC results follow the raw CRC32C convention (seed 0, no final inversion), matching _mm_crc32.

The memset entry point takes (dest, c, n) like the C library, replacing the earlier page-oriented helper so the offload API is uniform across operations.

Adds seven functional tests (copy, memset, crc, copy+crc, no-BOF fault contract, argument validation, batch) that pass both on DSA hardware and on the CPU-fallback path.

Example use:

#include <dto.h>

  dto_async_op op;                      /* plain stack variable, 192 B */

  int rc = dto_submit_memcpy_crc(&op, dst, src, n,
                                 DTO_SUBMIT_CC | DTO_SUBMIT_BOF);
  if (rc == DTO_ASYNC_SUBMITTED) {
          do_work();          /* overlap with the DSA */

          while ((rc = dto_async_poll(&op)) == DTO_ASYNC_PENDING)
                  _mm_pause();          /* or yield, or check later */

          if (rc == DTO_ASYNC_DONE) {
                  uint32_t crc = dto_async_crc_val(&op);
          } else {                      /* DTO_ASYNC_FAILED */
                  memcpy(dst, src, n);
                  uint32_t crc = my_sw_crc32c(src, n);
          }
  } else {                              /* DTO_ASYNC_FALLBACK: nothing copied */
          memcpy(dst, src, n);
          uint32_t crc = my_sw_crc32c(src, n);
  }

libdto so far only accelerated memcpy/memmove/memset/memcmp transparently
via LD_PRELOAD. This adds a public dto.h with an explicit submit/poll API so
applications can overlap DSA data movement with their own CPU work:

  dto_submit_memcpy / dto_submit_memset / dto_submit_memcpy_crc /
  dto_submit_crc  -> dto_async_poll -> dto_async_crc_val
  dto_batch_op_new / dto_submit_batch_copy / dto_batch_poll (up to 64 copies)

The operation state is caller-allocated (opaque dto_async_op) so it can be
polled from any thread. Every submit takes a flags argument: DTO_SUBMIT_CC
directs output toward the CPU cache when the device supports it, and
DTO_SUBMIT_BOF makes the device block on page faults instead of aborting
(requires a work queue configured with block-on-fault). Without BOF a
faulting operation surfaces as DTO_ASYNC_FAILED and the caller redoes it on
the CPU; submission returns DTO_ASYNC_FALLBACK when DSA is unavailable or
the request is out of range, with nothing copied. CRC results follow the
raw CRC32C convention (seed 0, no final inversion), matching _mm_crc32.

The memset entry point takes (dest, c, n) like the C library, replacing the
earlier page-oriented helper so the offload API is uniform across operations.

Adds seven functional tests (copy, memset, crc, copy+crc, no-BOF fault
contract, argument validation, batch) that pass both on DSA hardware and on
the CPU-fallback path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant