Skip to content

fix(ascend): keep unsafe operators out of device graphs - #568

Open
baominghelly wants to merge 1 commit into
refactor/adopt-modern-infini-stackfrom
fix/ascend-graph-capture-safety
Open

fix(ascend): keep unsafe operators out of device graphs#568
baominghelly wants to merge 1 commit into
refactor/adopt-modern-infini-stackfrom
fix/ascend-graph-capture-safety

Conversation

@baominghelly

Copy link
Copy Markdown
Contributor

Summary

  • Mark Ascend HCCL AllReduce as unsafe for device-graph capture in csrc/infinicore/src/ops/distributed/allreduce.cc.
  • Mark Ascend MultiheadAttentionVarlen as unsafe for device-graph capture because its provider synchronizes cumulative sequence lengths on the host.
  • Add static runtime-contract tests covering both graph segmentation decisions.

Motivation

Ascend graph replay produced corrupted model output when host-dependent attention and HCCL collective operations were captured into device-graph segments. The graph runtime already supports splitting operator sequences by GraphOperator::is_device_graph_capture_safe(); this change uses that existing mechanism to execute the two currently unsafe Ascend operator classes eagerly between captured segments.

This is a correctness fallback rather than native graph support for those operators. Once the relevant backends expose validated graph-capture capability, InfiniCore should query that capability instead of checking the device type here.

No issue is currently linked to this change.

Type of Change

  • feat — new feature / new model
  • fix — bug fix
  • perf — performance improvement (no behavioral change)
  • refactor — code restructuring without behavior change
  • test — adding or fixing tests only
  • docs — documentation only
  • build / ci — build system or CI configuration
  • chore — tooling, formatting, or other non-code changes
  • Breaking change

Test Results of Involved Models on Supported Platforms (Please attach screenshots)

Validated on Ascend 910C with CANN 9.0.0, the modern InfiniRT/InfiniOps stack, InfiniOps paged flash-attention PR #984, and InfiniLM refactor/adopt-modern-infini-stack.

Single-request graph tests used examples/test_infer.py with --device ascend --enable-paged-attn --attn=flash-attn --enable-graph.

ChatGLM3-6B TP1: exit 0; coherent self-introduction; segmented graph runtime enabled
InternLM3-8B-Instruct TP1: exit 0; coherent response; segmented graph runtime enabled
MiniCPM4-8B TP1: exit 0; coherent model introduction; segmented graph runtime enabled
ChatGLM3-6B TP2: exit 0 in 53 s; communicator established; segmented graph runtime enabled; coherent self-introduction

The final-head TP2 log is /workspace/model-matrix-route-a-20260910/infinilm-graph-pr-final/chatglm3_tp2_graph.log on the validation host.

Benchmark / Performance Impact

N/A. This is a correctness change. No performance claim is made.

Notes for Reviewers

  • AllReduce remains capture-safe on all existing platforms except Ascend; other platform behavior is unchanged.
  • MultiheadAttentionVarlen was already excluded on NVIDIA and is now also excluded on Ascend.
  • The long-term replacement for these device checks is an explicit backend capability query after HCCL graph capture/replay and host-free attention metadata handling are validated.
  • Diagnostic changes to graph.cc were intentionally excluded.

CI / ChatOps

CI does not run automatically on pull requests. It should be triggered manually from the Actions tab or by a maintainer with /retest or /test.


Checklist

Title, Branch, and Commits

  • PR title follows Conventional Commits.
  • Branch name follows <type>/xxx-yyyy-zzzz.
  • Each commit message follows Conventional Commits.
  • Small PR is a single squashable commit.
  • Branch is rebased cleanly on the current target branch.
  • No fixup! / squash! / wip commits remain.
  • Existing PR/branch/commit follows the current contribution format.

Scope and Design

  • Changes are minimal and limited to Ascend graph segmentation safety.
  • No dead code, commented-out blocks, debug prints, or unrelated formatting churn.
  • No unrelated formatting churn.
  • No public API changes.

General Code Hygiene

  • Code is self-explanatory; comments explain non-obvious backend contracts.
  • Every modified file ends with a single trailing newline.
  • No trailing whitespace, tab/space mixing, or stray BOMs.
  • Identifiers in added comments and messages use Markdown backticks where appropriate.
  • Added comments and messages are in English.
  • Added comments and messages follow language conventions.

C++ Specific

  • Code follows the existing C++ style.
  • Error and warning wording follows existing conventions.
  • Constructor initializer-list order matches member declaration order.
  • No new raw new/delete was introduced.
  • Changed files pass scripts/format.py.
  • No changes to csrc/models/llama_legacy/.

Python Specific

  • Test code follows PEP 8.
  • Added Python code is formatted by scripts/format.py.
  • No docstring changes were needed.
  • No changes to python/infinilm/auto_config.py.

Testing

  • Affected Ascend platform tested; other platforms are unchanged by this diff.
  • Passed single-request tests with graph mode enabled.
  • Offline performance test not run; this correctness change makes no performance claim.
  • Sanity suite not run; targeted graph and full static contract validation covered the changed runtime contracts.
  • Service test not run; no server behavior was changed.
  • Build and install passed on Ascend.

Build, CI, and Tooling

  • _infinilm rebuilt and installed successfully on Ascend.
  • CI still needs to be triggered manually.

Documentation

  • No user-facing build flag, API, or workflow documentation change is needed.

Security and Safety

  • No secrets, access tokens, internal URLs, customer data, or personal hardware identifiers were committed.
  • No third-party code was introduced.
  • No unsafe pointer arithmetic, uninitialized reads, or missing bounds checks were introduced.

Validation Evidence

python scripts/format.py --check --path <four changed files>
# passed

python -m unittest discover -s test/static -p 'test_*.py' -q
Ran 106 tests in 0.169s
OK

xmake build -r -j16 _infinilm
xmake install -y _infinilm
# build ok, install ok

python test/static/test_infinicore_runtime_contracts.py \
  InfiniCoreRuntimeContractsTest.test_ascend_hccl_allreduce_stays_out_of_device_graph \
  InfiniCoreRuntimeContractsTest.test_ascend_attention_with_host_lengths_stays_out_of_device_graph
# Ran 2 tests, OK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant