[server] Prevent WAL memory pool deadlocks

[platinumhamburg]

Purpose

Linked issue: close #4290

Brief change log

Tests

API and Format

Documentation

[Copilot]

🟡 Changes recommended

Unresolved critical page-ownership validation and moderate initialization-cleanup issues remain.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds scoped WAL memory allocations and deadlock recovery to prevent KV writes from stalling when the shared pool is exhausted.

Changes:

• Adds allocation ownership, lifecycle cleanup, and deadlock victim handling.
• Integrates scoped allocations into WAL writes.
• Adds regression tests for exhaustion, contention, rollback, and retries.

Review findings:

• Critical (1 vote): MemoryAllocation.java must validate that all bulk-returned pages belong to the allocation.
• Moderate (1 vote): ArrowWalBuilder.java must release pages acquired before builder construction fails.

File summaries

File	Description
fluss-server/src/test/java/org/apache/fluss/server/kv/KvTabletWalMemoryTest.java	Tests WAL exhaustion, rollback, contention, and retry correctness.
fluss-server/src/main/java/org/apache/fluss/server/kv/wal/ArrowWalBuilder.java	Handles Arrow writer initialization cleanup.
fluss-server/src/main/java/org/apache/fluss/server/kv/KvWriteProcessor.java	Applies scoped memory allocation to WAL writes.
fluss-common/src/test/java/org/apache/fluss/memory/MemoryAllocationTest.java	Tests allocation behavior and recovery.
fluss-common/src/main/java/org/apache/fluss/memory/MemorySegmentPool.java	Exposes scoped allocation creation.
fluss-common/src/main/java/org/apache/fluss/memory/MemoryAllocation.java	Tracks page ownership and cleanup.
fluss-common/src/main/java/org/apache/fluss/memory/LazyMemorySegmentPool.java	Detects and resolves allocation deadlocks.

Review details

Suppressed comments (1)

fluss-server/src/main/java/org/apache/fluss/server/kv/wal/ArrowWalBuilder.java:47

• If ManagedPagedOutputView is created successfully but MemoryLogRecordsArrowBuilder.builder(...) then throws, this catch only recycles the ArrowWriter; the initial page already acquired by outputView is leaked because construction never returns and callers cannot invoke deallocate(). Release outputView.allocatedPooledSegments() in this failure path as well, while handling the case where outputView creation itself failed.

        } catch (Throwable t) {
            writer.close();
            throw t;

• Files reviewed: 7/7 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.