fix: lock the actor instance by primary key

[cardmagic]

Why

SolidObjects::Mailbox#enqueue held the actor instance row longer than it
needed to, and could deadlock on MySQL.

find_or_create_instance called Instance.create_or_find_by!, which inserts
first. For an actor that already exists, which is every message after the
first, each enqueue paid an insert that failed on the unique key, a transaction
restart, and then two locking reads. The second locking read came from
instance.lock!, on a row the transaction already held.

Statement counts

Measured with sql.active_record, for one enqueue against an actor that
already exists.

Database	Before	After
MySQL 8.4 (mysql2 and Trilogy)	12	10
PostgreSQL 18	12	10
SQLite	13	10

Statements that touch solid_objects_instances drop from 4 to 3. The failed
insert and the transaction restart are gone. The row lock is now held across 7
statements rather than 8.

SQLite started one higher because it cannot restart a transaction in place. It
issued ROLLBACK and a second BEGIN where the servers issue
ROLLBACK AND CHAIN.

The deadlock

A duplicate-key insert leaves a shared lock on idx_so_instances_identity.
MySQL keeps that lock across a savepoint rollback, so the following
SELECT ... FOR UPDATE asks to upgrade shared to exclusive on the same index
record. Two callers that do this at once deadlock.

The old code escaped this only by accident. Active Record restarts the parent
transaction instead of opening a savepoint when that parent has not written
yet, and a restart releases the lock. Any caller that had already written got a
real savepoint and no release. The executor, the reminder scheduler, and the
effect executor all write before they enqueue.

On unmodified main, eight concurrent callers that each hold a written
transaction and create the same actor produce four ActiveRecord::Deadlocked
failures. test/integration/enqueue_test.rb covers this and fails on main
for that reason.

What changed

• Read the instance row without a lock, then lock it by primary key. Locking by
  primary key also keeps the enqueue path off the identity index, which is
  where the shared locks collect.
• Drop the second instance.lock!. Every path out of find_or_create_instance
  now returns a locked row, and a row that disappears still raises
  ActiveRecord::RecordNotFound, which with_instance_retry handles.
• Keep a savepoint on the create path. A failed statement poisons an open
  transaction on PostgreSQL, which this gem supports.
• After a duplicate key, read the winning row in shared mode on MySQL, then
  lock it by primary key. MySQL defaults to repeatable read, so a consistent
  read cannot see a row committed after the snapshot. The shared read costs
  nothing there because the failed insert already holds that shared lock.
  PostgreSQL and SQLite take no shared lock: PostgreSQL defaults to read
  committed and needs no current read, and a share lock there creates the very
  upgrade deadlock it prevents on MySQL. This is the new shared_lock adapter
  method, shaped like the existing claim_lock.

Tests

• test/integration/enqueue_statement_count_test.rb is new. It asserts a
  steady-state enqueue issues no insert to solid_objects_instances, opens one
  transaction and never restarts it, touches the instance row three times, and
  issues a fixed total. All four assertions fail on main.
• test/integration/enqueue_test.rb gains concurrent enqueue to an actor that
  already exists, concurrent creation from callers that already wrote, and the
  retry limit that raises ActorDestroyed. The existing create race now also
  asserts exactly one instance row.

Each new test was confirmed to fail without the change. The deadlock test fails
three times out of three on main with Trilogy.

Validation

bundle exec rake test passes on all four supported combinations. Skip counts
match main exactly, so nothing became a silent skip.

Backend	Before	After
SQLite	658 runs, 0 failures, 24 skips	665 runs, 0 failures, 24 skips
PostgreSQL 18	658 runs, 0 failures, 16 skips	665 runs, 0 failures, 16 skips
MySQL 8.4, mysql2	658 runs, 0 failures, 32 skips	665 runs, 0 failures, 32 skips
MySQL 8.4, Trilogy	658 runs, 0 failures, 32 skips	665 runs, 0 failures, 32 skips

bundle exec rake standard rubocop rbs steep security passes. Steep reports no
type error and Brakeman reports no warning. sig/generated/ is regenerated.

Compatibility

No migration and no API change. shared_lock is additive and defaults to
nil. A host application that runs PostgreSQL at repeatable read gets a
RecordNotFound retry on a lost create race rather than a shared read, which
with_instance_retry already absorbs.

Not in this change

One lock acquisition per message in the drain is left alone deliberately. Note
that the count is about two per message, not one: Activation#claim_next_message
and Executor#complete each open their own fenced_transaction. A 39-message
drain issues 81 FOR UPDATE statements against the instance row. Batching
would change failure isolation, attempt counting, and poison-message ordering,
and it would not make the lock fair, which is the actual cause of the observed
starvation. It belongs in its own change.

Release

This branch also cuts 0.15.2. lib/solid_objects/version.rb, CHANGELOG.md,
and Gemfile.lock are updated in a separate commit, and bundle exec rake
passes on the released tree. Tag v0.15.2 after this merges to publish.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no outstanding correctness, security, or repository-rule violations remain.

Summary

This PR changes mailbox enqueueing to look up actor instances before insertion and lock existing instances by primary key, avoiding MySQL’s duplicate-key lock-upgrade deadlock while reducing steady-state SQL work. It also releases version 0.15.2.

• Adds an adapter-specific shared lock for MySQL create-race recovery.
• Removes the redundant second row lock from the enqueue path.
• Adds concurrency, retry-limit, and SQL statement-count coverage.
• Inlines the single-use create helper while preserving the required @rbs signatures.
• Updates release metadata and documentation.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Enqueue actor message] --> B[Read instance ID by identity]
    B -->|Found| C[Lock instance by primary key]
    B -->|Missing| D[Create instance inside savepoint]
    D -->|Created| E[Return locked created instance]
    D -->|Duplicate identity| F[Read winning ID]
    F -->|MySQL| G[Use shared current read]
    F -->|Other adapters| H[Use ordinary read]
    G --> C
    H --> C
    C --> I[Deduplicate and allocate sequence]
    E --> I

Loading

_{Reviews (3) · Last reviewed commit: "refactor: inline the create race into th..."}

[cardmagic]

@greptileai review

[cardmagic]

@greptileai review