Skip to content

fix: read message and snapshot state uncached - #76

Merged
cardmagic merged 1 commit into
mainfrom
fix/stale-reads-under-query-cache
Sep 22, 2026
Merged

cardmagic merged 1 commit into
mainfrom
fix/stale-reads-under-query-cache

Conversation

@cardmagic

Copy link
Copy Markdown
Owner

Why

MessageReference#status, MessageReference#result, and ActorSnapshot read
through the Active Record query cache.

A caller that polls holds one query cache for the whole poll, and the worker that
finishes the message is a different process, so its write cannot clear that
cache. The reader keeps returning its first answer.

This reaches any poll inside one executor block: a controller action, an Active
Job, or a rails runner script. Rails opens a query cache for each of those.

Reproduced in a real application

A Rails 8.1 app running bundle exec solid_objects start in a separate process:

message = SubscriptionActor.ref("dave").async.start_trial

ActiveRecord::Base.cache do
  20.times { seen << message.status; sleep 0.3 }
end
inside query cache, statuses seen: ["ready"]
outside the cache, status is: "completed"
database says completed_at=2026-09-22 00:19:17 UTC

Twenty reads over six seconds, all "ready", while the row had been complete for
most of it.

What changed

def result
  Message.uncached { Message.find(id).result }
end

status, result, and the instance read in ActorSnapshot now bypass the query
cache. This is the convention already in the codebase for the same reason:
SynchronousInvocation wraps its polling loop in Message.uncached, and
SyncDiagnostics does the same. The public readers were simply missed.

Why the suite never caught it

The query cache is off in tests, so every existing test reads fresh. The new test
opens one explicitly and then writes from a second connection, because a write
on the reading connection clears that connection's cache and would make the test
pass without the fix:

ActiveRecord::Base.cache do
  assert_equal "ready", message.status

  from_another_connection do
    SolidObjects::ReadyMessage.where(message_id: message.id).delete_all
    SolidObjects::Message.find(message.id).update!(completed_at: Time.current)
  end

  assert_equal "completed", message.status
end

All three tests fail on main and pass here.

Validation

bundle exec rake passes. Steep reports no type error, Brakeman no warning.

Backend Result
SQLite 668 runs, 0 failures, 35 skips
PostgreSQL 18 668 runs, 0 failures, 27 skips
MySQL 8.4, mysql2 668 runs, 0 failures, 43 skips
MySQL 8.4, Trilogy 668 runs, 0 failures, 43 skips

main reports the same skip counts under the same conditions, measured back to
back, so this change skips nothing new. The absolute counts are higher than
earlier runs in this repository because the wake-up suites gate on services that
were not reachable during this session; the same branch reported 24 skips earlier
today with no code change, so treat CI as the authority on the absolute number.

Scope

No migration and no API change. Three reads, one behaviour: a public read of
durable state answers from the database rather than from the caller's cache.

MessageReference#status, MessageReference#result, and ActorSnapshot read
through the Active Record query cache. A caller that polls holds one
cache for the whole poll, and the worker that finishes the message is a
different process, so its write cannot clear that cache. The reader saw
its first answer forever.

This reaches any poll inside one executor block: a controller action, an
Active Job, or a rails runner script. A real application reproduced it,
reporting "ready" twenty times over six seconds while the row had
completed.

Read those three uncached, as SynchronousInvocation and SyncDiagnostics
already do for the same reason.

The suite never caught it because the query cache is off in tests. The
new test opens one, then writes from a second connection so the write
cannot clear it, which is what a worker process does.
@greptile-apps

greptile-apps Bot commented Sep 22, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

The PR appears safe to merge and directly addresses stale cross-process reads with focused regression coverage.

Summary

This PR ensures public message-state and actor-snapshot reads bypass Active Record’s query cache, allowing polling callers to observe writes committed by another process.

  • Wraps message status and result lookups in Message.uncached.
  • Reads actor instance state through Instance.uncached.
  • Adds cross-connection regression coverage for status, result, and snapshot refreshes.
  • Documents the query-cache fix in the changelog.

Reviews (1) · Last reviewed commit: "fix: read message and snapshot state unc..."

@cardmagic
cardmagic merged commit c1da8fc into main Sep 22, 2026
60 of 61 checks passed
@cardmagic
cardmagic deleted the fix/stale-reads-under-query-cache branch September 22, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant