fix: read message and snapshot state uncached - #76
Merged
Merged
Conversation
MessageReference#status, MessageReference#result, and ActorSnapshot read through the Active Record query cache. A caller that polls holds one cache for the whole poll, and the worker that finishes the message is a different process, so its write cannot clear that cache. The reader saw its first answer forever. This reaches any poll inside one executor block: a controller action, an Active Job, or a rails runner script. A real application reproduced it, reporting "ready" twenty times over six seconds while the row had completed. Read those three uncached, as SynchronousInvocation and SyncDiagnostics already do for the same reason. The suite never caught it because the query cache is off in tests. The new test opens one, then writes from a second connection so the write cannot clear it, which is what a worker process does.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
MessageReference#status,MessageReference#result, andActorSnapshotreadthrough the Active Record query cache.
A caller that polls holds one query cache for the whole poll, and the worker that
finishes the message is a different process, so its write cannot clear that
cache. The reader keeps returning its first answer.
This reaches any poll inside one executor block: a controller action, an Active
Job, or a
rails runnerscript. Rails opens a query cache for each of those.Reproduced in a real application
A Rails 8.1 app running
bundle exec solid_objects startin a separate process:Twenty reads over six seconds, all
"ready", while the row had been complete formost of it.
What changed
status,result, and the instance read inActorSnapshotnow bypass the querycache. This is the convention already in the codebase for the same reason:
SynchronousInvocationwraps its polling loop inMessage.uncached, andSyncDiagnosticsdoes the same. The public readers were simply missed.Why the suite never caught it
The query cache is off in tests, so every existing test reads fresh. The new test
opens one explicitly and then writes from a second connection, because a write
on the reading connection clears that connection's cache and would make the test
pass without the fix:
All three tests fail on
mainand pass here.Validation
bundle exec rakepasses. Steep reports no type error, Brakeman no warning.mainreports the same skip counts under the same conditions, measured back toback, so this change skips nothing new. The absolute counts are higher than
earlier runs in this repository because the wake-up suites gate on services that
were not reachable during this session; the same branch reported 24 skips earlier
today with no code change, so treat CI as the authority on the absolute number.
Scope
No migration and no API change. Three reads, one behaviour: a public read of
durable state answers from the database rather than from the caller's cache.