Repository navigation
Keep the H.264 PPS id readable in E2EE keyframes - #2181
Conversation
Chrome reads pic_parameter_set_id from an H.264 IDR slice before the decrypt transform runs, and drops the keyframe if the id names an unknown PPS. Encoders such as Intel Quick Sync code slice_type in 7 bits, so the id lies past the 2 unencrypted slice bytes and gets encrypted. The sender now encrypts again with a fresh IV, up to 16 times, until the sent bytes still read as the true PPS id. The receiver takes the IV from the frame trailer, so the wire format and receivers do not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: fb14286 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
size-limit report 📦
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
writeRbsp takes about 95% of an encryption attempt on a large keyframe. The retry loop escaped the whole frame on each attempt, but the PPS id check needs only the first 16 bytes. Escape those for the check, and escape the whole frame once, after the loop. A retry then costs only the AES-GCM encryption. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1egoman
left a comment
1egoman
left a comment
There was a problem hiding this comment.
👍 Would it be worth adding some sort of e2e test for this?
|
Thanks for the quick fix in #2181. We tested it before release on a build of the
We have production participants on Intel laptops with E2EE, so this fix matters to us. Is there an expected date for the 2.24.0 release (#2158)? |
With E2EE and H.264, Chrome subscribers drop keyframes from publishers that use Intel Quick Sync, and video stalls for seconds. The e2ee worker leaves only the first 2 bytes of the slice NALU unencrypted. Chrome reads
pic_parameter_set_idfrom an IDR slice before the decrypt transform runs. Quick Sync codesslice_typein 7 bits, so the PPS id is the first encrypted bit. The receiver reads a random id and drops the frame. In #2180, 48 of 94 Quick Sync keyframes read as the correct PPS id 0, and one run stalled for 21 s. Fixes #2180.Change
The sender keeps the encrypted bits readable as the true PPS id. The receiver takes the IV from the frame trailer, so the sender can pick any fresh IV, and receivers need no change.
parseH264IdrPpsId()readsfirst_mb_in_slice,slice_typeandpic_parameter_set_idafter RBSP unescaping, the way Chrome does.makeIV, up toMAX_PPS_ID_ENCRYPT_ATTEMPTS(16).writeRbspescapes the whole frame once, after the loop. A retry costs only the AES-GCM encryption.Delta frames and VideoToolbox keyframes, where the id sits in the plaintext byte, need no extra encryption. For Quick Sync, one bit must match, so a keyframe takes 2 encryptions on average.
A longer plaintext prefix that ends after
pic_parameter_set_idalso fixes the bug, but I dropped it because it changes the wire format. Old JS receivers and the native SDKs (payload_start_offset + 2inwebrtc-sdk/webrtc) then fail to decrypt.Failure case
A slice can have more than one PPS id bit past the plaintext byte, for example with a non-zero PPS id. Each extra bit halves the chance that an attempt matches. If all 16 attempts miss, the frame goes out as the last attempt produced it. That is the behavior without this change.
Verified
npx vitest run src/e2ee/workerpasses, 80 tests. The newparseH264IdrPpsIdtests use the Quick Sync and VideoToolbox bytes from E2EE with H.264: encrypted pic_parameter_set_id makes Chrome drop keyframes from some encoders #2180, a non-zero id, a non-IDR slice and a truncated slice.FrameCryptortest encrypts 16 Quick Sync keyframes. It checks that each one reads as PPS id 0 and decrypts to the original frame. WithMAX_PPS_ID_ENCRYPT_ATTEMPTSset to 1, the test fails on the first frame that reads as a non-zero PPS id.npx tsc --noEmit, prettier and eslint pass. CItestpasses.Not done yet
slice_pic_parameter_set_idis always encrypted. It is not known whether Chrome reads it before the transform.webrtc-sdk/webrtchave the same bug. That fix is a separate change in that repo.🤖 Generated with Claude Code