diff --git a/CHANGELOG.md b/CHANGELOG.md index cc942fc9..1154a81a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,15 +4,20 @@ ### New +- **WebP decoding**: lossy, lossless, with alpha and animated (#19, #20). Still images decode with `Image.decode()`, + animated ones with `Animation.decode()`. The output is bit-exact with libwebp (checked against libwebp's test + vectors and their reference checksums). +- **WebP encoding** (lossless): `Image.encodeWebP()`, typically 20-30% smaller than PNG. - **APNG** decoding and encoding (#6): `Animation.decode()` reads animated PNGs (broken animations fall back to the default image, like browsers do), `Animation.encodeAPNG()` writes them with full alpha. `Image.decode()` still returns an APNG's default image. -- **`Animation`**, the base class of `GIF`: an array of frames that decodes GIF and APNG (`Animation.decode()`) and - encodes `encodeGIF()` and `encodeAPNG()`. `GIF` keeps its API. +- **`Animation`**, the base class of `GIF`: an array of frames that decodes GIF, APNG and WebP + (`Animation.decode()`) and encodes `encodeGIF()`, `encodeAPNG()` and `encodeWebP()`. `GIF` keeps its API. +- `decode()` returns animated WebPs as `Animation`; `ImageType` recognizes WebP (`isWebP()`, `getType() === 'webp'`). ### Deprecated -- **`GIF`**: use `Animation`, which works the same and also handles APNG (`new GIF(frames).encode()` becomes +- **`GIF`**: use `Animation`, which works the same and also handles APNG and WebP (`new GIF(frames).encode()` becomes `new Animation(frames).encodeGIF()`, `GIF.decode()` becomes `Animation.decode()`). `GIF` will be removed in 4.0. ## 3.0.0 diff --git a/README.md b/README.md index 05579bde..6ee78621 100644 --- a/README.md +++ b/README.md @@ -25,8 +25,9 @@ deno add jsr:@matmen/imagescript # Deno (npm:imagescript works too) - PNGs (grayscale, RGB, indexed colors, 1-16 bit, interlaced) with and without alpha channels - JPEGs (baseline and progressive; grayscale, RGB, CMYK) - TIFFs (strips and tiles; uncompressed, LZW, deflate, PackBits and JPEG compressed; BigTIFF) - - JPEGs, TIFFs and PNGs are rotated as their Exif (or, for PNGs, XMP) orientation says -- [Decoding animations](https://imagescript.matmen.dev/classes/Animation.html#decode) (GIF and APNG) with all + - WebPs (lossy, lossless and with alpha; bit-exact with libwebp) + - JPEGs, TIFFs, PNGs and WebPs are rotated as their Exif (or, for PNGs, XMP) orientation says +- [Decoding animations](https://imagescript.matmen.dev/classes/Animation.html#decode) (GIF, APNG and WebP) with all frames, durations and disposal modes - [Rendering SVGs](https://imagescript.matmen.dev/classes/Image.html#rendersvg) (shapes, paths, gradients, strokes, clip paths, masks, CSS; no text, images or filters) @@ -45,8 +46,10 @@ deno add jsr:@matmen/imagescript # Deno (npm:imagescript works too) , [dominantColor](https://imagescript.matmen.dev/classes/Image.html#dominantcolor), ...) - Encoding images as [PNGs](https://imagescript.matmen.dev/classes/Image.html#encode) (smallest lossless color type) , [JPEGs](https://imagescript.matmen.dev/classes/Image.html#encodejpeg) -- Encoding animations as [GIF](https://imagescript.matmen.dev/classes/Animation.html#encodegif) and - [APNG](https://imagescript.matmen.dev/classes/Animation.html#encodeapng) + and lossless [WebPs](https://imagescript.matmen.dev/classes/Image.html#encodewebp) +- Encoding animations as [GIF](https://imagescript.matmen.dev/classes/Animation.html#encodegif), + [APNG](https://imagescript.matmen.dev/classes/Animation.html#encodeapng) and lossless + [WebP](https://imagescript.matmen.dev/classes/Animation.html#encodewebp) --- @@ -84,7 +87,8 @@ The deflate/inflate implementation is derived from [fflate](https://github.com/1 [Fast CRC32 in JavaScript](https://gist.github.com/101arrowz/e58695f7ccfdf74f60ba22018093edea), both by Arjun Barrett. The JPEG codec is based in part on the work of the Independent JPEG Group (libjpeg) and on [pdf.js](https://github.com/mozilla/pdf.js), the glyph rasterizer is translated from -[font-rs](https://github.com/raphlinus/font-rs). See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for their licenses. +[font-rs](https://github.com/raphlinus/font-rs). The WebP codecs follow [RFC 9649](https://www.rfc-editor.org/rfc/rfc9649) and [RFC 6386](https://www.rfc-editor.org/rfc/rfc6386) +and reproduces [libwebp](https://chromium.googlesource.com/webm/libwebp)'s color conversion. See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for their licenses. --- diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 2247069e..65b279c5 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -10,6 +10,9 @@ following projects, under their own licenses. Each derived file says so in its h | [libjpeg](https://ijg.org/) / [libjpeg-turbo](https://libjpeg-turbo.org/) by the Independent JPEG Group | `src/codecs/jpeg.ts` (IDCT, FDCT, upsampling, color conversion, progressive refinement, optimal Huffman tables) | IJG | | [pdf.js](https://github.com/mozilla/pdf.js) (forked from [jpgjs](https://github.com/notmasteryet/jpgjs) by notmasteryet) | `src/codecs/jpeg.ts` (structure of the scan decoder) | Apache-2.0 | | [font-rs](https://github.com/raphlinus/font-rs) by Raph Levien | `src/render/text.ts` (the `Rasterizer` class, also used for SVGs) | Apache-2.0 | +| [RFC 9649](https://www.rfc-editor.org/rfc/rfc9649) (WebP Image Format) | `src/codecs/vp8l.ts` (lossless bitstream, short distance code table), `src/codecs/webp.ts` (container) | BSD-3-Clause (Revised BSD) | +| [RFC 6386](https://www.rfc-editor.org/rfc/rfc6386) (VP8 Data Format and Decoding Guide, with the libvpx reference decoder) | `src/codecs/vp8-tables.ts` (probability and quantizer tables), `src/codecs/vp8.ts` (decoding procedures) | BSD-3-Clause | +| [libwebp](https://chromium.googlesource.com/webm/libwebp) | `src/codecs/vp8.ts` (YUV to RGB conversion, "fancy" chroma upsampling), `src/codecs/webp.ts` (animation frame blending) | BSD-3-Clause | ## fflate, Fast CRC32 in JavaScript @@ -72,3 +75,70 @@ The scan decoder of `src/codecs/jpeg.ts` follows the structure of pdf.js' `src/c one decode function per scan type). The `Rasterizer` class in `src/render/text.ts` is a translation of font-rs' `Raster` from Rust to TypeScript (`draw_line`, `draw_quad`, `accumulate`); its `cubic` method is adapted from `draw_quad`. The full license text is in [licenses/Apache-2.0.txt](licenses/Apache-2.0.txt). + +## RFC 6386, libwebp + +The VP8 decoder follows the procedures of RFC 6386 and its reference decoder, its constant tables are extracted from +the RFC. To decode exactly like libwebp, the YUV to RGB conversion, the chroma upsampling and the blending of animation +frames reproduce libwebp's integer arithmetic. + +``` +RFC 6386 reference decoder: Copyright (c) 2010, 2011, Google Inc. All rights reserved. +libwebp: Copyright (c) 2010, Google Inc. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + + * Neither the name of Google nor the names of its contributors may + be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +``` + +The RFC's reference decoder additionally comes with Google's patent grant for VP8 (RFC 6386 section 20.27). + +## RFC 9649 + +The WebP container and the lossless (VP8L) codec follow RFC 9649, the short distance code table is taken from it. + +``` +Copyright (c) 2024 IETF Trust and the persons identified as authors of the code. All rights reserved. + +Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +following conditions are met: + +- Redistributions of source code must retain the above copyright notice, this list of conditions and the following + disclaimer. +- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the + following disclaimer in the documentation and/or other materials provided with the distribution. +- Neither the name of Internet Society, IETF or IETF Trust, nor the names of specific contributors, may be used to + endorse or promote products derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, +INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +``` diff --git a/src/animation.ts b/src/animation.ts index 5358aa70..0a88a086 100644 --- a/src/animation.ts +++ b/src/animation.ts @@ -1,11 +1,12 @@ import * as gifCodec from './codecs/gif.ts'; import * as png from './codecs/png.ts'; +import * as webp from './codecs/webp.ts'; import { BinaryData, view } from './util/mem.ts'; import { Image, ResizeMode } from './image.ts'; import { Frame } from './frame.ts'; import { ImageType } from './image-type.ts'; -// loop counts: ImageScript (like GIF) counts repetitions (-1 forever, 0 play once), APNG counts plays (0 forever) +// loop counts: ImageScript (like GIF) counts repetitions (-1 forever, 0 play once), APNG and WebP count plays (0 forever) const toPlays = (loopCount: number): number => loopCount < 0 ? 0 : loopCount + 1; const fromPlays = (plays: number): number => plays === 0 ? -1 : plays - 1; @@ -19,7 +20,7 @@ function canvasFrames(width: number, height: number, frames: Array<{ pixels: Uin } /** - * An animation: an array of frames that can be decoded from and encoded to GIF and APNG. + * An animation: an array of frames that can be decoded from and encoded to GIF, APNG and WebP. * Array methods like map and filter return plain arrays. */ export class Animation extends Array { @@ -149,6 +150,37 @@ export class Animation extends Array { return png.encodeAnimation(this.__apngFrames__(), this.width, this.height, toPlays(this.loopCount), compression); } + /** + * Encodes the animation as lossless animated WebP + */ + async encodeWebP(): Promise { + this.__validate__(); + if (!this.length) throw new Error('An animation needs at least one frame'); + const width = this.width, height = this.height; + + // WebP knows no "previous" disposal and stores offsets halved: such animations are stored as full frames + const fullFrames = this.some(f => f.disposalMode === 'previous' || (f.disposalMode === 'background' && (f.xOffset & 1 || f.yOffset & 1))); + if (fullFrames) { + const canvases = png.compositeAnimation({ width, height, frames: this.__apngFrames__() }); + return webp.encodeAnimation(canvases.map(c => ({ x: 0, y: 0, width, height, duration: c.duration, dispose: true, pixels: c.pixels })), width, height, toPlays(this.loopCount)); + } + + const frames = Array.from(this, frame => { + let { xOffset: x, yOffset: y, width: w, height: h } = frame; + let pixels = new Uint8Array(frame.bitmap.buffer, frame.bitmap.byteOffset, frame.bitmap.byteLength); + // odd offsets: one transparent pixel more on the left/top (blending makes it invisible) + if (x & 1 || y & 1) { + const dx = x & 1, dy = y & 1; + const padded = new Uint8Array((w + dx) * (h + dy) * 4); + for (let row = 0; row < h; row++) padded.set(pixels.subarray(row * w * 4, (row + 1) * w * 4), ((row + dy) * (w + dx) + dx) * 4); + pixels = padded; + x -= dx; y -= dy; w += dx; h += dy; + } + return { x, y, width: w, height: h, duration: frame.duration, dispose: frame.disposalMode === 'background', pixels }; + }); + return webp.encodeAnimation(frames, width, height, toPlays(this.loopCount)); + } + /** @internal frames in the APNG codec's terms (alpha blended, disposal 0 none, 1 background, 2 previous) */ __apngFrames__(): png.APNGFrame[] { const dispose = [0, 0, 2, 1]; @@ -165,7 +197,7 @@ export class Animation extends Array { } /** - * Decodes an animated GIF or APNG into fully composited frames (still images become a single frame) + * Decodes an animated GIF, APNG or WebP into fully composited frames (still images become a single frame) * @param input The binary data to decode * @param onlyExtractFirstFrame Whether to stop decoding after the first frame */ @@ -175,6 +207,12 @@ export class Animation extends Array { if (type === 'gif') return Animation.__fromGIF__(data, onlyExtractFirstFrame, Animation); + if (type === 'webp') { + const decoded = webp.decodeAnimation(data, onlyExtractFirstFrame); + const frames = canvasFrames(decoded.width, decoded.height, webp.composite(decoded)); + return new Animation(frames, fromPlays(decoded.loopCount)); + } + if (type === 'png') { const decoded = await png.decodeAnimation(data, onlyExtractFirstFrame); const frames = canvasFrames(decoded.width, decoded.height, png.compositeAnimation(decoded)); diff --git a/src/codecs/vp8-tables.ts b/src/codecs/vp8-tables.ts new file mode 100644 index 00000000..8cc6f7c4 --- /dev/null +++ b/src/codecs/vp8-tables.ts @@ -0,0 +1,146 @@ +/*! + * VP8 constant tables, extracted from RFC 6386 ("VP8 Data Format and Decoding Guide", sections 9.6, 11.5, 13.4, + * 13.5 and 14.1). Copyright (c) 2010, 2011 Google Inc., BSD license (see THIRD_PARTY_NOTICES.md) + */ + +// [above mode][left mode][9] probabilities of the subblock mode tree in key frames +export const KF_BMODE_PROBS = Uint8Array.of( + 231, 120, 48, 89, 115, 113, 120, 152, 112, 152, 179, 64, 126, 170, 118, 46, 70, 95, 175, 69, 143, 80, 85, 82, 72, 155, 103, + 56, 58, 10, 171, 218, 189, 17, 13, 152, 144, 71, 10, 38, 171, 213, 144, 34, 26, 114, 26, 17, 163, 44, 195, 21, 10, 173, + 121, 24, 80, 195, 26, 62, 44, 64, 85, 170, 46, 55, 19, 136, 160, 33, 206, 71, 63, 20, 8, 114, 114, 208, 12, 9, 226, + 81, 40, 11, 96, 182, 84, 29, 16, 36, 134, 183, 89, 137, 98, 101, 106, 165, 148, 72, 187, 100, 130, 157, 111, 32, 75, 80, + 66, 102, 167, 99, 74, 62, 40, 234, 128, 41, 53, 9, 178, 241, 141, 26, 8, 107, 104, 79, 12, 27, 217, 255, 87, 17, 7, + 74, 43, 26, 146, 73, 166, 49, 23, 157, 65, 38, 105, 160, 51, 52, 31, 115, 128, 87, 68, 71, 44, 114, 51, 15, 186, 23, + 47, 41, 14, 110, 182, 183, 21, 17, 194, 66, 45, 25, 102, 197, 189, 23, 18, 22, 88, 88, 147, 150, 42, 46, 45, 196, 205, + 43, 97, 183, 117, 85, 38, 35, 179, 61, 39, 53, 200, 87, 26, 21, 43, 232, 171, 56, 34, 51, 104, 114, 102, 29, 93, 77, + 107, 54, 32, 26, 51, 1, 81, 43, 31, 39, 28, 85, 171, 58, 165, 90, 98, 64, 34, 22, 116, 206, 23, 34, 43, 166, 73, + 68, 25, 106, 22, 64, 171, 36, 225, 114, 34, 19, 21, 102, 132, 188, 16, 76, 124, 62, 18, 78, 95, 85, 57, 50, 48, 51, + 193, 101, 35, 159, 215, 111, 89, 46, 111, 60, 148, 31, 172, 219, 228, 21, 18, 111, 112, 113, 77, 85, 179, 255, 38, 120, 114, + 40, 42, 1, 196, 245, 209, 10, 25, 109, 100, 80, 8, 43, 154, 1, 51, 26, 71, 88, 43, 29, 140, 166, 213, 37, 43, 154, + 61, 63, 30, 155, 67, 45, 68, 1, 209, 142, 78, 78, 16, 255, 128, 34, 197, 171, 41, 40, 5, 102, 211, 183, 4, 1, 221, + 51, 50, 17, 168, 209, 192, 23, 25, 82, 125, 98, 42, 88, 104, 85, 117, 175, 82, 95, 84, 53, 89, 128, 100, 113, 101, 45, + 75, 79, 123, 47, 51, 128, 81, 171, 1, 57, 17, 5, 71, 102, 57, 53, 41, 49, 115, 21, 2, 10, 102, 255, 166, 23, 6, + 38, 33, 13, 121, 57, 73, 26, 1, 85, 41, 10, 67, 138, 77, 110, 90, 47, 114, 101, 29, 16, 10, 85, 128, 101, 196, 26, + 57, 18, 10, 102, 102, 213, 34, 20, 43, 117, 20, 15, 36, 163, 128, 68, 1, 26, 138, 31, 36, 171, 27, 166, 38, 44, 229, + 67, 87, 58, 169, 82, 115, 26, 59, 179, 63, 59, 90, 180, 59, 166, 93, 73, 154, 40, 40, 21, 116, 143, 209, 34, 39, 175, + 57, 46, 22, 24, 128, 1, 54, 17, 37, 47, 15, 16, 183, 34, 223, 49, 45, 183, 46, 17, 33, 183, 6, 98, 15, 32, 183, + 65, 32, 73, 115, 28, 128, 23, 128, 205, 40, 3, 9, 115, 51, 192, 18, 6, 223, 87, 37, 9, 115, 59, 77, 64, 21, 47, + 104, 55, 44, 218, 9, 54, 53, 130, 226, 64, 90, 70, 205, 40, 41, 23, 26, 57, 54, 57, 112, 184, 5, 41, 38, 166, 213, + 30, 34, 26, 133, 152, 116, 10, 32, 134, 75, 32, 12, 51, 192, 255, 160, 43, 51, 39, 19, 53, 221, 26, 114, 32, 73, 255, + 31, 9, 65, 234, 2, 15, 1, 118, 73, 88, 31, 35, 67, 102, 85, 55, 186, 85, 56, 21, 23, 111, 59, 205, 45, 37, 192, + 55, 38, 70, 124, 73, 102, 1, 34, 98, 102, 61, 71, 37, 34, 53, 31, 243, 192, 69, 60, 71, 38, 73, 119, 28, 222, 37, + 68, 45, 128, 34, 1, 47, 11, 245, 171, 62, 17, 19, 70, 146, 85, 55, 62, 70, 75, 15, 9, 9, 64, 255, 184, 119, 16, + 37, 43, 37, 154, 100, 163, 85, 160, 1, 63, 9, 92, 136, 28, 64, 32, 201, 85, 86, 6, 28, 5, 64, 255, 25, 248, 1, + 56, 8, 17, 132, 137, 255, 55, 116, 128, 58, 15, 20, 82, 135, 57, 26, 121, 40, 164, 50, 31, 137, 154, 133, 25, 35, 218, + 51, 103, 44, 131, 131, 123, 31, 6, 158, 86, 40, 64, 135, 148, 224, 45, 183, 128, 22, 26, 17, 131, 240, 154, 14, 1, 209, + 83, 12, 13, 54, 192, 255, 68, 47, 28, 45, 16, 21, 91, 64, 222, 7, 1, 197, 56, 21, 39, 155, 60, 138, 23, 102, 213, + 85, 26, 85, 85, 128, 128, 32, 146, 171, 18, 11, 7, 63, 144, 171, 4, 4, 246, 35, 27, 10, 146, 174, 171, 12, 26, 128, + 190, 80, 35, 99, 180, 80, 126, 54, 45, 85, 126, 47, 87, 176, 51, 41, 20, 32, 101, 75, 128, 139, 118, 146, 116, 128, 85, + 56, 41, 15, 176, 236, 85, 37, 9, 62, 146, 36, 19, 30, 171, 255, 97, 27, 20, 71, 30, 17, 119, 118, 255, 17, 18, 138, + 101, 38, 60, 138, 55, 70, 43, 26, 142, 138, 45, 61, 62, 219, 1, 81, 188, 64, 32, 41, 20, 117, 151, 142, 20, 21, 163, + 112, 19, 12, 61, 195, 128, 48, 4, 24 +); + +// [block type 4][band 8][context 3][11] +export const COEFF_UPDATE_PROBS = Uint8Array.of( + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 176, 246, 255, 255, 255, 255, 255, 255, 255, 255, 255, 223, 241, 252, 255, 255, 255, 255, 255, 255, 255, 255, 249, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 244, 252, 255, 255, 255, 255, 255, 255, 255, 255, 234, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 246, 254, 255, 255, 255, 255, 255, 255, 255, 255, 239, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 248, 254, 255, 255, 255, 255, 255, 255, 255, 255, 251, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255, 251, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 254, 253, 255, 254, 255, 255, 255, 255, 255, 255, 250, 255, 254, 255, 254, 255, 255, 255, 255, 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 217, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 225, 252, 241, 253, 255, 255, 254, 255, 255, 255, 255, 234, 250, 241, 250, 253, 255, 253, 254, 255, 255, 255, + 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 223, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 238, 253, 254, 254, 255, 255, 255, 255, 255, 255, 255, + 255, 248, 254, 255, 255, 255, 255, 255, 255, 255, 255, 249, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 247, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255, 252, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 254, 253, 255, 255, 255, 255, 255, 255, 255, 255, 250, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 186, 251, 250, 255, 255, 255, 255, 255, 255, 255, 255, 234, 251, 244, 254, 255, 255, 255, 255, 255, 255, 255, 251, 251, 243, 253, 254, 255, 254, 255, 255, 255, 255, + 255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255, 236, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255, 251, 253, 253, 254, 254, 255, 255, 255, 255, 255, 255, + 255, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 254, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 248, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 250, 254, 252, 254, 255, 255, 255, 255, 255, 255, 255, 248, 254, 249, 253, 255, 255, 255, 255, 255, 255, 255, + 255, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255, 246, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255, 252, 254, 251, 254, 254, 255, 255, 255, 255, 255, 255, + 255, 254, 252, 255, 255, 255, 255, 255, 255, 255, 255, 248, 254, 253, 255, 255, 255, 255, 255, 255, 255, 255, 253, 255, 254, 254, 255, 255, 255, 255, 255, 255, 255, + 255, 251, 254, 255, 255, 255, 255, 255, 255, 255, 255, 245, 251, 254, 255, 255, 255, 255, 255, 255, 255, 255, 253, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 251, 253, 255, 255, 255, 255, 255, 255, 255, 255, 252, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 252, 255, 255, 255, 255, 255, 255, 255, 255, 255, 249, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 253, 255, 255, 255, 255, 255, 255, 255, 255, 250, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255 +); + +export const DEFAULT_COEFF_PROBS = Uint8Array.of( + 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, + 253, 136, 254, 255, 228, 219, 128, 128, 128, 128, 128, 189, 129, 242, 255, 227, 213, 255, 219, 128, 128, 128, 106, 126, 227, 252, 214, 209, 255, 255, 128, 128, 128, + 1, 98, 248, 255, 236, 226, 255, 255, 128, 128, 128, 181, 133, 238, 254, 221, 234, 255, 154, 128, 128, 128, 78, 134, 202, 247, 198, 180, 255, 219, 128, 128, 128, + 1, 185, 249, 255, 243, 255, 128, 128, 128, 128, 128, 184, 150, 247, 255, 236, 224, 128, 128, 128, 128, 128, 77, 110, 216, 255, 236, 230, 128, 128, 128, 128, 128, + 1, 101, 251, 255, 241, 255, 128, 128, 128, 128, 128, 170, 139, 241, 252, 236, 209, 255, 255, 128, 128, 128, 37, 116, 196, 243, 228, 255, 255, 255, 128, 128, 128, + 1, 204, 254, 255, 245, 255, 128, 128, 128, 128, 128, 207, 160, 250, 255, 238, 128, 128, 128, 128, 128, 128, 102, 103, 231, 255, 211, 171, 128, 128, 128, 128, 128, + 1, 152, 252, 255, 240, 255, 128, 128, 128, 128, 128, 177, 135, 243, 255, 234, 225, 128, 128, 128, 128, 128, 80, 129, 211, 255, 194, 224, 128, 128, 128, 128, 128, + 1, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128, 246, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128, 255, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, + 198, 35, 237, 223, 193, 187, 162, 160, 145, 155, 62, 131, 45, 198, 221, 172, 176, 220, 157, 252, 221, 1, 68, 47, 146, 208, 149, 167, 221, 162, 255, 223, 128, + 1, 149, 241, 255, 221, 224, 255, 255, 128, 128, 128, 184, 141, 234, 253, 222, 220, 255, 199, 128, 128, 128, 81, 99, 181, 242, 176, 190, 249, 202, 255, 255, 128, + 1, 129, 232, 253, 214, 197, 242, 196, 255, 255, 128, 99, 121, 210, 250, 201, 198, 255, 202, 128, 128, 128, 23, 91, 163, 242, 170, 187, 247, 210, 255, 255, 128, + 1, 200, 246, 255, 234, 255, 128, 128, 128, 128, 128, 109, 178, 241, 255, 231, 245, 255, 255, 128, 128, 128, 44, 130, 201, 253, 205, 192, 255, 255, 128, 128, 128, + 1, 132, 239, 251, 219, 209, 255, 165, 128, 128, 128, 94, 136, 225, 251, 218, 190, 255, 255, 128, 128, 128, 22, 100, 174, 245, 186, 161, 255, 199, 128, 128, 128, + 1, 182, 249, 255, 232, 235, 128, 128, 128, 128, 128, 124, 143, 241, 255, 227, 234, 128, 128, 128, 128, 128, 35, 77, 181, 251, 193, 211, 255, 205, 128, 128, 128, + 1, 157, 247, 255, 236, 231, 255, 255, 128, 128, 128, 121, 141, 235, 255, 225, 227, 255, 255, 128, 128, 128, 45, 99, 188, 251, 195, 217, 255, 224, 128, 128, 128, + 1, 1, 251, 255, 213, 255, 128, 128, 128, 128, 128, 203, 1, 248, 255, 255, 128, 128, 128, 128, 128, 128, 137, 1, 177, 255, 224, 255, 128, 128, 128, 128, 128, + 253, 9, 248, 251, 207, 208, 255, 192, 128, 128, 128, 175, 13, 224, 243, 193, 185, 249, 198, 255, 255, 128, 73, 17, 171, 221, 161, 179, 236, 167, 255, 234, 128, + 1, 95, 247, 253, 212, 183, 255, 255, 128, 128, 128, 239, 90, 244, 250, 211, 209, 255, 255, 128, 128, 128, 155, 77, 195, 248, 188, 195, 255, 255, 128, 128, 128, + 1, 24, 239, 251, 218, 219, 255, 205, 128, 128, 128, 201, 51, 219, 255, 196, 186, 128, 128, 128, 128, 128, 69, 46, 190, 239, 201, 218, 255, 228, 128, 128, 128, + 1, 191, 251, 255, 255, 128, 128, 128, 128, 128, 128, 223, 165, 249, 255, 213, 255, 128, 128, 128, 128, 128, 141, 124, 248, 255, 255, 128, 128, 128, 128, 128, 128, + 1, 16, 248, 255, 255, 128, 128, 128, 128, 128, 128, 190, 36, 230, 255, 236, 255, 128, 128, 128, 128, 128, 149, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128, + 1, 226, 255, 128, 128, 128, 128, 128, 128, 128, 128, 247, 192, 255, 128, 128, 128, 128, 128, 128, 128, 128, 240, 128, 255, 128, 128, 128, 128, 128, 128, 128, 128, + 1, 134, 252, 255, 255, 128, 128, 128, 128, 128, 128, 213, 62, 250, 255, 255, 128, 128, 128, 128, 128, 128, 55, 93, 255, 128, 128, 128, 128, 128, 128, 128, 128, + 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, + 202, 24, 213, 235, 186, 191, 220, 160, 240, 175, 255, 126, 38, 182, 232, 169, 184, 228, 174, 255, 187, 128, 61, 46, 138, 219, 151, 178, 240, 170, 255, 216, 128, + 1, 112, 230, 250, 199, 191, 247, 159, 255, 255, 128, 166, 109, 228, 252, 211, 215, 255, 174, 128, 128, 128, 39, 77, 162, 232, 172, 180, 245, 178, 255, 255, 128, + 1, 52, 220, 246, 198, 199, 249, 220, 255, 255, 128, 124, 74, 191, 243, 183, 193, 250, 221, 255, 255, 128, 24, 71, 130, 219, 154, 170, 243, 182, 255, 255, 128, + 1, 182, 225, 249, 219, 240, 255, 224, 128, 128, 128, 149, 150, 226, 252, 216, 205, 255, 171, 128, 128, 128, 28, 108, 170, 242, 183, 194, 254, 223, 255, 255, 128, + 1, 81, 230, 252, 204, 203, 255, 192, 128, 128, 128, 123, 102, 209, 247, 188, 196, 255, 233, 128, 128, 128, 20, 95, 153, 243, 164, 173, 255, 203, 128, 128, 128, + 1, 222, 248, 255, 216, 213, 128, 128, 128, 128, 128, 168, 175, 246, 252, 235, 205, 255, 255, 128, 128, 128, 47, 116, 215, 255, 211, 212, 255, 255, 128, 128, 128, + 1, 121, 236, 253, 212, 214, 255, 255, 128, 128, 128, 141, 84, 213, 252, 201, 202, 255, 219, 128, 128, 128, 42, 80, 160, 240, 162, 185, 255, 205, 128, 128, 128, + 1, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128, 244, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128, 238, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128 +); + +// quantizer step sizes by index +export const DC_QLOOKUP = Uint8Array.of( + 4, 5, 6, 7, 8, 9, 10, 10, 11, 12, 13, 14, 15, 16, 17, 17, + 18, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 25, 25, 26, 27, 28, + 29, 30, 31, 32, 33, 34, 35, 36, 37, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, + 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, + 75, 76, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, + 91, 93, 95, 96, 98, 100, 101, 102, 104, 106, 108, 110, 112, 114, 116, 118, + 122, 124, 126, 128, 130, 132, 134, 136, 138, 140, 143, 145, 148, 151, 154, 157 +); + +export const AC_QLOOKUP = Int16Array.of( + 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, + 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, + 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, + 52, 53, 54, 55, 56, 57, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, + 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100, 102, 104, 106, 108, + 110, 112, 114, 116, 119, 122, 125, 128, 131, 134, 137, 140, 143, 146, 149, 152, + 155, 158, 161, 164, 167, 170, 173, 177, 181, 185, 189, 193, 197, 201, 205, 209, + 213, 217, 221, 225, 229, 234, 239, 245, 249, 254, 259, 264, 269, 274, 279, 284 +); + +// extra bits probabilities of the dct_cat tokens +export const PCAT = [ + Uint8Array.of(159), + Uint8Array.of(165, 145), + Uint8Array.of(173, 148, 140), + Uint8Array.of(176, 155, 140, 135), + Uint8Array.of(180, 157, 141, 134, 130), + Uint8Array.of(254, 254, 243, 230, 196, 177, 153, 140, 133, 130, 129) +]; diff --git a/src/codecs/vp8.ts b/src/codecs/vp8.ts new file mode 100644 index 00000000..288cbfcc --- /dev/null +++ b/src/codecs/vp8.ts @@ -0,0 +1,787 @@ +/*! + * VP8 key frame decoder (WebP lossy), following RFC 6386 and its reference decoder (Copyright (c) 2010, 2011 Google + * Inc., BSD license). The YUV -> RGB conversion and the "fancy" chroma upsampling reproduce libwebp's (Copyright (c) + * 2010 Google Inc., BSD license), so the output matches libwebp exactly. See THIRD_PARTY_NOTICES.md + */ +import { KF_BMODE_PROBS, COEFF_UPDATE_PROBS, DEFAULT_COEFF_PROBS, DC_QLOOKUP, AC_QLOOKUP, PCAT } from './vp8-tables.ts'; + +/** + * Boolean entropy decoder, RFC 6386 section 7 + */ +class BoolDecoder { + private data: Uint8Array; + private pos: number; + private end: number; + private value: number; + private range = 255; + private bitCount = 0; + + constructor(data: Uint8Array, start: number, end: number) { + this.data = data; + this.end = Math.min(end, data.length); + this.pos = start; + this.value = this.next() << 8 | this.next(); + } + + private next(): number { + // past the end, the stream continues with zeros (like libvpx) + return this.pos < this.end ? this.data[this.pos++] : (this.pos++, 0); + } + + read(probability: number): number { + const split = 1 + ((this.range - 1) * probability >> 8); + const bigSplit = split << 8; + let bit: number; + if (this.value >= bigSplit) { + bit = 1; + this.range -= split; + this.value -= bigSplit; + } else { + bit = 0; + this.range = split; + } + + if (this.range < 128) { + const shift = Math.clz32(this.range) - 24; + this.range <<= shift; + this.value <<= shift; + this.bitCount += shift; + if (this.bitCount >= 8) { + this.bitCount -= 8; + this.value |= this.next() << this.bitCount; + } + } + + return bit; + } + + /** unsigned n-bit literal, most significant bit first */ + literal(bits: number): number { + let v = 0; + while (bits-- > 0) v = v << 1 | this.read(128); + return v; + } + + /** magnitude followed by a sign bit, if the flag is set */ + optionalSigned(bits: number): number { + if (!this.read(128)) return 0; + const v = this.literal(bits); + return this.read(128) ? -v : v; + } + + /** reads a value through a tree (RFC 6386 section 8.1) */ + tree(tree: Int8Array, probs: ArrayLike, offset = 0): number { + let i = 0; + while ((i = tree[i + this.read(probs[offset + (i >> 1)])]) > 0); + return -i; + } + + /** whether the decoder read beyond its data */ + get exhausted(): boolean { + return this.pos > this.end + 2; + } +} + +// luma (16x16) modes and subblock modes +const DC_PRED = 0, V_PRED = 1, H_PRED = 2, TM_PRED = 3, B_PRED = 4; +const B_DC_PRED = 0, B_TM_PRED = 1, B_VE_PRED = 2, B_HE_PRED = 3, B_LD_PRED = 4, B_RD_PRED = 5, B_VR_PRED = 6, B_VL_PRED = 7, B_HD_PRED = 8, B_HU_PRED = 9; + +// trees, RFC 6386 sections 11.2, 11.4, 9.3 (leaves are stored negated, B_DC_PRED/DC_PRED/segment 0 as -0) +const KF_YMODE_TREE = Int8Array.of(-B_PRED, 2, 4, 6, -DC_PRED, -V_PRED, -H_PRED, -TM_PRED); +const KF_YMODE_PROBS = Uint8Array.of(145, 156, 163, 128); +const UV_MODE_TREE = Int8Array.of(-DC_PRED, 2, -V_PRED, 4, -H_PRED, -TM_PRED); +const KF_UV_MODE_PROBS = Uint8Array.of(142, 114, 183); +const BMODE_TREE = Int8Array.of( + -B_DC_PRED, 2, -B_TM_PRED, 4, -B_VE_PRED, 6, 8, 12, -B_HE_PRED, 10, -B_RD_PRED, -B_VR_PRED, -B_LD_PRED, 14, + -B_VL_PRED, 16, -B_HD_PRED, -B_HU_PRED +); +const SEGMENT_TREE = Int8Array.of(2, 4, -0, -1, -2, -3); + +// subblock mode implied by a 16x16 mode, for the subblock mode contexts +const IMPLIED_BMODE = [B_DC_PRED, B_VE_PRED, B_HE_PRED, B_TM_PRED]; + +const ZIGZAG = Uint8Array.of(0, 1, 4, 8, 5, 2, 3, 6, 9, 12, 13, 10, 7, 11, 14, 15); +const BANDS = Uint8Array.of(0, 1, 2, 3, 6, 4, 5, 6, 6, 6, 6, 6, 6, 6, 6, 7, 0); +const CATEGORY_BASE = [5, 7, 11, 19, 35, 67]; + +// coefficient block types: Y after Y2, Y2, chroma, Y with DC +const TYPE_Y_AC = 0, TYPE_Y2 = 1, TYPE_UV = 2, TYPE_Y = 3; + +interface Header { + width: number; + height: number; + filterType: number; + filterLevel: number; + sharpness: number; + lfDeltas: { enabled: boolean, ref: Int32Array, mode: Int32Array }; + segmentation: { enabled: boolean, updateMap: boolean, absolute: boolean, quant: Int32Array, filter: Int32Array, probs: Uint8Array }; + partitions: BoolDecoder[]; + coeffProbs: Uint8Array; + skipProb: number; + /** [segment][y1 dc, y1 ac, y2 dc, y2 ac, uv dc, uv ac] */ + quant: Int32Array[]; +} + +const clampQ = (q: number): number => q < 0 ? 0 : q > 127 ? 127 : q; + +function readHeader(data: Uint8Array): { header: Header, first: BoolDecoder } { + if (data.length < 10) throw new Error('VP8: truncated header'); + const tag = data[0] | data[1] << 8 | data[2] << 16; + if (tag & 1) throw new Error('VP8: not a key frame'); + if ((tag >> 1 & 7) > 3) throw new Error('VP8: unknown version'); + const firstSize = tag >>> 5; + if (data[3] !== 0x9d || data[4] !== 0x01 || data[5] !== 0x2a) throw new Error('VP8: invalid start code'); + const width = (data[6] | data[7] << 8) & 0x3fff; + const height = (data[8] | data[9] << 8) & 0x3fff; + if (!width || !height) throw new Error('VP8: invalid dimensions'); + if (10 + firstSize > data.length) throw new Error('VP8: truncated first partition'); + + const d = new BoolDecoder(data, 10, 10 + firstSize); + d.literal(1); // color space + d.literal(1); // clamping type, clamping is always done + + const segmentation = { enabled: false, updateMap: false, absolute: false, quant: new Int32Array(4), filter: new Int32Array(4), probs: Uint8Array.of(255, 255, 255) }; + if (d.literal(1)) { + segmentation.enabled = true; + segmentation.updateMap = d.literal(1) === 1; + if (d.literal(1)) { + segmentation.absolute = d.literal(1) === 1; + for (let i = 0; i < 4; i++) segmentation.quant[i] = d.optionalSigned(7); + for (let i = 0; i < 4; i++) segmentation.filter[i] = d.optionalSigned(6); + } + if (segmentation.updateMap) + for (let i = 0; i < 3; i++) segmentation.probs[i] = d.literal(1) ? d.literal(8) : 255; + } + + const filterType = d.literal(1); + const filterLevel = d.literal(6); + const sharpness = d.literal(3); + const lfDeltas = { enabled: false, ref: new Int32Array(4), mode: new Int32Array(4) }; + if (d.literal(1)) { + lfDeltas.enabled = true; + if (d.literal(1)) { + for (let i = 0; i < 4; i++) if (d.literal(1)) lfDeltas.ref[i] = d.literal(6) * (d.literal(1) ? -1 : 1); + for (let i = 0; i < 4; i++) if (d.literal(1)) lfDeltas.mode[i] = d.literal(6) * (d.literal(1) ? -1 : 1); + } + } + + // DCT token partitions: their sizes (3 bytes each, all but the last) follow the first partition + const count = 1 << d.literal(2); + let offset = 10 + firstSize; + const sizesStart = offset; + offset += 3 * (count - 1); + if (offset > data.length) throw new Error('VP8: truncated partitions'); + const partitions: BoolDecoder[] = []; + for (let i = 0; i < count; i++) { + const s = sizesStart + 3 * i; + const size = i < count - 1 ? data[s] | data[s + 1] << 8 | data[s + 2] << 16 : data.length - offset; + partitions.push(new BoolDecoder(data, offset, offset + size)); + offset += size; + } + + const base = d.literal(7); + const y1dc = d.optionalSigned(4), y2dc = d.optionalSigned(4), y2ac = d.optionalSigned(4), uvdc = d.optionalSigned(4), uvac = d.optionalSigned(4); + const quant: Int32Array[] = []; + for (let s = 0; s < 4; s++) { + let q = base; + if (segmentation.enabled) q = segmentation.absolute ? segmentation.quant[s] : q + segmentation.quant[s]; + const y2acFactor = AC_QLOOKUP[clampQ(q + y2ac)] * 155 / 100 | 0; + quant.push(Int32Array.of( + DC_QLOOKUP[clampQ(q + y1dc)], + AC_QLOOKUP[clampQ(q)], + DC_QLOOKUP[clampQ(q + y2dc)] * 2, + y2acFactor < 8 ? 8 : y2acFactor, + Math.min(132, DC_QLOOKUP[clampQ(q + uvdc)]), + AC_QLOOKUP[clampQ(q + uvac)] + )); + } + + d.literal(1); // refresh entropy probs, irrelevant for a single frame + + const coeffProbs = DEFAULT_COEFF_PROBS.slice(); + for (let i = 0; i < coeffProbs.length; i++) + if (d.read(COEFF_UPDATE_PROBS[i])) coeffProbs[i] = d.literal(8); + + const skipProb = d.literal(1) ? d.literal(8) : -1; + + return { + header: { width, height, filterType, filterLevel, sharpness, lfDeltas, segmentation, partitions, coeffProbs, skipProb, quant }, + first: d + }; +} + +const clamp255 = (v: number): number => v < 0 ? 0 : v > 255 ? 255 : v; + +/** + * Decodes a VP8 key frame (the payload of a VP8 chunk) into RGBA + */ +export function decode(data: Uint8Array): { width: number, height: number, pixels: Uint8Array } { + const { header: h, first: d } = readHeader(data); + const { width, height } = h; + const mbw = (width + 15) >> 4, mbh = (height + 15) >> 4; + + // planes with a one pixel border on the top and left (127 above, 129 to the left, as the spec defines) + const ys = mbw * 16 + 1, cs = mbw * 8 + 1; + const Y = new Uint8Array(ys * (mbh * 16 + 1)); + const U = new Uint8Array(cs * (mbh * 8 + 1)); + const V = new Uint8Array(cs * (mbh * 8 + 1)); + for (const [plane, stride] of [[Y, ys], [U, cs], [V, cs]] as const) { + plane.fill(127, 0, stride); + for (let i = stride; i < plane.length; i += stride) plane[i] = 129; + } + + // per macroblock info needed by the loop filter + const mbCount = mbw * mbh; + const mbSegment = new Uint8Array(mbCount); + const mbInner = new Uint8Array(mbCount); // filter the inner edges (B_PRED or coefficients present) + const mbBpred = new Uint8Array(mbCount); + + // contexts: subblock modes above (4 per mb column) and left; non-zero flags above (9 per column) and left + const aboveModes = new Uint8Array(mbw * 4); + const leftModes = new Uint8Array(4); + const aboveNz = new Uint8Array(mbw * 9); + const leftNz = new Uint8Array(9); + + const coeffs = new Int16Array(25 * 16); + const bmodes = new Uint8Array(16); + const edge = new Uint8Array(21); // P, 16 pixels above, 4 above right (luma) / P, 8 above (chroma) + const left = new Uint8Array(16); + + for (let mby = 0; mby < mbh; mby++) { + const tokens = h.partitions[mby % h.partitions.length]; + leftModes.fill(B_DC_PRED); + leftNz.fill(0); + + for (let mbx = 0; mbx < mbw; mbx++) { + const mb = mby * mbw + mbx; + + // macroblock header (first partition) + const segment = h.segmentation.updateMap ? d.tree(SEGMENT_TREE, h.segmentation.probs) : 0; + mbSegment[mb] = segment; + const skip = h.skipProb >= 0 ? d.read(h.skipProb) : 0; + const ymode = d.tree(KF_YMODE_TREE, KF_YMODE_PROBS); + + if (ymode === B_PRED) { + for (let b = 0; b < 16; b++) { + const above = b < 4 ? aboveModes[mbx * 4 + b] : bmodes[b - 4]; + const l = (b & 3) === 0 ? leftModes[b >> 2] : bmodes[b - 1]; + bmodes[b] = d.tree(BMODE_TREE, KF_BMODE_PROBS, (above * 10 + l) * 9); + } + } else bmodes.fill(IMPLIED_BMODE[ymode]); + + for (let i = 0; i < 4; i++) { + aboveModes[mbx * 4 + i] = bmodes[12 + i]; + leftModes[i] = bmodes[i * 4 + 3]; + } + const uvmode = d.tree(UV_MODE_TREE, KF_UV_MODE_PROBS); + + // residual data (token partition) + const hasY2 = ymode !== B_PRED; + const an = mbx * 9; // above non-zero flags: 0-3 Y, 4-5 U, 6-7 V, 8 Y2 + coeffs.fill(0); + let nonZero = 0; // bit per block (0-15 Y, 16-23 U/V) with a non-zero coefficient, 24: Y2 AC + + if (!skip) { + const q = h.quant[segment]; + if (hasY2) { + const ctx = aboveNz[an + 8] + leftNz[8]; + const n = readCoefficients(tokens, h.coeffProbs, TYPE_Y2, ctx, 0, q[2], q[3], coeffs, 384); + aboveNz[an + 8] = leftNz[8] = n > 0 ? 1 : 0; + inverseWHT(coeffs, 384, n); + } + + const type = hasY2 ? TYPE_Y_AC : TYPE_Y, first = hasY2 ? 1 : 0; + for (let b = 0; b < 16; b++) { + const x = b & 3, y = b >> 2; + const ctx = aboveNz[an + x] + leftNz[y]; + const n = readCoefficients(tokens, h.coeffProbs, type, ctx, first, q[0], q[1], coeffs, b * 16); + aboveNz[an + x] = leftNz[y] = n > first ? 1 : 0; + if (n > first || coeffs[b * 16] !== 0) nonZero |= 1 << b; + } + + for (let b = 16; b < 24; b++) { + const plane = b < 20 ? 4 : 6, x = b & 1, y = (b >> 1) & 1; + const ctx = aboveNz[an + plane + x] + leftNz[plane + y]; + const n = readCoefficients(tokens, h.coeffProbs, TYPE_UV, ctx, 0, q[4], q[5], coeffs, b * 16); + aboveNz[an + plane + x] = leftNz[plane + y] = n > 0 ? 1 : 0; + if (n > 0) nonZero |= 1 << b; + } + } else { + // a skipped macroblock has no coefficients, its contexts are cleared (Y2's only if it has one) + aboveNz.fill(0, an, an + 8); + leftNz.fill(0, 0, 8); + if (hasY2) aboveNz[an + 8] = leftNz[8] = 0; + } + + mbBpred[mb] = ymode === B_PRED ? 1 : 0; + mbInner[mb] = ymode === B_PRED || nonZero !== 0 ? 1 : 0; + + // prediction and reconstruction + const yBase = (mby * 16 + 1) * ys + mbx * 16 + 1; + if (ymode === B_PRED) { + for (let b = 0; b < 16; b++) { + const bx = b & 3, by = b >> 2; + const base = yBase + by * 4 * ys + bx * 4; + // above row with above-right pixels: blocks on the right edge use the ones of block 3 + for (let i = 0; i < 5; i++) edge[i] = Y[base - ys - 1 + i]; + if (bx < 3) for (let i = 5; i < 9; i++) edge[i] = Y[base - ys - 1 + i]; + else { + const top = yBase - ys + 16; + for (let i = 5; i < 9; i++) + edge[i] = mby === 0 ? 127 : mbx === mbw - 1 ? Y[yBase - ys + 15] : Y[top + i - 5]; + } + for (let i = 0; i < 4; i++) left[i] = Y[base + i * ys - 1]; + predictSubblock(bmodes[b], edge, left, Y, base, ys); + if (nonZero >> b & 1) inverseDCT(coeffs, b * 16, Y, base, ys); + } + } else { + predictBlock(ymode, Y, yBase, ys, 16, mbx, mby); + for (let b = 0; b < 16; b++) + if (nonZero >> b & 1) inverseDCT(coeffs, b * 16, Y, yBase + (b >> 2) * 4 * ys + (b & 3) * 4, ys); + } + + const cBase = (mby * 8 + 1) * cs + mbx * 8 + 1; + predictBlock(uvmode, U, cBase, cs, 8, mbx, mby); + predictBlock(uvmode, V, cBase, cs, 8, mbx, mby); + for (let b = 16; b < 24; b++) { + if (!(nonZero >> b & 1)) continue; + const plane = b < 20 ? U : V, i = b & 3; + inverseDCT(coeffs, b * 16, plane, cBase + (i >> 1) * 4 * cs + (i & 1) * 4, cs); + } + } + + if (tokens.exhausted && d.exhausted) throw new Error('VP8: unexpected end of data'); + } + + // a frame level of 0 disables the filter, whatever the segments say (RFC 6386 section 15) + if (h.filterLevel) loopFilter(h, Y, U, V, ys, cs, mbw, mbh, mbSegment, mbInner, mbBpred); + + return { width, height, pixels: toRGBA(Y, U, V, ys, cs, width, height) }; +} + +/** + * Reads one block's tokens (RFC 6386 section 13), dequantizing into out[offset + zigzag position] + * @return the position after the last decoded token (index of the end of block) + */ +function readCoefficients(d: BoolDecoder, probs: Uint8Array, type: number, ctx: number, first: number, dcFactor: number, acFactor: number, out: Int16Array, offset: number): number { + let i = first; + let p = ((type * 8 + BANDS[i]) * 3 + ctx) * 11; + let previousZero = false; + + for (; i < 16; i++) { + if (!previousZero && !d.read(probs[p])) return i; // end of block + + if (!d.read(probs[p + 1])) { + // zero, the next token can't be the end of block + previousZero = true; + p = ((type * 8 + BANDS[i + 1]) * 3) * 11; + continue; + } + + let v: number; + if (!d.read(probs[p + 2])) v = 1; + else if (!d.read(probs[p + 3])) v = d.read(probs[p + 4]) ? 3 + d.read(probs[p + 5]) : 2; + else { + let category: number; + if (!d.read(probs[p + 6])) category = d.read(probs[p + 7]); + else if (!d.read(probs[p + 8])) category = 2 + d.read(probs[p + 9]); + else category = 4 + d.read(probs[p + 10]); + + const extra = PCAT[category]; + let e = 0; + for (let k = 0; k < extra.length; k++) e = e << 1 | d.read(extra[k]); + v = CATEGORY_BASE[category] + e; + } + + if (d.read(128)) v = -v; + out[offset + ZIGZAG[i]] = v * (i > 0 ? acFactor : dcFactor); + previousZero = false; + p = ((type * 8 + BANDS[i + 1]) * 3 + (v === 1 || v === -1 ? 1 : 2)) * 11; + } + + return 16; +} + +// inverse Walsh-Hadamard transform of the Y2 block (RFC 6386 section 14.3), the results become the Y blocks' DCs +function inverseWHT(c: Int16Array, offset: number, count: number): void { + const out = new Int32Array(16); + if (count > 1) { + for (let i = 0; i < 4; i++) { + const a1 = c[offset + i] + c[offset + 12 + i]; + const b1 = c[offset + 4 + i] + c[offset + 8 + i]; + const c1 = c[offset + 4 + i] - c[offset + 8 + i]; + const d1 = c[offset + i] - c[offset + 12 + i]; + out[i] = a1 + b1; + out[4 + i] = c1 + d1; + out[8 + i] = a1 - b1; + out[12 + i] = d1 - c1; + } + for (let i = 0; i < 4; i++) { + const r = i * 4; + const a1 = out[r] + out[r + 3]; + const b1 = out[r + 1] + out[r + 2]; + const c1 = out[r + 1] - out[r + 2]; + const d1 = out[r] - out[r + 3]; + out[r] = a1 + b1 + 3 >> 3; + out[r + 1] = c1 + d1 + 3 >> 3; + out[r + 2] = a1 - b1 + 3 >> 3; + out[r + 3] = d1 - c1 + 3 >> 3; + } + } else out.fill(c[offset] + 3 >> 3); + + for (let b = 0; b < 16; b++) c[b * 16] = out[b]; +} + +// inverse DCT (RFC 6386 section 14.4), added to the prediction in place +const tmp = new Int32Array(16); + +function inverseDCT(c: Int16Array, offset: number, dst: Uint8Array, base: number, stride: number): void { + for (let i = 0; i < 4; i++) { + const i0 = c[offset + i], i4 = c[offset + 4 + i], i8 = c[offset + 8 + i], i12 = c[offset + 12 + i]; + const a1 = i0 + i8, b1 = i0 - i8; + let t1 = i4 * 35468 >> 16, t2 = i12 + (i12 * 20091 >> 16); + const c1 = t1 - t2; + t1 = i4 + (i4 * 20091 >> 16); + t2 = i12 * 35468 >> 16; + const d1 = t1 + t2; + tmp[i] = a1 + d1; + tmp[12 + i] = a1 - d1; + tmp[4 + i] = b1 + c1; + tmp[8 + i] = b1 - c1; + } + for (let i = 0; i < 4; i++) { + const r = i * 4; + const i0 = tmp[r], i1 = tmp[r + 1], i2 = tmp[r + 2], i3 = tmp[r + 3]; + const a1 = i0 + i2, b1 = i0 - i2; + let t1 = i1 * 35468 >> 16, t2 = i3 + (i3 * 20091 >> 16); + const c1 = t1 - t2; + t1 = i1 + (i1 * 20091 >> 16); + t2 = i3 * 35468 >> 16; + const d1 = t1 + t2; + const o = base + i * stride; + dst[o] = clamp255(dst[o] + (a1 + d1 + 4 >> 3)); + dst[o + 3] = clamp255(dst[o + 3] + (a1 - d1 + 4 >> 3)); + dst[o + 1] = clamp255(dst[o + 1] + (b1 + c1 + 4 >> 3)); + dst[o + 2] = clamp255(dst[o + 2] + (b1 - c1 + 4 >> 3)); + } +} + +// 16x16 luma or 8x8 chroma prediction (RFC 6386 sections 12.2, 12.3) +function predictBlock(mode: number, p: Uint8Array, base: number, stride: number, size: number, mbx: number, mby: number): void { + const above = base - stride; + if (mode === DC_PRED) { + const shift = size === 16 ? 3 : 2; + let sum = 0, count = 0; + if (mby > 0) { for (let i = 0; i < size; i++) sum += p[above + i]; count++; } + if (mbx > 0) { for (let i = 0; i < size; i++) sum += p[base + i * stride - 1]; count++; } + const v = count === 0 ? 128 : sum + (1 << shift + count - 1) >> shift + count; + for (let y = 0; y < size; y++) p.fill(v, base + y * stride, base + y * stride + size); + } else if (mode === V_PRED) { + for (let y = 0; y < size; y++) p.copyWithin(base + y * stride, above, above + size); + } else if (mode === H_PRED) { + for (let y = 0; y < size; y++) { + const row = base + y * stride; + p.fill(p[row - 1], row, row + size); + } + } else { + const corner = p[above - 1]; + for (let y = 0; y < size; y++) { + const row = base + y * stride, l = p[row - 1] - corner; + for (let x = 0; x < size; x++) p[row + x] = clamp255(l + p[above + x]); + } + } +} + +const avg3 = (x: number, y: number, z: number): number => x + 2 * y + z + 2 >> 2; +const avg2 = (x: number, y: number): number => x + y + 1 >> 1; + +// 4x4 subblock prediction (RFC 6386 section 12.3); edge = P, A[0..7], left = L[0..3]; writes into the plane +const B = new Uint8Array(16); + +function predictSubblock(mode: number, edge: Uint8Array, L: Uint8Array, p: Uint8Array, base: number, stride: number): void { + const P = edge[0]; + const A0 = edge[1], A1 = edge[2], A2 = edge[3], A3 = edge[4], A4 = edge[5], A5 = edge[6], A6 = edge[7], A7 = edge[8]; + const L0 = L[0], L1 = L[1], L2 = L[2], L3 = L[3]; + // E = L3, L2, L1, L0, P, A0..A3 (the edge from bottom left to top right) + + switch (mode) { + case B_DC_PRED: + B.fill(A0 + A1 + A2 + A3 + L0 + L1 + L2 + L3 + 4 >> 3); + break; + case B_TM_PRED: + for (let r = 0; r < 4; r++) { + const l = L[r] - P; + B[r * 4] = clamp255(l + A0); + B[r * 4 + 1] = clamp255(l + A1); + B[r * 4 + 2] = clamp255(l + A2); + B[r * 4 + 3] = clamp255(l + A3); + } + break; + case B_VE_PRED: { + const v0 = avg3(P, A0, A1), v1 = avg3(A0, A1, A2), v2 = avg3(A1, A2, A3), v3 = avg3(A2, A3, A4); + for (let r = 0; r < 16; r += 4) { B[r] = v0; B[r + 1] = v1; B[r + 2] = v2; B[r + 3] = v3; } + break; + } + case B_HE_PRED: + B.fill(avg3(P, L0, L1), 0, 4); + B.fill(avg3(L0, L1, L2), 4, 8); + B.fill(avg3(L1, L2, L3), 8, 12); + B.fill(avg3(L2, L3, L3), 12, 16); + break; + case B_LD_PRED: + B[0] = avg3(A0, A1, A2); + B[1] = B[4] = avg3(A1, A2, A3); + B[2] = B[5] = B[8] = avg3(A2, A3, A4); + B[3] = B[6] = B[9] = B[12] = avg3(A3, A4, A5); + B[7] = B[10] = B[13] = avg3(A4, A5, A6); + B[11] = B[14] = avg3(A5, A6, A7); + B[15] = avg3(A6, A7, A7); + break; + case B_RD_PRED: + B[12] = avg3(L3, L2, L1); + B[13] = B[8] = avg3(L2, L1, L0); + B[14] = B[9] = B[4] = avg3(L1, L0, P); + B[15] = B[10] = B[5] = B[0] = avg3(L0, P, A0); + B[11] = B[6] = B[1] = avg3(P, A0, A1); + B[7] = B[2] = avg3(A0, A1, A2); + B[3] = avg3(A1, A2, A3); + break; + case B_VR_PRED: + B[12] = avg3(L2, L1, L0); + B[8] = avg3(L1, L0, P); + B[13] = B[4] = avg3(L0, P, A0); + B[9] = B[0] = avg2(P, A0); + B[14] = B[5] = avg3(P, A0, A1); + B[10] = B[1] = avg2(A0, A1); + B[15] = B[6] = avg3(A0, A1, A2); + B[11] = B[2] = avg2(A1, A2); + B[7] = avg3(A1, A2, A3); + B[3] = avg2(A2, A3); + break; + case B_VL_PRED: + B[0] = avg2(A0, A1); + B[4] = avg3(A0, A1, A2); + B[8] = B[1] = avg2(A1, A2); + B[5] = B[12] = avg3(A1, A2, A3); + B[9] = B[2] = avg2(A2, A3); + B[13] = B[6] = avg3(A2, A3, A4); + B[10] = B[3] = avg2(A3, A4); + B[14] = B[7] = avg3(A3, A4, A5); + B[11] = avg3(A4, A5, A6); + B[15] = avg3(A5, A6, A7); + break; + case B_HD_PRED: + B[12] = avg2(L3, L2); + B[13] = avg3(L3, L2, L1); + B[8] = B[14] = avg2(L2, L1); + B[9] = B[15] = avg3(L2, L1, L0); + B[10] = B[4] = avg2(L1, L0); + B[11] = B[5] = avg3(L1, L0, P); + B[6] = B[0] = avg2(L0, P); + B[7] = B[1] = avg3(L0, P, A0); + B[2] = avg3(P, A0, A1); + B[3] = avg3(A0, A1, A2); + break; + case B_HU_PRED: + B[0] = avg2(L0, L1); + B[1] = avg3(L0, L1, L2); + B[2] = B[4] = avg2(L1, L2); + B[3] = B[5] = avg3(L1, L2, L3); + B[6] = B[8] = avg2(L2, L3); + B[7] = B[9] = avg3(L2, L3, L3); + B[10] = B[11] = B[12] = B[13] = B[14] = B[15] = L3; + break; + } + + for (let r = 0; r < 4; r++) { + const o = base + r * stride, b = r * 4; + p[o] = B[b]; p[o + 1] = B[b + 1]; p[o + 2] = B[b + 2]; p[o + 3] = B[b + 3]; + } +} + +// loop filter (RFC 6386 section 15), applied to the whole frame after reconstruction + +const s8 = (v: number): number => v < -128 ? -128 : v > 127 ? 127 : v; + +// common_adjust: moves p0 and q0 closer; returns the filter value a +function commonAdjust(p: Uint8Array, i: number, step: number, outerTaps: boolean): number { + const p1 = p[i - 2 * step] - 128, p0 = p[i - step] - 128, q0 = p[i] - 128, q1 = p[i + step] - 128; + let a = s8((outerTaps ? s8(p1 - q1) : 0) + 3 * (q0 - p0)); + const b = s8(a + 3) >> 3; + a = s8(a + 4) >> 3; + p[i] = s8(q0 - a) + 128; + p[i - step] = s8(p0 + b) + 128; + return a; +} + +const edgeOk = (p: Uint8Array, i: number, step: number, limit: number): boolean => + Math.abs(p[i - step] - p[i]) * 2 + (Math.abs(p[i - 2 * step] - p[i + step]) >> 1) <= limit; + +/** + * Filters the segments across an edge: `count` positions starting at i, each `along` apart, pixels `step` apart + */ +function filterEdge(p: Uint8Array, i: number, along: number, step: number, count: number, macroblock: boolean, edgeLimit: number, interior: number, hevThreshold: number): void { + for (let k = 0; k < count; k++, i += along) { + const p3 = p[i - 4 * step], p2 = p[i - 3 * step], p1 = p[i - 2 * step], p0 = p[i - step]; + const q0 = p[i], q1 = p[i + step], q2 = p[i + 2 * step], q3 = p[i + 3 * step]; + + // filter_yes: edge difference and interior differences within their limits + if (Math.abs(p0 - q0) * 2 + (Math.abs(p1 - q1) >> 1) > edgeLimit + || Math.abs(p3 - p2) > interior || Math.abs(p2 - p1) > interior || Math.abs(p1 - p0) > interior + || Math.abs(q3 - q2) > interior || Math.abs(q2 - q1) > interior || Math.abs(q1 - q0) > interior) continue; + + const hev = Math.abs(p1 - p0) > hevThreshold || Math.abs(q1 - q0) > hevThreshold; + if (macroblock) { + if (hev) { + commonAdjust(p, i, step, true); + continue; + } + const w = s8(s8(p1 - q1) + 3 * (q0 - p0)); + let a = s8(27 * w + 63 >> 7); + p[i] = s8(q0 - 128 - a) + 128; + p[i - step] = s8(p0 - 128 + a) + 128; + a = s8(18 * w + 63 >> 7); + p[i + step] = s8(q1 - 128 - a) + 128; + p[i - 2 * step] = s8(p1 - 128 + a) + 128; + a = s8(9 * w + 63 >> 7); + p[i + 2 * step] = s8(q2 - 128 - a) + 128; + p[i - 3 * step] = s8(p2 - 128 + a) + 128; + } else { + const a = commonAdjust(p, i, step, hev) + 1 >> 1; + if (!hev) { + p[i + step] = s8(q1 - 128 - a) + 128; + p[i - 2 * step] = s8(p1 - 128 + a) + 128; + } + } + } +} + +function filterEdgeSimple(p: Uint8Array, i: number, along: number, step: number, count: number, limit: number): void { + for (let k = 0; k < count; k++, i += along) + if (edgeOk(p, i, step, limit)) commonAdjust(p, i, step, true); +} + +function loopFilter(h: Header, Y: Uint8Array, U: Uint8Array, V: Uint8Array, ys: number, cs: number, mbw: number, mbh: number, segments: Uint8Array, inner: Uint8Array, bpred: Uint8Array): void { + for (let mby = 0; mby < mbh; mby++) + for (let mbx = 0; mbx < mbw; mbx++) { + const mb = mby * mbw + mbx; + + let level = h.filterLevel; + if (h.segmentation.enabled) { + level = h.segmentation.filter[segments[mb]]; + if (!h.segmentation.absolute) level += h.filterLevel; + } + if (h.lfDeltas.enabled) { + level += h.lfDeltas.ref[0]; // intra frame + if (bpred[mb]) level += h.lfDeltas.mode[0]; + } + level = level < 0 ? 0 : level > 63 ? 63 : level; + if (level === 0) continue; + + let interior = level; + if (h.sharpness) { + interior >>= h.sharpness > 4 ? 2 : 1; + if (interior > 9 - h.sharpness) interior = 9 - h.sharpness; + } + if (!interior) interior = 1; + const hevThreshold = level >= 40 ? 2 : level >= 15 ? 1 : 0; + const mbLimit = (level + 2) * 2 + interior, subLimit = level * 2 + interior; + const filterInner = inner[mb] === 1; + + const y0 = (mby * 16 + 1) * ys + mbx * 16 + 1; + if (h.filterType === 1) { + // simple filter: luma only + if (mbx > 0) filterEdgeSimple(Y, y0, ys, 1, 16, mbLimit); + if (filterInner) for (let x = 4; x < 16; x += 4) filterEdgeSimple(Y, y0 + x, ys, 1, 16, subLimit); + if (mby > 0) filterEdgeSimple(Y, y0, 1, ys, 16, mbLimit); + if (filterInner) for (let y = 4; y < 16; y += 4) filterEdgeSimple(Y, y0 + y * ys, 1, ys, 16, subLimit); + continue; + } + + const c0 = (mby * 8 + 1) * cs + mbx * 8 + 1; + if (mbx > 0) { + filterEdge(Y, y0, ys, 1, 16, true, mbLimit, interior, hevThreshold); + filterEdge(U, c0, cs, 1, 8, true, mbLimit, interior, hevThreshold); + filterEdge(V, c0, cs, 1, 8, true, mbLimit, interior, hevThreshold); + } + if (filterInner) { + for (let x = 4; x < 16; x += 4) filterEdge(Y, y0 + x, ys, 1, 16, false, subLimit, interior, hevThreshold); + filterEdge(U, c0 + 4, cs, 1, 8, false, subLimit, interior, hevThreshold); + filterEdge(V, c0 + 4, cs, 1, 8, false, subLimit, interior, hevThreshold); + } + if (mby > 0) { + filterEdge(Y, y0, 1, ys, 16, true, mbLimit, interior, hevThreshold); + filterEdge(U, c0, 1, cs, 8, true, mbLimit, interior, hevThreshold); + filterEdge(V, c0, 1, cs, 8, true, mbLimit, interior, hevThreshold); + } + if (filterInner) { + for (let y = 4; y < 16; y += 4) filterEdge(Y, y0 + y * ys, 1, ys, 16, false, subLimit, interior, hevThreshold); + filterEdge(U, c0 + 4 * cs, 1, cs, 8, false, subLimit, interior, hevThreshold); + filterEdge(V, c0 + 4 * cs, 1, cs, 8, false, subLimit, interior, hevThreshold); + } + } +} + +// libwebp's YUV -> RGB (BT.601, 14 bit fixed point) +const multHi = (v: number, coeff: number): number => v * coeff >> 8; +const clip8 = (v: number): number => (v & ~16383) === 0 ? v >> 6 : v < 0 ? 0 : 255; + +function yuvToRGBA(y: number, u: number, v: number, out: Uint8Array, o: number): void { + const yy = multHi(y, 19077); + out[o] = clip8(yy + multHi(v, 26149) - 14234); + out[o + 1] = clip8(yy - multHi(u, 6419) - multHi(v, 13320) + 8708); + out[o + 2] = clip8(yy + multHi(u, 33050) - 17685); + out[o + 3] = 255; +} + +/** + * Converts to RGBA with libwebp's "fancy" upsampling: chroma is interpolated from the four nearest samples + * (9-3-3-1 weights), working on pairs of output rows that lie between two chroma rows + */ +function toRGBA(Y: Uint8Array, U: Uint8Array, V: Uint8Array, ys: number, cs: number, width: number, height: number): Uint8Array { + const out = new Uint8Array(width * height * 4); + const yRow = (y: number): number => (y + 1) * ys + 1; + const cRow = (y: number): number => (y + 1) * cs + 1; + const lastChroma = (height + 1 >> 1) - 1; + + // first row: mirrors the chroma row at the boundary + upsamplePair(Y, U, V, yRow(0), -1, cRow(0), cRow(0), out, 0, -1, width); + // rows 2k - 1 and 2k lie between chroma rows k - 1 and k + for (let y = 1; y + 1 < height; y += 2) { + const k = y + 1 >> 1; + upsamplePair(Y, U, V, yRow(y), yRow(y + 1), cRow(k - 1), cRow(k), out, y * width * 4, (y + 1) * width * 4, width); + } + // the last row of an even height picture + if (!(height & 1)) upsamplePair(Y, U, V, yRow(height - 1), -1, cRow(lastChroma), cRow(lastChroma), out, (height - 1) * width * 4, -1, width); + + return out; +} + +// one pair of rows: top is nearer to the chroma row "top", bottom nearer to "current" (no bottom row if -1) +function upsamplePair(Y: Uint8Array, U: Uint8Array, V: Uint8Array, topY: number, bottomY: number, topC: number, curC: number, out: Uint8Array, topOut: number, bottomOut: number, width: number): void { + let tlu = U[topC], tlv = V[topC], lu = U[curC], lv = V[curC]; + yuvToRGBA(Y[topY], 3 * tlu + lu + 2 >> 2, 3 * tlv + lv + 2 >> 2, out, topOut); + if (bottomY >= 0) yuvToRGBA(Y[bottomY], 3 * lu + tlu + 2 >> 2, 3 * lv + tlv + 2 >> 2, out, bottomOut); + + const pairs = width - 1 >> 1; + for (let x = 1; x <= pairs; x++) { + const tu = U[topC + x], tv = V[topC + x], u = U[curC + x], v = V[curC + x]; + const avgU = tlu + tu + lu + u + 8, avgV = tlv + tv + lv + v + 8; + const d12u = avgU + 2 * (tu + lu) >> 3, d03u = avgU + 2 * (tlu + u) >> 3; + const d12v = avgV + 2 * (tv + lv) >> 3, d03v = avgV + 2 * (tlv + v) >> 3; + + yuvToRGBA(Y[topY + 2 * x - 1], d12u + tlu >> 1, d12v + tlv >> 1, out, topOut + (2 * x - 1) * 4); + yuvToRGBA(Y[topY + 2 * x], d03u + tu >> 1, d03v + tv >> 1, out, topOut + 2 * x * 4); + if (bottomY >= 0) { + yuvToRGBA(Y[bottomY + 2 * x - 1], d03u + lu >> 1, d03v + lv >> 1, out, bottomOut + (2 * x - 1) * 4); + yuvToRGBA(Y[bottomY + 2 * x], d12u + u >> 1, d12v + v >> 1, out, bottomOut + 2 * x * 4); + } + tlu = tu; tlv = tv; lu = u; lv = v; + } + + if (!(width & 1)) { + const x = width - 1; + yuvToRGBA(Y[topY + x], 3 * tlu + lu + 2 >> 2, 3 * tlv + lv + 2 >> 2, out, topOut + x * 4); + if (bottomY >= 0) yuvToRGBA(Y[bottomY + x], 3 * lu + tlu + 2 >> 2, 3 * lv + tlv + 2 >> 2, out, bottomOut + x * 4); + } +} diff --git a/src/codecs/vp8l.ts b/src/codecs/vp8l.ts new file mode 100644 index 00000000..110f8dc5 --- /dev/null +++ b/src/codecs/vp8l.ts @@ -0,0 +1,1013 @@ +/*! + * WebP lossless (VP8L) bitstream, as specified in RFC 9649 ("WebP Image Format") section 3 onwards, including its + * table of short distance codes. Copyright (c) 2024 IETF Trust and the persons identified as the document authors, + * Revised BSD License (see THIRD_PARTY_NOTICES.md) + */ + +/** decoded VP8L image: ARGB pixels (0xAARRGGBB) */ +export interface VP8LImage { + width: number; + height: number; + argb: Uint32Array; + hasAlpha: boolean; +} + +const SIGNATURE = 0x2f; + +const enum Transform { PREDICTOR, COLOR, SUBTRACT_GREEN, COLOR_INDEXING } + +// order in which the code length code lengths are stored +const CODE_LENGTH_ORDER = [17, 18, 0, 1, 2, 3, 4, 5, 16, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]; + +// the 120 short distance codes as (dx, dy) pairs, RFC 9649 section 4.2.2 +const DISTANCE_MAP = Int8Array.of( + 0, 1, 1, 0, 1, 1, -1, 1, 0, 2, 2, 0, 1, 2, -1, 2, 2, 1, -2, 1, 2, 2, -2, 2, 0, 3, 3, 0, 1, 3, -1, 3, 3, 1, -3, 1, + 2, 3, -2, 3, 3, 2, -3, 2, 0, 4, 4, 0, 1, 4, -1, 4, 4, 1, -4, 1, 3, 3, -3, 3, 2, 4, -2, 4, 4, 2, -4, 2, 0, 5, + 3, 4, -3, 4, 4, 3, -4, 3, 5, 0, 1, 5, -1, 5, 5, 1, -5, 1, 2, 5, -2, 5, 5, 2, -5, 2, 4, 4, -4, 4, 3, 5, -3, 5, + 5, 3, -5, 3, 0, 6, 6, 0, 1, 6, -1, 6, 6, 1, -6, 1, 2, 6, -2, 6, 6, 2, -6, 2, 4, 5, -4, 5, 5, 4, -5, 4, 3, 6, + -3, 6, 6, 3, -6, 3, 0, 7, 7, 0, 1, 7, -1, 7, 5, 5, -5, 5, 7, 1, -7, 1, 4, 6, -4, 6, 6, 4, -6, 4, 2, 7, -2, 7, + 7, 2, -7, 2, 3, 7, -3, 7, 7, 3, -7, 3, 5, 6, -5, 6, 6, 5, -6, 5, 8, 0, 4, 7, -4, 7, 7, 4, -7, 4, 8, 1, 8, 2, + 6, 6, -6, 6, 8, 3, 5, 7, -5, 7, 7, 5, -7, 5, 8, 4, 6, 7, -6, 7, 7, 6, -7, 6, 8, 5, 7, 7, -7, 7, 8, 6, 8, 7 +); + +const NUM_LITERAL_CODES = 256; +const NUM_LENGTH_CODES = 24; +const NUM_DISTANCE_CODES = 40; + +/** + * LSB-first bit reader + */ +export class BitReader { + data: Uint8Array; + pos = 0; + // bits not consumed yet, the next one is the lowest + private buffer = 0; + private count = 0; + + constructor(data: Uint8Array, offset = 0) { + this.data = data; + this.pos = offset; + } + + private fill(): void { + while (this.count <= 24) { + const byte = this.pos < this.data.length ? this.data[this.pos] : 0; + if (this.pos > this.data.length + 8) throw new Error('VP8L: unexpected end of data'); + this.pos++; + this.buffer |= byte << this.count; + this.count += 8; + } + } + + /** reads up to 24 bits */ + read(bits: number): number { + if (bits === 0) return 0; + if (this.count < bits) this.fill(); + const value = this.buffer & (1 << bits) - 1; + this.buffer >>>= bits; + this.count -= bits; + return value; + } + + /** the next 24 bits without consuming them */ + peek(): number { + if (this.count < 24) this.fill(); + return this.buffer & 0xffffff; + } + + skip(bits: number): void { + this.buffer >>>= bits; + this.count -= bits; + } + + /** whether reading went past the end of the data */ + get overrun(): boolean { + return this.pos - (this.count >> 3) > this.data.length; + } +} + +// two level huffman lookup table: entries are (symbol << 4 | length) for codes up to ROOT_BITS long, +// longer codes point to a second level table ((offset << 4 | bits) with the 0x8000_0000 flag) +const ROOT_BITS = 8; +const SUBTABLE = 0x40000000; + +/** canonical prefix code as lookup table */ +export interface PrefixCode { + table: Int32Array; + /** set if the code has a single symbol (which takes zero bits) */ + single: number; +} + +/** + * Builds a lookup table from code lengths (RFC 9649 section 3.7.2.1.1: canonical codes, read LSB first) + */ +export function buildCode(lengths: ArrayLike, size: number): PrefixCode { + let nonZero = 0, last = 0, maxLength = 0; + const counts = new Int32Array(16); + for (let s = 0; s < size; s++) { + const l = lengths[s]; + if (l) { + nonZero++; + last = s; + counts[l]++; + if (l > maxLength) maxLength = l; + } + } + + if (nonZero === 0) throw new Error('VP8L: empty prefix code'); + // a single used symbol is coded with zero bits + if (nonZero === 1) return { table: new Int32Array(0), single: last }; + + // first code of each length, checking that the code is complete (the spec requires it) + const next = new Int32Array(16); + let code = 0, left = 1; + for (let l = 1; l <= 15; l++) { + code = code + counts[l - 1] << 1; + next[l] = code; + left = (left << 1) - counts[l]; + if (left < 0) throw new Error('VP8L: invalid prefix code (oversubscribed)'); + } + if (left !== 0) throw new Error('VP8L: invalid prefix code (incomplete)'); + + // codes are consumed LSB first, so the table is indexed by the bit-reversed code + const reverse = (c: number, len: number): number => { + let r = 0; + for (let i = 0; i < len; i++, c >>= 1) r = r << 1 | c & 1; + return r; + }; + + // size the second level tables: for every root prefix, the longest code below it + const rootSize = 1 << ROOT_BITS; + const subBits = new Int32Array(rootSize); + const symbolCodes = new Int32Array(size); + for (let s = 0; s < size; s++) { + const l = lengths[s]; + if (!l) continue; + const c = reverse(next[l]++, l); + symbolCodes[s] = c; + if (l > ROOT_BITS) { + const root = c & rootSize - 1; + subBits[root] = Math.max(subBits[root], l - ROOT_BITS); + } + } + + let total = rootSize; + const subOffset = new Int32Array(rootSize); + for (let r = 0; r < rootSize; r++) + if (subBits[r]) { + subOffset[r] = total; + total += 1 << subBits[r]; + } + + const table = new Int32Array(total); + for (let r = 0; r < rootSize; r++) + if (subBits[r]) table[r] = SUBTABLE | subOffset[r] << 4 | subBits[r]; + + for (let s = 0; s < size; s++) { + const l = lengths[s]; + if (!l) continue; + const c = symbolCodes[s]; + if (l <= ROOT_BITS) { + for (let i = c; i < rootSize; i += 1 << l) table[i] = s << 4 | l; + } else { + const root = c & rootSize - 1; + const bits = subBits[root]; + const rest = c >> ROOT_BITS, restLength = l - ROOT_BITS; + for (let i = rest; i < 1 << bits; i += 1 << restLength) table[subOffset[root] + i] = s << 4 | l; + } + } + + return { table, single: -1 }; +} + +/** reads one symbol */ +export function readSymbol(reader: BitReader, code: PrefixCode): number { + if (code.single >= 0) return code.single; + const bits = reader.peek(); + let entry = code.table[bits & (1 << ROOT_BITS) - 1]; + if (entry & SUBTABLE) { + const offset = (entry & ~SUBTABLE) >> 4, sub = entry & 15; + entry = code.table[offset + (bits >> ROOT_BITS & (1 << sub) - 1)]; + } + reader.skip(entry & 15); + return entry >> 4; +} + +// RFC 9649 section 3.7.2.1: a prefix code is either "simple" (1-2 symbols) or "normal" (code length code) +function readCode(reader: BitReader, alphabetSize: number): PrefixCode { + const lengths = new Uint8Array(alphabetSize); + + if (reader.read(1)) { + const count = reader.read(1) + 1; + const first = reader.read(reader.read(1) ? 8 : 1); + if (first >= alphabetSize) throw new Error('VP8L: invalid simple prefix code'); + if (count === 1) return { table: new Int32Array(0), single: first }; + + const second = reader.read(8); + if (second >= alphabetSize) throw new Error('VP8L: invalid simple prefix code'); + lengths[first] = 1; + lengths[second] = 1; + if (first === second) return { table: new Int32Array(0), single: first }; + return buildCode(lengths, alphabetSize); + } + + const codeLengthLengths = new Uint8Array(19); + const count = reader.read(4) + 4; + if (count > 19) throw new Error('VP8L: invalid code length code'); + for (let i = 0; i < count; i++) codeLengthLengths[CODE_LENGTH_ORDER[i]] = reader.read(3); + const lengthCode = buildCode(codeLengthLengths, 19); + + let maxSymbol = alphabetSize; + if (reader.read(1)) { + const bits = 2 + 2 * reader.read(3); + maxSymbol = 2 + reader.read(bits); + if (maxSymbol > alphabetSize) throw new Error('VP8L: invalid max symbol'); + } + + let previous = 8; + for (let symbol = 0; symbol < alphabetSize;) { + if (maxSymbol-- === 0) break; + const length = readSymbol(reader, lengthCode); + + if (length < 16) { + lengths[symbol++] = length; + if (length) previous = length; + continue; + } + + const repeat = length === 16 ? 3 + reader.read(2) : length === 17 ? 3 + reader.read(3) : 11 + reader.read(7); + const value = length === 16 ? previous : 0; + if (symbol + repeat > alphabetSize) throw new Error('VP8L: invalid code lengths'); + lengths.fill(value, symbol, symbol + repeat); + symbol += repeat; + } + + return buildCode(lengths, alphabetSize); +} + +const subSize = (size: number, bits: number): number => (size + (1 << bits) - 1) >> bits; + +// length and distance values: a prefix symbol plus extra bits (RFC 9649 section 3.6.1) +function prefixValue(reader: BitReader, symbol: number): number { + if (symbol < 4) return symbol + 1; + const extra = (symbol - 2) >> 1; + const offset = (2 + (symbol & 1)) << extra; + return offset + reader.read(extra) + 1; +} + +/** + * Decodes the pixels of an entropy coded image: the main image (with meta prefix codes) or a transform's sub-image + */ +function decodeImage(reader: BitReader, width: number, height: number, isMain: boolean): Uint32Array { + let cacheBits = 0; + if (reader.read(1)) { + cacheBits = reader.read(4); + if (cacheBits < 1 || cacheBits > 11) throw new Error('VP8L: invalid color cache size'); + } + + let prefixBits = 0, prefixWidth = 1, groupImage: Uint32Array | null = null, groupCount = 1; + if (isMain && reader.read(1)) { + prefixBits = reader.read(3) + 2; + prefixWidth = subSize(width, prefixBits); + groupImage = decodeImage(reader, prefixWidth, subSize(height, prefixBits), false); + for (let i = 0; i < groupImage.length; i++) { + const group = groupImage[i] >> 8 & 0xffff; + groupImage[i] = group; + if (group >= groupCount) groupCount = group + 1; + } + } + + const cacheSize = cacheBits ? 1 << cacheBits : 0; + const greenSize = NUM_LITERAL_CODES + NUM_LENGTH_CODES + cacheSize; + const groups: PrefixCode[][] = []; + for (let g = 0; g < groupCount; g++) + groups.push([ + readCode(reader, greenSize), + readCode(reader, NUM_LITERAL_CODES), + readCode(reader, NUM_LITERAL_CODES), + readCode(reader, NUM_LITERAL_CODES), + readCode(reader, NUM_DISTANCE_CODES) + ]); + + const total = width * height; + const pixels = new Uint32Array(total); + const cache = cacheSize ? new Uint32Array(cacheSize) : null; + const cacheShift = 32 - cacheBits; + let cached = 0; // pixels before this index are in the cache already + + const prefixMask = (1 << prefixBits) - 1; + let group = groups[0]; + + for (let i = 0, x = 0, y = 0; i < total;) { + if (groupImage && (x & prefixMask) === 0) group = groups[groupImage[(y >> prefixBits) * prefixWidth + (x >> prefixBits)]]; + + const green = readSymbol(reader, group[0]); + if (green < NUM_LITERAL_CODES) { + const red = readSymbol(reader, group[1]); + const blue = readSymbol(reader, group[2]); + const alpha = readSymbol(reader, group[3]); + pixels[i++] = (alpha << 24 | red << 16 | green << 8 | blue) >>> 0; + if (++x === width) { x = 0; y++; } + } else if (green < NUM_LITERAL_CODES + NUM_LENGTH_CODES) { + const length = prefixValue(reader, green - NUM_LITERAL_CODES); + const distanceSymbol = readSymbol(reader, group[4]); + let distance = prefixValue(reader, distanceSymbol); + if (distance > 120) distance -= 120; + else { + const dx = DISTANCE_MAP[2 * distance - 2], dy = DISTANCE_MAP[2 * distance - 1]; + distance = Math.max(1, dy * width + dx); + } + + if (distance > i || i + length > total) throw new Error('VP8L: invalid backward reference'); + for (let k = 0; k < length; k++, i++) pixels[i] = pixels[i - distance]; + x += length; + while (x >= width) { x -= width; y++; } + // the prefix group can change within the copied run, it is looked up again at the next block start + if (groupImage) group = groups[groupImage[(y >> prefixBits) * prefixWidth + (x >> prefixBits)]]; + } else { + if (!cache) throw new Error('VP8L: invalid color cache symbol'); + // bring the cache up to date before looking anything up in it + for (; cached < i; cached++) cache[Math.imul(pixels[cached], 0x1e35a7bd) >>> cacheShift] = pixels[cached]; + pixels[i++] = cache[green - NUM_LITERAL_CODES - NUM_LENGTH_CODES]; + if (++x === width) { x = 0; y++; } + } + + if (reader.overrun) throw new Error('VP8L: unexpected end of data'); + } + + return pixels; +} + +const average2 = (a: number, b: number): number => (((a ^ b) & 0xfefefefe) >>> 1) + (a & b) >>> 0; + +// per channel (a + b - c) clamped to 0..255 +function clampAddSubtractFull(a: number, b: number, c: number): number { + let out = 0; + for (let shift = 0; shift < 32; shift += 8) { + const v = (a >>> shift & 255) + (b >>> shift & 255) - (c >>> shift & 255); + out |= (v < 0 ? 0 : v > 255 ? 255 : v) << shift; + } + return out >>> 0; +} + +// per channel a + (a - b) / 2 clamped to 0..255 (C division, truncating) +function clampAddSubtractHalf(a: number, b: number): number { + let out = 0; + for (let shift = 0; shift < 32; shift += 8) { + const ac = a >>> shift & 255, d = ac - (b >>> shift & 255); + const v = ac + (d / 2 | 0); + out |= (v < 0 ? 0 : v > 255 ? 255 : v) << shift; + } + return out >>> 0; +} + +function select(l: number, t: number, tl: number): number { + let pl = 0, pt = 0; + for (let shift = 0; shift < 32; shift += 8) { + const lc = l >>> shift & 255, tc = t >>> shift & 255, tlc = tl >>> shift & 255; + // distance of the estimate (l + t - tl) to l and to t + pl += Math.abs(tc - tlc); + pt += Math.abs(lc - tlc); + } + return pl < pt ? l : t; +} + +// adds two ARGB pixels per channel, modulo 256 +const addPixels = (a: number, b: number): number => ((a & 0xff00ff00) + (b & 0xff00ff00) & 0xff00ff00 | (a & 0x00ff00ff) + (b & 0x00ff00ff) & 0x00ff00ff) >>> 0; + +function predict(mode: number, p: Uint32Array, i: number, width: number): number { + const l = p[i - 1], t = p[i - width], tl = p[i - width - 1], tr = p[i - width + 1]; + switch (mode) { + case 0: return 0xff000000; + case 1: return l; + case 2: return t; + case 3: return tr; + case 4: return tl; + case 5: return average2(average2(l, tr), t); + case 6: return average2(l, tl); + case 7: return average2(l, t); + case 8: return average2(tl, t); + case 9: return average2(t, tr); + case 10: return average2(average2(l, tl), average2(t, tr)); + case 11: return select(l, t, tl); + case 12: return clampAddSubtractFull(l, t, tl); + case 13: return clampAddSubtractHalf(average2(l, t), tl); + default: return 0xff000000; // modes 14 and 15 aren't defined, libwebp predicts black like mode 0 + } +} + +function inversePredictor(p: Uint32Array, width: number, height: number, bits: number, modes: Uint32Array): void { + const blocksPerRow = subSize(width, bits); + + // top left pixel: black, rest of the first row: left + p[0] = addPixels(p[0], 0xff000000); + for (let x = 1; x < width; x++) p[x] = addPixels(p[x], p[x - 1]); + + for (let y = 1; y < height; y++) { + const row = y * width; + // first column: top + p[row] = addPixels(p[row], p[row - width]); + const modeRow = (y >> bits) * blocksPerRow; + for (let x = 1; x < width; x++) { + const mode = modes[modeRow + (x >> bits)] >> 8 & 15; + const i = row + x; + p[i] = addPixels(p[i], predict(mode, p, i, width)); + } + } +} + +// (t * c) >> 5 with both as signed 8 bit values +const colorDelta = (t: number, c: number): number => ((t << 24 >> 24) * (c << 24 >> 24)) >> 5; + +function inverseColorTransform(p: Uint32Array, width: number, height: number, bits: number, elements: Uint32Array): void { + const blocksPerRow = subSize(width, bits); + for (let y = 0; y < height; y++) { + const elementRow = (y >> bits) * blocksPerRow; + for (let x = 0; x < width; x++) { + const e = elements[elementRow + (x >> bits)]; + const greenToRed = e & 255, greenToBlue = e >> 8 & 255, redToBlue = e >> 16 & 255; + const i = y * width + x, argb = p[i]; + const green = argb >> 8 & 255; + const red = (argb >> 16) + colorDelta(greenToRed, green) & 255; + const blue = (argb & 255) + colorDelta(greenToBlue, green) + colorDelta(redToBlue, red) & 255; + p[i] = (argb & 0xff00ff00 | red << 16 | blue) >>> 0; + } + } +} + +function addGreen(p: Uint32Array): void { + for (let i = 0; i < p.length; i++) { + const argb = p[i], green = argb >> 8 & 255; + const redBlue = (argb & 0x00ff00ff) + (green << 16 | green) & 0x00ff00ff; + p[i] = (argb & 0xff00ff00 | redBlue) >>> 0; + } +} + +function inverseColorIndexing(packed: Uint32Array, packedWidth: number, width: number, height: number, bits: number, palette: Uint32Array): Uint32Array { + const out = new Uint32Array(width * height); + const lookup = (index: number): number => index < palette.length ? palette[index] : 0; + + if (bits === 0) { + for (let i = 0; i < out.length; i++) out[i] = lookup(packed[i] >> 8 & 255); + return out; + } + + const perByte = 1 << bits, bitsPerIndex = 8 >> bits, mask = (1 << bitsPerIndex) - 1; + for (let y = 0; y < height; y++) + for (let x = 0; x < width; x++) { + const green = packed[y * packedWidth + (x >> bits)] >> 8 & 255; + out[y * width + x] = lookup(green >> (x & perByte - 1) * bitsPerIndex & mask); + } + + return out; +} + +interface TransformData { + type: Transform; + width: number; + bits: number; + data: Uint32Array; +} + +/** + * Reads the VP8L header (signature, dimensions) + */ +export function readHeader(data: Uint8Array): { width: number, height: number, hasAlpha: boolean } { + if (data.length < 5 || data[0] !== SIGNATURE) throw new Error('VP8L: invalid signature'); + const reader = new BitReader(data, 1); + const width = reader.read(14) + 1; + const height = reader.read(14) + 1; + const hasAlpha = reader.read(1) === 1; + if (reader.read(3) !== 0) throw new Error('VP8L: unsupported version'); + return { width, height, hasAlpha }; +} + +/** + * Decodes a VP8L bitstream (the payload of a VP8L chunk) + */ +export function decode(data: Uint8Array): VP8LImage { + const { width, height, hasAlpha } = readHeader(data); + const reader = new BitReader(data, 5); + return { width, height, hasAlpha, argb: decodeStream(reader, width, height) }; +} + +/** + * Decodes transforms and the main image from the bit reader (used by the ALPH chunk too, which has no header) + */ +export function decodeStream(reader: BitReader, width: number, height: number): Uint32Array { + const transforms: TransformData[] = []; + let codedWidth = width; + const seen = new Set(); + + while (reader.read(1)) { + const type = reader.read(2) as Transform; + if (seen.has(type)) throw new Error('VP8L: transform used twice'); + seen.add(type); + + if (type === Transform.PREDICTOR || type === Transform.COLOR) { + const bits = reader.read(3) + 2; + const data = decodeImage(reader, subSize(codedWidth, bits), subSize(height, bits), false); + transforms.push({ type, width: codedWidth, bits, data }); + } else if (type === Transform.SUBTRACT_GREEN) { + transforms.push({ type, width: codedWidth, bits: 0, data: new Uint32Array(0) }); + } else { + const size = reader.read(8) + 1; + const palette = decodeImage(reader, size, 1, false); + // the palette is stored as differences to the previous entry + for (let i = 1; i < size; i++) palette[i] = addPixels(palette[i], palette[i - 1]); + const bits = size <= 2 ? 3 : size <= 4 ? 2 : size <= 16 ? 1 : 0; + transforms.push({ type, width: codedWidth, bits, data: palette }); + codedWidth = subSize(codedWidth, bits); + } + } + + let pixels = decodeImage(reader, codedWidth, height, true); + + for (let t = transforms.length - 1; t >= 0; t--) { + const { type, width: w, bits, data } = transforms[t]; + if (type === Transform.PREDICTOR) inversePredictor(pixels, w, height, bits, data); + else if (type === Transform.COLOR) inverseColorTransform(pixels, w, height, bits, data); + else if (type === Transform.SUBTRACT_GREEN) addGreen(pixels); + else pixels = inverseColorIndexing(pixels, subSize(w, bits), w, height, bits, data); + } + + return pixels; +} + +// --------------------------------------------------------------------------- +// encoder +// --------------------------------------------------------------------------- + +/** + * LSB-first bit writer + */ +class BitWriter { + private bytes = new Uint8Array(1 << 16); + private length = 0; + private buffer = 0; + private count = 0; + + write(value: number, bits: number): void { + while (bits > 0) { + const take = Math.min(bits, 24 - this.count); + this.buffer |= (value & (1 << take) - 1) << this.count; + this.count += take; + value >>>= take; + bits -= take; + while (this.count >= 8) { + if (this.length === this.bytes.length) { + const grown = new Uint8Array(this.bytes.length * 2); + grown.set(this.bytes); + this.bytes = grown; + } + this.bytes[this.length++] = this.buffer & 255; + this.buffer >>>= 8; + this.count -= 8; + } + } + } + + finish(): Uint8Array { + if (this.count > 0) this.write(0, 8 - this.count); + return this.bytes.slice(0, this.length); + } +} + +/** + * Huffman code lengths for the given symbol counts, limited to maxLength bits + */ +function codeLengths(counts: ArrayLike, maxLength: number): Uint8Array { + const size = counts.length; + const lengths = new Uint8Array(size); + let freq = Array.from(counts); + + for (; ;) { + const used: number[] = []; + for (let s = 0; s < size; s++) if (freq[s] > 0) used.push(s); + if (used.length === 0) return lengths; + if (used.length === 1) { + lengths[used[0]] = 1; + return lengths; + } + + // two queue huffman construction over the sorted leaves + used.sort((a, b) => freq[a] - freq[b] || a - b); + const n = used.length; + const weight = new Float64Array(2 * n), parent = new Int32Array(2 * n); + for (let i = 0; i < n; i++) weight[i] = freq[used[i]]; + let leaf = 0, node = n, next = n; + const pick = (): number => { + if (leaf < n && (node >= next || weight[leaf] <= weight[node])) return leaf++; + return node++; + }; + for (; next < 2 * n - 1; next++) { + const a = pick(), b = pick(); + weight[next] = weight[a] + weight[b]; + parent[a] = parent[b] = next; + } + + let longest = 0; + const depth = new Int32Array(2 * n); + for (let i = 2 * n - 3; i >= 0; i--) depth[i] = depth[parent[i]] + 1; + for (let i = 0; i < n; i++) { + lengths[used[i]] = depth[i]; + if (depth[i] > longest) longest = depth[i]; + } + if (longest <= maxLength) return lengths; + + // too deep: flatten the distribution and try again + lengths.fill(0); + freq = freq.map(f => f > 0 ? (f >> 1) + 1 : 0); + } +} + +/** canonical codes (LSB-first, i.e. bit-reversed) for the given lengths */ +function canonicalCodes(lengths: Uint8Array): Uint16Array { + const counts = new Int32Array(16), next = new Int32Array(16); + for (const l of lengths) if (l) counts[l]++; + let code = 0; + for (let l = 1; l <= 15; l++) { + code = code + counts[l - 1] << 1; + next[l] = code; + } + const codes = new Uint16Array(lengths.length); + for (let s = 0; s < lengths.length; s++) { + const l = lengths[s]; + if (!l) continue; + let c = next[l]++, r = 0; + for (let i = 0; i < l; i++, c >>= 1) r = r << 1 | c & 1; + codes[s] = r; + } + return codes; +} + +interface Code { + lengths: Uint8Array; + codes: Uint16Array; +} + +function buildEncodeCode(counts: ArrayLike, maxLength = 15): Code { + const lengths = codeLengths(counts, maxLength); + return { lengths, codes: canonicalCodes(lengths) }; +} + +const writeSymbol = (w: BitWriter, code: Code, symbol: number): void => { + const l = code.lengths[symbol]; + // a code with a single used symbol has length 1 here but is written as a zero bit code + if (l && code.codes.length) w.write(code.codes[symbol], l); +}; + +/** + * Writes a prefix code (RFC 9649 section 3.7.2.1) and returns the code to write symbols with + */ +function writeCode(w: BitWriter, counts: ArrayLike): Code { + const used: number[] = []; + for (let s = 0; s < counts.length; s++) if (counts[s] > 0) used.push(s); + + // simple codes: up to two symbols below 256; a single symbol takes zero bits + if (used.length <= 2 && used.every(s => s < 256)) { + const symbols = used.length ? used : [0]; + w.write(1, 1); + w.write(symbols.length - 1, 1); + if (symbols[0] <= 1) { + w.write(0, 1); + w.write(symbols[0], 1); + } else { + w.write(1, 1); + w.write(symbols[0], 8); + } + if (symbols.length === 2) w.write(symbols[1], 8); + + const lengths = new Uint8Array(counts.length); + if (symbols.length === 1) return { lengths, codes: new Uint16Array(0) }; + lengths[symbols[0]] = lengths[symbols[1]] = 1; + return { lengths, codes: canonicalCodes(lengths) }; + } + + const code = buildEncodeCode(counts); + // a normal code needs at least two symbols with non-zero length, which is given here + + // run length encode the code lengths: 16 repeats the previous non-zero length, 17/18 are runs of zeros + const tokens: number[] = [], extras: number[] = []; + const lengths = code.lengths; + let previous = 8; + for (let i = 0; i < lengths.length;) { + const l = lengths[i]; + let run = 1; + while (i + run < lengths.length && lengths[i + run] === l) run++; + + if (l === 0) { + let left = run; + while (left > 0) { + if (left < 3) { tokens.push(0); extras.push(0); left--; } + else if (left <= 10) { tokens.push(17); extras.push(left - 3); left = 0; } + else { const n = Math.min(left, 138); tokens.push(18); extras.push(n - 11); left -= n; } + } + } else { + let left = run; + if (l !== previous) { tokens.push(l); extras.push(0); left--; previous = l; } + while (left > 0) { + if (left < 3) { tokens.push(l); extras.push(0); left--; } + else { const n = Math.min(left, 6); tokens.push(16); extras.push(n - 3); left -= n; } + } + } + i += run; + } + + const lengthCounts = new Int32Array(19); + for (const t of tokens) lengthCounts[t]++; + const lengthCode = buildEncodeCode(lengthCounts, 7); + // the code length code needs two used symbols too, unless only one is used (then it takes zero bits) + let lengthCodeLengths = lengthCode.lengths; + if (lengthCodeLengths.filter(l => l).length === 1) { + // a single code length symbol: give it a sibling so the code is complete + lengthCodeLengths = lengthCodeLengths.slice(); + const only = lengthCodeLengths.findIndex(l => l !== 0); + lengthCodeLengths[only] = 1; + lengthCodeLengths[only === 0 ? 1 : 0] = 1; + } + const lengthCodes = canonicalCodes(lengthCodeLengths); + + let count = 19; + while (count > 4 && lengthCodeLengths[CODE_LENGTH_ORDER[count - 1]] === 0) count--; + w.write(0, 1); // normal code + w.write(count - 4, 4); + for (let i = 0; i < count; i++) w.write(lengthCodeLengths[CODE_LENGTH_ORDER[i]], 3); + w.write(0, 1); // max_symbol: the whole alphabet + + for (let i = 0; i < tokens.length; i++) { + const t = tokens[i]; + w.write(lengthCodes[t], lengthCodeLengths[t]); + if (t === 16) w.write(extras[i], 2); + else if (t === 17) w.write(extras[i], 3); + else if (t === 18) w.write(extras[i], 7); + } + + return code; +} + +// prefix symbol and extra bits of a length or distance value (>= 1) +function prefixEncode(value: number): { symbol: number, bits: number, extra: number } { + const d = value - 1; + if (d < 4) return { symbol: d, bits: 0, extra: 0 }; + const h = 31 - Math.clz32(d); + const second = d >> h - 1 & 1; + return { symbol: 2 * h + second, bits: h - 1, extra: d & (1 << h - 1) - 1 }; +} + +// encoded stream symbols: literals (argb), backward references, cache hits +const enum Op { LITERAL, COPY, CACHE } + +interface Stream { + ops: Uint8Array; + values: Uint32Array; // argb, length, cache index + distances: Uint32Array; // distance codes for copies + count: number; +} + +/** + * Finds backward references (greedy, hash chains over pixel pairs) and color cache hits + */ +function tokenize(pixels: Uint32Array, width: number, cacheBits: number): Stream { + const n = pixels.length; + const ops = new Uint8Array(n), values = new Uint32Array(n), distances = new Uint32Array(n); + let count = 0; + + // distance codes of the 120 short 2D distances for this width + const shortCodes = new Map(); + for (let c = 119; c >= 0; c--) { + const d = DISTANCE_MAP[2 * c + 1] * width + DISTANCE_MAP[2 * c]; + if (d >= 1) shortCodes.set(d, c + 1); + } + const distanceCode = (d: number): number => shortCodes.get(d) ?? d + 120; + + // distances are coded with at most 20 bits (minus the 120 short codes) + const HASH_BITS = 15, CHAIN = 24, MAX_LENGTH = 4096, MIN_LENGTH = 3, WINDOW = (1 << 20) - 120; + const head = new Int32Array(1 << HASH_BITS).fill(-1); + const chain = new Int32Array(n); + const hash = (i: number): number => Math.imul(Math.imul(pixels[i], 0x1e35a7bd) ^ pixels[i + 1], 0x9e3779b1) >>> 32 - HASH_BITS; + const insert = (i: number): void => { + if (i + 1 >= n) return; + const h = hash(i); + chain[i] = head[h]; + head[h] = i; + }; + const matchLength = (i: number, j: number): number => { + const max = Math.min(MAX_LENGTH, n - i); + let l = 0; + while (l < max && pixels[i + l] === pixels[j + l]) l++; + return l; + }; + + const cache = cacheBits ? new Uint32Array(1 << cacheBits) : null; + const cacheShift = 32 - cacheBits; + let cached = 0; + + for (let i = 0; i < n;) { + let bestLength = 0, bestDistance = 0; + + // the pixel to the left and the one above are the cheapest references + for (const d of [1, width]) { + if (d > i) continue; + const l = matchLength(i, i - d); + if (l > bestLength) { bestLength = l; bestDistance = d; } + } + if (i + 1 < n && bestLength < MAX_LENGTH) { + for (let j = head[hash(i)], k = 0; j >= 0 && k < CHAIN && i - j <= WINDOW; j = chain[j], k++) { + const l = matchLength(i, j); + // longer, or as long and closer + if (l > bestLength || l === bestLength && l > 0 && i - j < bestDistance) { bestLength = l; bestDistance = i - j; } + if (l === MAX_LENGTH) break; + } + } + + if (bestLength >= MIN_LENGTH) { + ops[count] = Op.COPY; + values[count] = bestLength; + distances[count++] = distanceCode(bestDistance); + for (let k = 0; k < bestLength; k++) insert(i + k); + i += bestLength; + continue; + } + + const argb = pixels[i]; + if (cache) { + for (; cached < i; cached++) cache[Math.imul(pixels[cached], 0x1e35a7bd) >>> cacheShift] = pixels[cached]; + const key = Math.imul(argb, 0x1e35a7bd) >>> cacheShift; + if (cache[key] === argb) { + ops[count] = Op.CACHE; + values[count++] = key; + insert(i++); + continue; + } + } + ops[count] = Op.LITERAL; + values[count++] = argb; + insert(i++); + } + + return { ops, values, distances, count }; +} + +/** + * Writes an entropy coded image (main image with a single prefix group, or a transform's sub-image) + */ +function writeImage(w: BitWriter, pixels: Uint32Array, width: number, isMain: boolean): void { + // the color cache pays off for photos, sub-images are tiny + const cacheBits = isMain && pixels.length > 4096 ? 10 : 0; + const stream = tokenize(pixels, width, cacheBits); + + if (cacheBits) { + w.write(1, 1); + w.write(cacheBits, 4); + } else w.write(0, 1); + if (isMain) w.write(0, 1); // no meta prefix codes + + const greenSize = NUM_LITERAL_CODES + NUM_LENGTH_CODES + (cacheBits ? 1 << cacheBits : 0); + const counts = [new Int32Array(greenSize), new Int32Array(256), new Int32Array(256), new Int32Array(256), new Int32Array(NUM_DISTANCE_CODES)]; + const { ops, values, distances, count } = stream; + for (let k = 0; k < count; k++) { + const v = values[k]; + if (ops[k] === Op.LITERAL) { + counts[0][v >> 8 & 255]++; + counts[1][v >> 16 & 255]++; + counts[2][v & 255]++; + counts[3][v >>> 24]++; + } else if (ops[k] === Op.COPY) { + counts[0][NUM_LITERAL_CODES + prefixEncode(v).symbol]++; + counts[4][prefixEncode(distances[k]).symbol]++; + } else counts[0][NUM_LITERAL_CODES + NUM_LENGTH_CODES + v]++; + } + + const codes = counts.map(c => writeCode(w, c)); + for (let k = 0; k < count; k++) { + const v = values[k]; + if (ops[k] === Op.LITERAL) { + writeSymbol(w, codes[0], v >> 8 & 255); + writeSymbol(w, codes[1], v >> 16 & 255); + writeSymbol(w, codes[2], v & 255); + writeSymbol(w, codes[3], v >>> 24); + } else if (ops[k] === Op.COPY) { + const length = prefixEncode(v), distance = prefixEncode(distances[k]); + writeSymbol(w, codes[0], NUM_LITERAL_CODES + length.symbol); + w.write(length.extra, length.bits); + writeSymbol(w, codes[4], distance.symbol); + w.write(distance.extra, distance.bits); + } else writeSymbol(w, codes[0], NUM_LITERAL_CODES + NUM_LENGTH_CODES + v); + } +} + +// subtracts two ARGB pixels per channel, modulo 256 +const subPixels = (a: number, b: number): number => ((a | 0x00ff00ff) - (b & 0xff00ff00) & 0xff00ff00 | (a | 0xff00ff00) - (b & 0x00ff00ff) & 0x00ff00ff) >>> 0; + +// cost estimate of a residual: sum of the channels' distances to zero (as signed bytes) +function residualCost(r: number): number { + let cost = 0; + for (let shift = 0; shift < 32; shift += 8) { + const c = r >>> shift & 255; + cost += c < 128 ? c : 256 - c; + } + return cost; +} + +/** + * Picks a predictor per block and replaces the pixels with their residuals + * @return the sub-image of modes + */ +function applyPredictor(p: Uint32Array, width: number, height: number, bits: number): Uint32Array { + const blocksX = subSize(width, bits), blocksY = subSize(height, bits); + const modes = new Uint32Array(blocksX * blocksY); + const size = 1 << bits; + + // predictions use the original pixels, so modes are chosen and residuals computed on a copy + const src = p.slice(); + const predictAt = (mode: number, i: number, x: number, y: number): number => + y === 0 ? (x === 0 ? 0xff000000 : src[i - 1]) : x === 0 ? src[i - width] : predict(mode, src, i, width); + + for (let by = 0; by < blocksY; by++) + for (let bx = 0; bx < blocksX; bx++) { + let best = 0, bestCost = Infinity; + const x0 = bx * size, y0 = by * size, x1 = Math.min(width, x0 + size), y1 = Math.min(height, y0 + size); + for (let mode = 0; mode < 14; mode++) { + let cost = 0; + for (let y = y0; y < y1 && cost < bestCost; y++) + for (let x = x0; x < x1; x++) { + const i = y * width + x; + cost += residualCost(subPixels(src[i], predictAt(mode, i, x, y))); + } + if (cost < bestCost) { bestCost = cost; best = mode; } + } + modes[by * blocksX + bx] = (0xff000000 | best << 8) >>> 0; + for (let y = y0; y < y1; y++) + for (let x = x0; x < x1; x++) { + const i = y * width + x; + p[i] = subPixels(src[i], predictAt(best, i, x, y)); + } + } + + return modes; +} + +/** + * Encodes ARGB pixels into a VP8L bitstream (the payload of a VP8L chunk) + */ +export function encode(argb: Uint32Array, width: number, height: number): Uint8Array { + if (width < 1 || height < 1 || width > 16384 || height > 16384) throw new RangeError('VP8L: dimensions must be 1-16384'); + const w = new BitWriter(); + let hasAlpha = false; + for (let i = 0; i < argb.length; i++) if (argb[i] >>> 24 !== 255) { hasAlpha = true; break; } + + w.write(SIGNATURE, 8); + w.write(width - 1, 14); + w.write(height - 1, 14); + w.write(hasAlpha ? 1 : 0, 1); + w.write(0, 3); + + // up to 256 colors: a palette, with several pixels packed per byte for up to 16 colors + const colors = new Map(); + for (let i = 0; i < argb.length && colors.size <= 256; i++) if (!colors.has(argb[i])) colors.set(argb[i], colors.size); + + let pixels: Uint32Array, codedWidth = width; + if (colors.size <= 256) { + // sorted palettes delta-code better + const palette = Uint32Array.from([...colors.keys()].sort((a, b) => a - b)); + palette.forEach((c, i) => colors.set(c, i)); + const bits = palette.length <= 2 ? 3 : palette.length <= 4 ? 2 : palette.length <= 16 ? 1 : 0; + + w.write(1, 1); + w.write(Transform.COLOR_INDEXING, 2); + w.write(palette.length - 1, 8); + const deltas = palette.slice(); + for (let i = palette.length - 1; i > 0; i--) deltas[i] = subPixels(palette[i], palette[i - 1]); + writeImage(w, deltas, palette.length, false); + + codedWidth = subSize(width, bits); + pixels = new Uint32Array(codedWidth * height); + const perPixel = 8 >> bits; + for (let y = 0; y < height; y++) + for (let x = 0; x < width; x++) { + const o = y * codedWidth + (x >> bits); + pixels[o] = (pixels[o] | 0xff000000 | (colors.get(argb[y * width + x]) as number) << 8 + (x & (1 << bits) - 1) * perPixel) >>> 0; + } + } else { + pixels = argb.slice(); + w.write(1, 1); + w.write(Transform.SUBTRACT_GREEN, 2); + for (let i = 0; i < pixels.length; i++) { + const p = pixels[i], green = p >> 8 & 255; + // the set bits between red and blue absorb blue's borrow + pixels[i] = (p & 0xff00ff00 | (p | 0x0000ff00) - (green << 16 | green) & 0x00ff00ff) >>> 0; + } + + const bits = 4; + w.write(1, 1); + w.write(Transform.PREDICTOR, 2); + w.write(bits - 2, 3); + const modes = applyPredictor(pixels, width, height, bits); + writeImage(w, modes, subSize(width, bits), false); + } + + w.write(0, 1); // no more transforms + writeImage(w, pixels, codedWidth, true); + return w.finish(); +} diff --git a/src/codecs/webp.ts b/src/codecs/webp.ts new file mode 100644 index 00000000..b5f3825e --- /dev/null +++ b/src/codecs/webp.ts @@ -0,0 +1,360 @@ +/*! + * WebP container. Animation frames are blended with libwebp's integer arithmetic (Copyright (c) 2010 Google Inc., BSD + * license, see THIRD_PARTY_NOTICES.md) + */ +// RIFF container (RFC 9649, https://www.rfc-editor.org/rfc/rfc9649): simple lossy/lossless files, extended files with +// alpha and animation +import * as vp8l from './vp8l.ts'; +import * as vp8 from './vp8.ts'; +import { orientation as exifOrientation } from './exif.ts'; + +const fourCC = (s: string): number => s.charCodeAt(0) | s.charCodeAt(1) << 8 | s.charCodeAt(2) << 16 | s.charCodeAt(3) << 24; +const RIFF = fourCC('RIFF'), WEBP = fourCC('WEBP'); +const VP8 = fourCC('VP8 '), VP8L = fourCC('VP8L'), VP8X = fourCC('VP8X'); +const ALPH = fourCC('ALPH'), ANIM = fourCC('ANIM'), ANMF = fourCC('ANMF'), EXIF = fourCC('EXIF'); + +export interface WebPFrame { + x: number; + y: number; + width: number; + height: number; + /** milliseconds */ + duration: number; + /** whether the frame is alpha blended onto the canvas (false: it replaces the covered area) */ + blend: boolean; + /** whether the frame's area is cleared to transparent after it was shown */ + dispose: boolean; + /** RGBA */ + pixels: Uint8Array; +} + +export interface WebPAnimation { + width: number; + height: number; + /** 0 = forever */ + loopCount: number; + /** ARGB background color hint (decoders, like browsers, are free to ignore it) */ + background: number; + frames: WebPFrame[]; +} + +interface Chunk { + type: number; + data: Uint8Array; +} + +function chunks(data: Uint8Array, start: number, end: number): Chunk[] { + const view = new DataView(data.buffer, data.byteOffset, data.byteLength); + const out: Chunk[] = []; + let offset = start; + while (offset + 8 <= end) { + const type = view.getUint32(offset, true); + const size = view.getUint32(offset + 4, true); + const dataStart = offset + 8; + // tolerate a truncated last chunk, the bitstream decoders report missing data + out.push({ type, data: data.subarray(dataStart, Math.min(end, dataStart + size)) }); + offset = dataStart + size + (size & 1); + } + return out; +} + +function container(data: Uint8Array): Chunk[] { + if (data.length < 12) throw new Error('WebP: invalid header'); + const view = new DataView(data.buffer, data.byteOffset, data.byteLength); + if (view.getUint32(0, true) !== RIFF || view.getUint32(8, true) !== WEBP) throw new Error('WebP: invalid header'); + const size = view.getUint32(4, true); + return chunks(data, 12, Math.min(data.length, 8 + size)); +} + +const u24 = (d: Uint8Array, o: number): number => d[o] | d[o + 1] << 8 | d[o + 2] << 16; + +/** + * Decodes a frame's image data: a VP8L chunk, or a VP8 chunk with an optional ALPH chunk before it + */ +function decodeBitstream(list: Chunk[]): { width: number, height: number, pixels: Uint8Array } { + let alpha: Chunk | null = null; + for (const chunk of list) { + if (chunk.type === ALPH) alpha ??= chunk; + else if (chunk.type === VP8L) { + const { width, height, argb } = vp8l.decode(chunk.data); + // ARGB -> RGBA in memory, one 32-bit word per pixel (written little endian, the byte order doesn't matter) + const pixels = new Uint8Array(width * height * 4); + const out = new DataView(pixels.buffer); + for (let i = 0; i < argb.length; i++) { + const p = argb[i]; + out.setUint32(i * 4, (p << 8 | p >>> 24) >>> 0); + } + return { width, height, pixels }; + } else if (chunk.type === VP8) { + const image = vp8.decode(chunk.data); + if (alpha) applyAlpha(image.pixels, image.width, image.height, alpha.data); + return image; + } + } + throw new Error('WebP: no image data'); +} + +// ALPH chunk, RFC 9649 section 2.7.1.2: preprocessing (ignored), filtering, compression (raw or VP8L) +function applyAlpha(rgba: Uint8Array, width: number, height: number, chunk: Uint8Array): void { + if (!chunk.length) return; + const header = chunk[0]; + const compression = header & 3, filter = header >> 2 & 3; + const total = width * height; + let alpha: Uint8Array; + + if (compression === 0) { + alpha = new Uint8Array(total); + alpha.set(chunk.subarray(1, 1 + total)); + } else if (compression === 1) { + // a VP8L stream without its header, the alpha values are in the green channel + const argb = vp8l.decodeStream(new vp8l.BitReader(chunk, 1), width, height); + alpha = new Uint8Array(total); + for (let i = 0; i < total; i++) alpha[i] = argb[i] >> 8 & 255; + } else throw new Error('WebP: invalid alpha compression'); + + if (filter) unfilterAlpha(alpha, width, height, filter); + for (let i = 0; i < total; i++) rgba[i * 4 + 3] = alpha[i]; +} + +// alpha filtering: 1 horizontal, 2 vertical, 3 gradient; the first row and column predict from left/top +function unfilterAlpha(a: Uint8Array, width: number, height: number, filter: number): void { + for (let y = 0; y < height; y++) { + const row = y * width; + for (let x = 0; x < width; x++) { + const i = row + x; + let prediction: number; + if (x === 0 && y === 0) prediction = 0; + else if (y === 0) prediction = a[i - 1]; + else if (x === 0) prediction = a[i - width]; + else if (filter === 1) prediction = a[i - 1]; + else if (filter === 2) prediction = a[i - width]; + else { + const g = a[i - 1] + a[i - width] - a[i - width - 1]; + prediction = g < 0 ? 0 : g > 255 ? 255 : g; + } + a[i] = a[i] + prediction & 255; + } + } +} + +/** Whether the WebP is animated */ +export function isAnimated(data: Uint8Array): boolean { + const list = container(data); + const header = list[0]; + return header && header.type === VP8X && header.data.length >= 10 && (header.data[0] & 2) !== 0 && list.some(c => c.type === ANMF); +} + +/** + * Decodes a still WebP into RGBA pixels. Animated WebPs decode to their first frame composited on the canvas + */ +export function decode(data: Uint8Array): { width: number, height: number, pixels: Uint8Array, orientation: number } { + const list = container(data); + if (!list.length) throw new Error('WebP: no image data'); + + let orientation = 1; + const exif = list.find(c => c.type === EXIF); + if (exif) orientation = exifOrientation(exif.data); + + if (list.some(c => c.type === ANMF)) { + const animation = decodeAnimation(data, true); + const frame = composite(animation)[0]; + return { width: animation.width, height: animation.height, pixels: frame.pixels, orientation }; + } + + const image = decodeBitstream(list); + if (list[0].type === VP8X && list[0].data.length >= 10) { + // the canvas size wins over the bitstream's, they have to match + const width = u24(list[0].data, 4) + 1, height = u24(list[0].data, 7) + 1; + if (width !== image.width || height !== image.height) throw new Error('WebP: canvas and image sizes differ'); + } + return { ...image, orientation }; +} + +/** + * Decodes an animated WebP into its frames (not composited); still WebPs become a single frame + */ +export function decodeAnimation(data: Uint8Array, onlyFirstFrame = false): WebPAnimation { + const list = container(data); + const header = list[0]; + if (!header || header.type !== VP8X || !list.some(c => c.type === ANMF)) { + const { width, height, pixels } = decodeBitstream(list); + return { width, height, loopCount: 0, background: 0, frames: [{ x: 0, y: 0, width, height, duration: 0, blend: false, dispose: false, pixels }] }; + } + + if (header.data.length < 10) throw new Error('WebP: invalid VP8X chunk'); + const width = u24(header.data, 4) + 1, height = u24(header.data, 7) + 1; + let loopCount = 0, background = 0; + const frames: WebPFrame[] = []; + + for (const chunk of list) { + if (chunk.type === ANIM && chunk.data.length >= 6) { + const d = chunk.data; + // stored as BGRA + background = (d[3] << 24 | d[2] << 16 | d[1] << 8 | d[0]) >>> 0; + loopCount = d[4] | d[5] << 8; + } else if (chunk.type === ANMF) { + const d = chunk.data; + if (d.length < 16) throw new Error('WebP: invalid ANMF chunk'); + const frame = decodeBitstream(chunks(d, 16, d.length)); + frames.push({ + x: u24(d, 0) * 2, + y: u24(d, 3) * 2, + width: frame.width, + height: frame.height, + duration: u24(d, 12), + blend: (d[15] & 2) === 0, + dispose: (d[15] & 1) === 1, + pixels: frame.pixels + }); + if (onlyFirstFrame) break; + } + } + + if (!frames.length) throw new Error('WebP: animation without frames'); + return { width, height, loopCount, background, frames }; +} + +/** + * Composites animation frames onto the canvas like browsers do (transparent background, the background color hint + * is ignored as the spec allows) + */ +export function composite({ width, height, frames }: WebPAnimation): Array<{ pixels: Uint8Array, duration: number }> { + const canvas = new Uint8Array(width * height * 4); + const output = []; + + for (const frame of frames) { + const x0 = Math.min(frame.x, width), x1 = Math.min(frame.x + frame.width, width); + const y0 = Math.min(frame.y, height), y1 = Math.min(frame.y + frame.height, height); + + for (let y = y0; y < y1; y++) + for (let x = x0; x < x1; x++) { + const s = ((y - frame.y) * frame.width + x - frame.x) * 4, d = (y * width + x) * 4; + const sa = frame.pixels[s + 3]; + // nothing to blend with on a transparent canvas pixel (blending would only add rounding errors) + if (!frame.blend || sa === 255 || canvas[d + 3] === 0) { + canvas[d] = frame.pixels[s]; + canvas[d + 1] = frame.pixels[s + 1]; + canvas[d + 2] = frame.pixels[s + 2]; + canvas[d + 3] = sa; + } else if (sa !== 0) blendOver(frame.pixels, s, canvas, d); + } + + output.push({ pixels: canvas.slice(), duration: frame.duration }); + + if (frame.dispose) + for (let y = y0; y < y1; y++) canvas.fill(0, (y * width + x0) * 4, (y * width + x1) * 4); + } + + return output; +} + +// source over destination, non-premultiplied: libwebp's integer version of the spec's formula +function blendOver(src: Uint8Array, s: number, dst: Uint8Array, d: number): void { + const sa = src[s + 3], da = dst[d + 3]; + const dstFactor = da * (256 - sa) >> 8; + const alpha = sa + dstFactor; + const scale = alpha ? Math.floor((1 << 24) / alpha) : 0; + for (let c = 0; c < 3; c++) { + const blended = src[s + c] * sa + dst[d + c] * dstFactor; + dst[d + c] = blended * scale >>> 24; + } + dst[d + 3] = alpha; +} + +// --------------------------------------------------------------------------- +// encoder (lossless) +// --------------------------------------------------------------------------- + +function riffChunk(type: string, body: Uint8Array): Uint8Array { + const out = new Uint8Array(8 + body.length + (body.length & 1)); + const view = new DataView(out.buffer); + view.setUint32(0, fourCC(type), true); + view.setUint32(4, body.length, true); + out.set(body, 8); + return out; +} + +function riff(chunks: Uint8Array[]): Uint8Array { + let size = 4; + for (const c of chunks) size += c.length; + const out = new Uint8Array(8 + size); + const view = new DataView(out.buffer); + view.setUint32(0, RIFF, true); + view.setUint32(4, size, true); + view.setUint32(8, WEBP, true); + let offset = 12; + for (const c of chunks) { + out.set(c, offset); + offset += c.length; + } + return out; +} + +function toARGB(rgba: Uint8Array, count: number): Uint32Array { + const argb = new Uint32Array(count); + for (let i = 0, o = 0; i < count; i++, o += 4) argb[i] = (rgba[o + 3] << 24 | rgba[o] << 16 | rgba[o + 1] << 8 | rgba[o + 2]) >>> 0; + return argb; +} + +const set24 = (d: Uint8Array, o: number, v: number): void => { d[o] = v & 255; d[o + 1] = v >> 8 & 255; d[o + 2] = v >> 16 & 255; }; + +/** + * Encodes RGBA pixels into a lossless WebP + */ +export function encode(rgba: Uint8Array, width: number, height: number): Uint8Array { + return riff([riffChunk('VP8L', vp8l.encode(toARGB(rgba, width * height), width, height))]); +} + +export interface WebPEncodeFrame { + x: number; + y: number; + width: number; + height: number; + /** milliseconds */ + duration: number; + /** whether the frame's area is cleared after it was shown */ + dispose: boolean; + /** RGBA, alpha blended onto the canvas */ + pixels: Uint8Array; +} + +/** + * Encodes frames into a lossless animated WebP. Frame offsets have to be even (the format stores them halved) + * @param plays How often to play the animation, 0 = forever + */ +export function encodeAnimation(frames: WebPEncodeFrame[], width: number, height: number, plays: number): Uint8Array { + if (!frames.length) throw new Error('WebP: an animation needs at least one frame'); + if (width > 1 << 24 || height > 1 << 24) throw new RangeError('WebP: canvas too large'); + + let alpha = false; + for (const f of frames) + for (let i = 3; i < f.pixels.length && !alpha; i += 4) if (f.pixels[i] !== 255) alpha = true; + + const vp8x = new Uint8Array(10); + vp8x[0] = 2 | (alpha ? 16 : 0); // animation, alpha + set24(vp8x, 4, width - 1); + set24(vp8x, 7, height - 1); + + const anim = new Uint8Array(6); // transparent background color + anim[4] = plays & 255; + anim[5] = plays >> 8 & 255; + + const chunks = [riffChunk('VP8X', vp8x), riffChunk('ANIM', anim)]; + for (const f of frames) { + if (f.x & 1 || f.y & 1) throw new RangeError('WebP: frame offsets have to be even'); + const bitstream = riffChunk('VP8L', vp8l.encode(toARGB(f.pixels, f.width * f.height), f.width, f.height)); + const header = new Uint8Array(16); + set24(header, 0, f.x >> 1); + set24(header, 3, f.y >> 1); + set24(header, 6, f.width - 1); + set24(header, 9, f.height - 1); + set24(header, 12, Math.min(0xffffff, Math.max(0, Math.round(f.duration)))); + header[15] = f.dispose ? 1 : 0; // blending (bit 1 unset) and disposal + const body = new Uint8Array(16 + bitstream.length); + body.set(header); + body.set(bitstream, 16); + chunks.push(riffChunk('ANMF', body)); + } + + return riff(chunks); +} diff --git a/src/image-type.ts b/src/image-type.ts index 05563290..85c3b465 100644 --- a/src/image-type.ts +++ b/src/image-type.ts @@ -1,7 +1,7 @@ import { BinaryData, view as bytes } from './util/mem.ts'; /** Image formats recognized by {@link ImageType.getType} */ -export type ImageTypeName = 'png' | 'jpeg' | 'tiff' | 'gif'; +export type ImageTypeName = 'png' | 'jpeg' | 'tiff' | 'gif' | 'webp'; const MAGIC_NUMBERS = { PNG: 0x89504e47, @@ -10,7 +10,9 @@ const MAGIC_NUMBERS = { TIFF_BE: 0x4d4d002a, BIGTIFF_LE: 0x49492b00, BIGTIFF_BE: 0x4d4d002b, - GIF: 0x474946 + GIF: 0x474946, + RIFF: 0x52494646, + WEBP: 0x57454250 }; // any binary data as a DataView (DataViews are used as is) @@ -34,6 +36,7 @@ export class ImageType { if (this.isJPEG(view)) return 'jpeg'; if (this.isTIFF(view)) return 'tiff'; if (this.isGIF(view)) return 'gif'; + if (this.isWebP(view)) return 'webp'; return null; } @@ -62,4 +65,10 @@ export class ImageType { const view = dataView(data); return view.byteLength >= 4 && (view.getUint32(0, false) >>> 8) === MAGIC_NUMBERS.GIF; } + + /** Whether the data starts like a WebP */ + static isWebP(data: BinaryData): boolean { + const view = dataView(data); + return view.byteLength >= 12 && view.getUint32(0, false) === MAGIC_NUMBERS.RIFF && view.getUint32(8, false) === MAGIC_NUMBERS.WEBP; + } } diff --git a/src/image.ts b/src/image.ts index 2340b7c6..ae0ce4f5 100644 --- a/src/image.ts +++ b/src/image.ts @@ -1,6 +1,7 @@ import * as png from './codecs/png.ts'; import * as jpeg from './codecs/jpeg.ts'; import * as tiff from './codecs/tiff.ts'; +import * as webp from './codecs/webp.ts'; import * as quantize from './codecs/quantize.ts'; import * as textRenderer from './render/text.ts'; import * as svgRenderer from './render/svg.ts'; @@ -37,7 +38,7 @@ export interface CompositeOptions { /** Options of {@link Image.decode} */ export interface DecodeOptions { - /** Whether to rotate/mirror JPEGs, TIFFs and PNGs as their orientation tag (Exif or XMP) says. Default: `true` */ + /** Whether to rotate/mirror JPEGs, TIFFs, PNGs and WebPs as their orientation tag (Exif or XMP) says. Default: `true` */ applyOrientation?: boolean; } @@ -1576,7 +1577,15 @@ export class Image { } /** - * Decodes an image (PNG, JPEG or TIFF) + * Encodes the image as lossless WebP (usually 20-30% smaller than PNG, but slower to encode) + */ + async encodeWebP(): Promise { + return webp.encode(this.__bytes__(), this.width, this.height); + } + + /** + * Decodes an image (PNG, JPEG, TIFF or WebP). For APNGs this is the default image; animated WebPs and GIFs have + * no single image, decode those with {@link Animation.decode} * @param input The binary data to decode * @param options See {@link DecodeOptions}. JPEGs, TIFFs and PNGs are rotated/mirrored as their orientation tag * (Exif, for PNGs also XMP) says by default: photos taken with a rotated camera are stored sideways @@ -1591,7 +1600,10 @@ export class Image { if (ImageType.isPNG(dataView)) decoded = await png.decode(data); else if (ImageType.isJPEG(dataView)) decoded = jpeg.decode(data); else if (ImageType.isTIFF(dataView)) decoded = await tiff.decode(data); - else throw new Error('Unsupported image type'); + else if (ImageType.isWebP(dataView)) { + if (webp.isAnimated(data)) throw new Error('Animated WebPs have to be decoded with Animation.decode()'); + decoded = webp.decode(data); + } else throw new Error('Unsupported image type'); const image = new Image(decoded.width, decoded.height); image.bitmap.set(decoded.pixels); diff --git a/src/index.ts b/src/index.ts index dbfba9c2..37ca354d 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,6 +1,9 @@ import { BinaryData } from './util/mem.ts'; import { Image } from './image.ts'; import { GIF } from './gif.ts'; +import { Animation } from './animation.ts'; +import * as webp from './codecs/webp.ts'; +import { view } from './util/mem.ts'; import { ImageType } from './image-type.ts'; export { Image } from './image.ts'; @@ -16,13 +19,15 @@ export type { ImageTypeName } from './image-type.ts'; export type { BinaryData } from './util/mem.ts'; /** - * Decodes any supported image: GIFs are returned as {@link GIF} (an {@link Animation}), everything else (including - * APNGs, as their default image) as {@link Image}. {@link Animation.decode} decodes the frames of GIFs and APNGs + * Decodes any supported image: GIFs are returned as {@link GIF} (an {@link Animation}), animated WebPs as + * {@link Animation}, everything else (including APNGs, as their default image) as {@link Image}. + * {@link Animation.decode} decodes the frames of GIFs, APNGs and WebPs * @param data The image data * @param onlyExtractFirstFrame Whether to end animation decoding after the first frame */ -export function decode(data: BinaryData, onlyExtractFirstFrame = false): Promise { +export function decode(data: BinaryData, onlyExtractFirstFrame = false): Promise { const type = ImageType.getType(data); if (type === 'gif') return GIF.decode(data, onlyExtractFirstFrame); + if (type === 'webp' && webp.isAnimated(view(data))) return Animation.decode(data, onlyExtractFirstFrame); return Image.decode(data); } diff --git a/tests/animation.js b/tests/animation.js index b69664b6..c1ca4f5c 100644 --- a/tests/animation.js +++ b/tests/animation.js @@ -31,7 +31,7 @@ const square = (color, w, h) => new Image(w, h).fill((x, y) => (x + y) % 5 === 0 ], 2); const reference = expected(animation); - for (const format of ['APNG']) { + for (const format of ['APNG', 'WebP']) { const bytes = await animation[`encode${format}`](); const back = await Animation.decode(bytes); if (back.length !== 4 || back.loopCount !== 2) panic(`${format}: wrong frame or loop count`); @@ -58,7 +58,7 @@ const square = (color, w, h) => new Image(w, h).fill((x, y) => (x + y) % 5 === 0 // loop counts: -1 forever, 0 once, n repeats for (const loopCount of [-1, 0, 5]) { const two = new Animation([Frame.from(still), Frame.from(still)], loopCount); - for (const format of ['APNG', 'GIF']) + for (const format of ['APNG', 'WebP', 'GIF']) if ((await Animation.decode(await two[`encode${format}`]())).loopCount !== loopCount) panic(`${format}: loop count ${loopCount} not kept`); } })(); diff --git a/tests/webp.js b/tests/webp.js new file mode 100644 index 00000000..1ea682f1 --- /dev/null +++ b/tests/webp.js @@ -0,0 +1,65 @@ +const fs = require('fs').promises; +const crypto = require('crypto'); +const {Image, Animation, ImageType, decode} = require('..'); + +const panic = message => { + console.error(message); + process.exit(1); +}; + +const sha256 = data => crypto.createHash('sha256').update(data).digest('hex'); + +// sha256 of the RGBA output of libwebp (through Pillow) for files written by libwebp +const expected = { + lossy: 'c015437d073e901626218276568b0e275d6e486b1024ad70bcd76ddeebe82b95', // VP8 + 'lossy-alpha': 'f20e9fec03a515321cec9901ccd7ba9bbc4a10b9371119da7e580150fda47f93', // VP8X + ALPH + VP8 + lossless: 'a3d6d80178932c33e744693b9ec6149f8bbbf0f84f2efc6a31cccc30bfe5f8f6' // VP8L +}; + +const animated = [ + '4e8bee3d04fd577a2d8e28be4b3db1c79c12be8b53050e45310851b35d4ba21c', + 'ad57ef20633236b1284b090451cae3f5e82d45bba656ba966950f0803cc052a9', + '0c985b39077eb7ff7715e97f8362e207b0bb700a3d8f3e19d0acb26ff89eae34', + 'b853de342e12ba2cb5403a45caf8d306788597e00702b83ee13b79c364670e00' +]; + +(async () => { + for (const [name, hash] of Object.entries(expected)) { + const data = await fs.readFile(`./tests/webp/${name}.webp`); + if (ImageType.getType(data) !== 'webp') panic(`${name}: not detected as WebP`); + const image = await Image.decode(data); + if (image.width !== 96 || image.height !== 72) panic(`${name}: wrong dimensions`); + if (sha256(image.bitmap) !== hash) panic(`${name}: decoded pixels don't match libwebp`); + } + + // animated WebPs: Image.decode refuses them, Animation.decode composites the frames + const data = await fs.readFile('./tests/webp/animated.webp'); + try { + await Image.decode(data); + panic('Image.decode should reject animated WebPs'); + } catch (err) { + if (!/Animation\.decode/.test(err.message)) panic(`unexpected error ${err.message}`); + } + const animation = await Animation.decode(data); + if (!(await decode(data) instanceof Animation)) panic('decode() should return an Animation for animated WebPs'); + if (animation.length !== 4 || animation.loopCount !== -1) panic('animated WebP: wrong frame or loop count'); + if (animation.map(f => f.duration).join() !== '80,120,80,200') panic('animated WebP: wrong durations'); + animation.forEach((frame, i) => { + if (sha256(frame.bitmap) !== animated[i]) panic(`animated WebP: frame ${i} doesn't match libwebp`); + }); + + // lossless encoding: photo-like content (predictor), few colors (palette), alpha, odd sizes + const photo = await Image.decode(await fs.readFile('./tests/webp/lossy.webp')); + const cases = [ + photo, + new Image(37, 23).fill((x, y) => (x ^ y) & 4 ? 0xff0000ff : 0x00000000), + new Image(64, 9).fill((x, y) => Image.rgbaToColor(x * 4, y * 28, 7, (x * 9) & 255)), + new Image(1, 1).fill(0x12345678) + ]; + for (const image of cases) { + const encoded = await image.encodeWebP(); + const back = await Image.decode(encoded); + if (back.width !== image.width || back.height !== image.height || !Buffer.from(back.bitmap).equals(Buffer.from(image.bitmap))) + panic(`encodeWebP: ${image.width}x${image.height} didn't round trip`); + } +})(); diff --git a/tests/webp/animated.webp b/tests/webp/animated.webp new file mode 100644 index 00000000..7b22f78c Binary files /dev/null and b/tests/webp/animated.webp differ diff --git a/tests/webp/lossless.webp b/tests/webp/lossless.webp new file mode 100644 index 00000000..13d51bd2 Binary files /dev/null and b/tests/webp/lossless.webp differ diff --git a/tests/webp/lossy-alpha.webp b/tests/webp/lossy-alpha.webp new file mode 100644 index 00000000..033f1f2e Binary files /dev/null and b/tests/webp/lossy-alpha.webp differ diff --git a/tests/webp/lossy.webp b/tests/webp/lossy.webp new file mode 100644 index 00000000..feab6c72 Binary files /dev/null and b/tests/webp/lossy.webp differ