From 5bc32c57d64b79992f11bbef84adfd581b0d994f Mon Sep 17 00:00:00 2001 From: Marco Russo Date: Wed, 30 Sep 2026 16:23:10 +0200 Subject: [PATCH] Import ink tables and legacy stickers from Microsoft Whiteboard An ink table from the Windows 10 app arrives as an Unknown object holding the table's grid. Each cell becomes a square-cornered rectangle in the border's color, and each cell's ink is read through its group's place inside the cell. A legacy sticker becomes its picture and a label for its caption, centered across the caption's box. Both are read from the web client's TableComponent and caption aspect, because no sample holds either. They join the 1.7.3 notes, which is not released yet. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 7 +- README.md | 2 +- docs/microsoft-whiteboard-import.md | 12 +- site/export-microsoft-whiteboard.html | 2 +- site/guide.html | 2 +- .../Import/MicrosoftWhiteboardExport.cs | 188 ++++++++++++++++++ .../MicrosoftWhiteboardSmokeTests.cs | 60 ++++++ 7 files changed, 263 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 57f88bc..28343ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,10 +23,11 @@ and **bold**. Vertical and diagonal highlighter strokes now come out about as wide as they were, instead of up to four times wider. Sticky notes and note grids have square corners. -### Lists and GIFs from Microsoft Whiteboard +### Lists, tables, stickers, and GIFs from Microsoft Whiteboard Text, bulleted, and task lists now come across as text containers, with a box for each -task that shows whether it is done, and GIFs come across as pictures. They used to be -listed as left out. +task that shows whether it is done. Tables drawn with ink come across as rectangles with +their ink, stickers from the Windows 10 app as pictures with their captions, and GIFs as +pictures. They used to be listed as left out. ## 1.7.2 - 30 September 2026 diff --git a/README.md b/README.md index fa91791..1aed492 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ How the project is developed and shipped is documented separately: - Whole-stroke erasing - PNG, JPEG, BMP, GIF, and SVG import, clipboard bitmap paste, and Explorer drag-and-drop of images, text files, `.wimport` recipes, PowerPoint decks, and Microsoft Whiteboard exports - PowerPoint decks imported as one picture per slide through PowerPoint from Microsoft 365: SVG, or PNG for a slide whose fonts are not installed, placed in one row per section, with an optional frame around each slide. **File → Open** creates a new board from the deck; **File → Import** and drag-and-drop add it below the board's content -- Boards exported from Microsoft Whiteboard as a `.zip` imported with their ink, pictures, shapes, sticky notes, text, lists, connectors, links, and comments, and the pen's varying width kept. **File → Open** creates a new board from the export; **File → Import** and drag-and-drop add it below the board's content. [docs/microsoft-whiteboard-import.md](docs/microsoft-whiteboard-import.md) lists what is not carried over +- Boards exported from Microsoft Whiteboard as a `.zip` imported with their ink, pictures, shapes, sticky notes, text, lists, tables, connectors, links, and comments, and the pen's varying width kept. **File → Open** creates a new board from the export; **File → Import** and drag-and-drop add it below the board's content. [docs/microsoft-whiteboard-import.md](docs/microsoft-whiteboard-import.md) lists what is not carried over - SVG is stored as its markup and redrawn as vectors at every zoom and resize. Pasting SVG markup that was copied as text — the output of a DAX SVG measure, for instance — creates a picture, and copying an SVG container puts both the markup and a bitmap on the clipboard. Text set in a Microsoft 365 font such as Aptos is drawn in that font when Office has it on the machine - Image selection, movement, resizing, and deletion - Text containers created by pasting plain text, with display and in-place edit modes diff --git a/docs/microsoft-whiteboard-import.md b/docs/microsoft-whiteboard-import.md index 4ccea17..d366a10 100644 --- a/docs/microsoft-whiteboard-import.md +++ b/docs/microsoft-whiteboard-import.md @@ -55,6 +55,7 @@ no model of the board beyond what it draws. | `FluidImage` | `div.imageComponent` with the size, and an `img` with a base64 data URI labelled `text/plain` whatever the picture is. | | `AzureImage` | A picture on a board made with an older version of the app, written as a `FluidImage` is, with the data URI labelled `image/*`. | | `AzureGif` | An animated GIF, written as an `AzureImage` is. Not seen in a sample; the web client converts it with the same function as `AzureImage`. | +| `LegacySticker` | A sticker from the Windows 10 app: a picture written as an `AzureImage` is, and a caption in a `div` moved from the picture's corner by its `transform`, with `font-size` 30 and the family and weight in its style. The text box inside has the caption's width, and a smaller `font-size` when the text shrank to fit. Not seen in a sample; read from the web client's `ImageUI` caption aspect. | | `DocumentPage` | A page of an inserted PDF, written as a `FluidImage` is. | | `ReactionStickers` | A sticker, written as a 64 × 64 `FluidImage` with an SVG data URI. | | `Shape` | An SVG whose `g` has `fill`, `stroke`, and `stroke-width` (in `pt`), and whose `path` draws the outline around the middle of the box. The shape's text sits in `div.textBoxContainer`. | @@ -65,6 +66,7 @@ no model of the board beyond what it draws. | `VerticalList`, `VerticalBulletList`, `VerticalCheckboxList`, `UnknownList` | A list of text, bullets, or tasks, in `div.legacyListContainer`: a title editor in `div.verticalListTitle`, a `div.listColumnHeading` with the column names, then one `div.listChild` per item with its icon, its text editor, who a task is assigned to (`div.assignedUserDisplayName`), and its likes. A done task's icon is `CheckmarkCircle` with the class `checkedListItemIcon`, an open one `StatusCircle` with `uncheckedListItemIcon`. The list is 44 + 284 + 76 pixels wide, or 44 + 364 + 76 when tasks have an Assigned to column. An `UnknownList` draws no items. Not seen in a sample; read from the web client's `ListContainerComponent`. | | `PlainText` | A text box: `div.textbox.plainText` with `max-width` and `font-size`, inside a wrapper whose `justify-content` centers the text in a wider box when the text is centered. | | `Connector` | An SVG with the route as a path from the anchor, and the head as a small path moved to one end with `translate`. | +| `Unknown` holding `div.inkTableContainer` | A table drawn with ink in the Windows 10 app, which the web client registers under no type of its own. The container is a CSS grid whose `grid-template-columns` and `grid-template-rows` give each column and row as `minmax(px, auto)`. One `div.inkCellContainer` per cell, row by row, has a `4px solid` border in the table's color and a `-2px` margin, so the border sits over the grid line. Inside it, a `div` shifts the cell's ink by `left` and `top`, and each ink group sits in a `div` with its own `left`, `top`, and `transform`, holding the same SVG an `InkGroup` has. The web client draws a placeholder instead when its `EnableTables` flag is off. Not seen in a sample; read from the web client's `TableComponent`. | | `Hyperlink` | A preview card with the page's picture, an `a` with the link and its title, and the description. | | `CommentThread` | A pin, described above. | @@ -125,6 +127,8 @@ Windows client offers, from `paleYellowGradient` (#FEE15A) to `grayGradient` (#C | Note grid | A white rectangle for the panel, a label for the title, and one note per cell | | Text box | `FreeTextBoardObject`, wrapped into the lines the box showed | | Connector | `ConnectorBoardObject`, straight, with its ends bound to shapes they sit on | +| Ink table | One `ShapeBoardObject` rectangle per cell with the border's color and 4-unit outline, then the cells' ink as strokes | +| Legacy sticker | The picture, and its caption as a `FreeTextBoardObject` centered across the caption's box | | Link card | Markdown `TextBoardObject` titled *Link* | | List | Markdown `TextBoardObject` titled *List*: the list's title in bold, then a Markdown list of bullets, or one line per item with ☒ for a done task and ☐ for an open one | | Comment thread | Markdown `TextBoardObject` titled *Comment*, in a column to the right of the board | @@ -228,14 +232,14 @@ parses in about 160 ms. - PowerPoint pages, which the Windows client cannot insert, videos, and Loop components - SVG pictures, which the Windows client refuses to insert - Groups, which the Windows client does not offer -- GIFs and lists, which are read from the web client's code, not from a sample +- GIFs, lists, ink tables, and legacy stickers, which are read from the web client's code, + not from a sample The web client's code names every type an export can hold. Its `data-whiteboard-type` values, as of version 26.10910.101 in September 2026, are the types above and these, -which the importer reports and leaves out: `Table` (an ink-to-table grid from the -Windows 10 app), `LegacyTemplate`, `LegacySticker`, `LoopObject`, +which the importer reports and leaves out: `LegacyTemplate`, `LoopObject`, `HostedFluidObject`, `GroupFluidObject`, `AppIframeHost`, `WorkItem`, `Frame`, and -`CustomElement`. The export replaces a Loop component and an app frame with a +`CustomElement`, and an `Unknown` that holds no table. The export replaces a Loop component and an app frame with a placeholder picture. Each is reported by type after the import. Sticky notes cannot be diff --git a/site/export-microsoft-whiteboard.html b/site/export-microsoft-whiteboard.html index c883fef..700cbe5 100644 --- a/site/export-microsoft-whiteboard.html +++ b/site/export-microsoft-whiteboard.html @@ -223,7 +223,7 @@

What comes across

  • Pictures, GIFs, stickers, and the pages of inserted PDFs.
  • Shapes with their fill, outline, angle, and text.
  • Sticky notes and note grids, as shapes in the same colors with their text.
    • -
  • Text boxes, lists, connectors, and links.
    • +
  • Text boxes, lists, tables drawn with ink, connectors, and links.
  • Comments, as text beside the board with the names of their authors.

    • A highlighter stroke has one width, and Rainbow and Galaxy ink become one color. Elbow connectors become straight. The background color and reactions are left out. After the import, a message lists anything the board did not bring across.

    diff --git a/site/guide.html b/site/guide.html index 1a92385..5ddf9a3 100644 --- a/site/guide.html +++ b/site/guide.html @@ -646,7 +646,7 @@

    Bring a PowerPoint deck in

    Bring a Microsoft Whiteboard board in

    -

    A board exported from Microsoft Whiteboard as a ZIP comes across with its ink at the same pen widths, pictures, shapes, sticky notes, text, lists, connectors, links, and comments. It can start a new board or join an existing one, below its content, and anything left out is listed after the import. Export from Microsoft Whiteboard shows how to make the ZIP.

    +

    A board exported from Microsoft Whiteboard as a ZIP comes across with its ink at the same pen widths, pictures, shapes, sticky notes, text, lists, tables, connectors, links, and comments. It can start a new board or join an existing one, below its content, and anything left out is listed after the import. Export from Microsoft Whiteboard shows how to make the ZIP.

    diff --git a/src/SQLBI.Whiteboard.Core/Import/MicrosoftWhiteboardExport.cs b/src/SQLBI.Whiteboard.Core/Import/MicrosoftWhiteboardExport.cs index 239223c..0b99af9 100644 --- a/src/SQLBI.Whiteboard.Core/Import/MicrosoftWhiteboardExport.cs +++ b/src/SQLBI.Whiteboard.Core/Import/MicrosoftWhiteboardExport.cs @@ -29,6 +29,16 @@ public sealed partial class MicrosoftWhiteboardExport private const string CommentTitle = "Comment"; private const string ListTitle = "List"; + /// + /// A table cell's border, which the web client draws over the grid line. + /// + private const double TableBorder = 4; + + /// + /// The largest size a sticker's caption is drawn at before it shrinks to fit. + /// + private const double StickerCaptionFontSize = 30; + /// /// A vertical list's columns in the web client: the icon, the text, the text /// beside an Assigned to column, that column, and the likes. @@ -231,6 +241,21 @@ void Skip(string what, int count = 1) items.Add(link); } + break; + case "LegacySticker": + items.AddRange(ReadSticker(anchor, tags)); + break; + case "Unknown": + case "Table": + if (tags.Any(tag => tag.Name == "div" && tag.HasClass("inkTableContainer"))) + { + items.AddRange(ReadTable(anchor, tags)); + } + else + { + Skip(anchor.Type); + } + break; case "CommentThread": if (ReadComment(anchor, tags, threads) is { } comment) @@ -766,6 +791,155 @@ private static IEnumerable ReadGrid(Anchor anchor, List string.Join("\n\n", lines.Where(line => line.Length > 0))); } + /// + /// A table drawn with ink in the Windows 10 app. The web client names its type + /// Unknown and draws it as a CSS grid: each column and row is as long as its + /// minmax() says, each cell has a 4-pixel border in the table's color that + /// sits over the grid line, and each cell's ink is a collection of ink groups + /// placed from the cell's corner, moved by the offset that keeps ink reaching left + /// of or above the cell inside it. The cells become rectangles, then the ink comes + /// on top of them. + /// + private static IEnumerable ReadTable(Anchor anchor, List tags) + { + var gridIndex = tags.FindIndex(tag => tag.Name == "div" && tag.HasClass("inkTableContainer")); + var gridStyle = tags[gridIndex].Attribute("style"); + var columns = Numbers(StyleValue(gridStyle, "grid-template-columns")); + var rows = Numbers(StyleValue(gridStyle, "grid-template-rows")); + var cells = new List(); + for (var index = gridIndex + 1; index < tags.Count; index++) + { + if (tags[index].Name == "div" && tags[index].HasClass("inkCellContainer")) + { + cells.Add(index); + } + } + + if (columns.Count == 0 || rows.Count == 0) + { + yield break; + } + + var border = cells.Count > 0 ? StyleValue(tags[cells[0]].Attribute("style"), "border") : null; + var solid = border?.IndexOf("solid", StringComparison.OrdinalIgnoreCase) ?? -1; + var color = (solid >= 0 ? PaintOrNull(border![(solid + 5)..]) : null) ?? Black; + var lefts = Starts(columns); + var tops = Starts(rows); + for (var row = 0; row < rows.Count; row++) + { + for (var column = 0; column < columns.Count; column++) + { + yield return new MicrosoftWhiteboardShape( + anchor.ToCanvas(new PointD(lefts[column] + (columns[column] / 2), tops[row] + (rows[row] / 2))), + columns[column] * anchor.Scale, + rows[row] * anchor.Scale, + anchor.AngleDegrees, + ShapeKind.Rectangle, + color, + null, + TableBorder * anchor.Scale, + string.Empty, + new MicrosoftWhiteboardFont(LabelStyles.DefaultFontFamily, 20, Black, false, false, false)); + } + } + + for (var cell = 0; cell < cells.Count && cell / columns.Count < rows.Count; cell++) + { + var start = cells[cell]; + var end = cell + 1 < cells.Count ? cells[cell + 1] : tags.Count; + var shiftStyle = start + 1 < end ? tags[start + 1].Attribute("style") : null; + var corner = new PointD( + lefts[cell % columns.Count] + (TableBorder / 2) + (StyleLength(shiftStyle, "left") ?? 0), + tops[cell / columns.Count] + (TableBorder / 2) + (StyleLength(shiftStyle, "top") ?? 0)); + var groups = Enumerable.Range(start, end - start) + .Where(index => tags[index].Name == "svg" && tags[index].HasClass("inkGroup")) + .ToArray(); + for (var group = 0; group < groups.Length; group++) + { + // The collection child that places the group is the nearest div before + // it with a position of its own, after the div that shifts the cell. + var place = Enumerable.Range(start + 2, Math.Max(0, groups[group] - start - 2)) + .Reverse() + .Select(index => tags[index]) + .FirstOrDefault(tag => tag.Name == "div" && StyleValue(tag.Attribute("style"), "left") is not null); + var child = place is null + ? new Anchor("InkGroup", default, 1, 0, 0, 1, 0, 0) + : Anchor.From(place, "InkGroup"); + var groupEnd = group + 1 < groups.Length ? groups[group + 1] : end; + foreach (var item in ReadInkGroup(child.Inside(anchor, corner), tags.GetRange(groups[group], groupEnd - groups[group]))) + { + yield return item; + } + } + } + } + + private static double[] Starts(List lengths) + { + var starts = new double[lengths.Count]; + for (var index = 1; index < lengths.Count; index++) + { + starts[index] = starts[index - 1] + lengths[index - 1]; + } + + return starts; + } + + /// + /// A sticker from the Windows 10 app is a picture with a caption over it. The web + /// client draws the picture as any other and the caption as a text editor in a box + /// moved from the picture's corner, at 30 pixels or smaller when it shrinks the text + /// to fit. The caption becomes a label centered across that box. + /// + private static IEnumerable ReadSticker(Anchor anchor, List tags) + { + if (ReadImage(anchor, tags) is { } picture) + { + yield return picture; + } + + var captionIndex = tags.FindIndex(tag => tag.Name == "div" && + StyleValue(tag.Attribute("style"), "font-size") is not null && + string.Equals(StyleValue(tag.Attribute("style"), "position"), "absolute", StringComparison.OrdinalIgnoreCase)); + var box = tags.FirstOrDefault(tag => tag.Name == "div" && tag.HasClass("imageComponent")); + if (captionIndex < 0 || box is null) + { + yield break; + } + + var text = ReadText(tags, captionIndex, tags.Count); + if (text.Length == 0) + { + yield break; + } + + var boxWidth = StyleLength(box.Attribute("style"), "width") ?? 0; + var boxHeight = StyleLength(box.Attribute("style"), "height") ?? 0; + var caption = Anchor.From(tags[captionIndex], "Caption"); + var captionStyle = tags[captionIndex].Attribute("style"); + var width = tags.Skip(captionIndex + 1) + .Select(tag => StyleLength(tag.Attribute("style"), "width")) + .FirstOrDefault(value => value > 0) ?? boxWidth; + var fontSize = tags.Skip(captionIndex) + .Select(tag => StyleLength(tag.Attribute("style"), "font-size")) + .OfType() + .Where(value => value > 0) + .DefaultIfEmpty(StickerCaptionFontSize) + .Min(); + var core = tags.Skip(captionIndex).FirstOrDefault(tag => tag.HasClass("textBoxCore"))?.Attribute("style"); + yield return new MicrosoftWhiteboardLabel( + anchor.ToCanvas(new PointD( + (-boxWidth / 2) + caption.Origin.X + caption.E, + (-boxHeight / 2) + caption.Origin.Y + caption.F)), + anchor.Scale, + anchor.AngleDegrees, + default, + text, + ReadFont($"{core};{captionStyle}", fontSize, defaultBold: false), + Math.Max(1, width), + width); + } + /// /// A connector's SVG draws its route from the anchor, and its head as a small /// path moved to one end. A route with corners is kept as a straight line. @@ -1243,6 +1417,20 @@ public static Anchor From(HtmlTag tag, string type) Origin.X + (A * local.X) + (C * local.Y) + E, Origin.Y + (B * local.X) + (D * local.Y) + F); + /// + /// This anchor placed at a point inside another, so that a point here reaches + /// the canvas through both transforms. + /// + public Anchor Inside(Anchor parent, PointD at) => new( + Type, + parent.ToCanvas(at + Origin + new PointD(E, F)), + (parent.A * A) + (parent.C * B), + (parent.B * A) + (parent.D * B), + (parent.A * C) + (parent.C * D), + (parent.B * C) + (parent.D * D), + 0, + 0); + private static double[] StyleMatrix(string? style) { if (StyleValue(style, "transform") is { } value && value.Contains("matrix(", StringComparison.Ordinal)) diff --git a/tests/SQLBI.Whiteboard.Core.SmokeTests/MicrosoftWhiteboardSmokeTests.cs b/tests/SQLBI.Whiteboard.Core.SmokeTests/MicrosoftWhiteboardSmokeTests.cs index 5bf6c8f..095a7c3 100644 --- a/tests/SQLBI.Whiteboard.Core.SmokeTests/MicrosoftWhiteboardSmokeTests.cs +++ b/tests/SQLBI.Whiteboard.Core.SmokeTests/MicrosoftWhiteboardSmokeTests.cs @@ -139,6 +139,7 @@ public static void Run() RunObjects(); RunOlderObjects(); RunLists(); + RunTables(); } /// @@ -454,6 +455,65 @@ private static void RunLists() "A GIF is a picture like any other."); } + /// + /// An ink table and a sticker from the Windows 10 app, as the web client draws them: + /// the table as a grid of bordered cells holding collections of ink groups, and the + /// sticker as a picture with a caption moved from its corner. + /// + private static readonly string TablesPage = $$""" +
    +
    +
    +
    +
    + + + +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    Great idea!
    +
    +
    +
    +
    + """; + + private static void RunTables() + { + var board = MicrosoftWhiteboardExport.Parse(TablesPage, "Tables"); + Assert(board.Skipped.Count == 1 && board.Skipped["Unknown"] == 1, + "An Unknown object without a table in it is still reported."); + + var cells = board.Items.OfType().ToArray(); + Assert(cells.Length == 4 && cells.All(cell => cell is { Kind: ShapeKind.Rectangle, OutlineArgb: 0xFF0069BF, FillArgb: null, Thickness: 4 }), + "Each cell of a table is a square-cornered rectangle outlined in the table's color."); + Assert(cells[0] is { Width: 200, Height: 50 } && cells[0].Center == new PointD(200, 225) && + cells[3] is { Width: 100, Height: 60 } && cells[3].Center == new PointD(350, 280), + "Cells take their columns' widths and their rows' heights, row by row."); + + var stroke = board.Strokes.Single(); + AssertNear(117, stroke.Points[0].X, "A cell's ink starts at the cell's corner inside the border, moved by the cell's shift and its group's place."); + AssertNear(208, stroke.Points[0].Y, "A cell's ink starts at the cell's corner inside the border, moved by the cell's shift and its group's place."); + AssertNear(137, stroke.Points[^1].X, "A cell's ink keeps its length."); + var order = board.Items.ToList(); + Assert(order.IndexOf(stroke) > order.IndexOf(cells[^1]), + "The ink is drawn over the table's cells."); + + Assert(board.Items.OfType().Single().Bounds == new RectD(300, 400, 400, 200), + "A sticker's picture is read as any other picture."); + Assert(board.Items.OfType().Single() is + { Text: "Great idea!", Scale: 2, WrapWidth: 160, CenteredWidth: 160, Font: { Family: "Segoe Print", Size: 24, Argb: 0xFFC10051 } } caption && + caption.Origin == new PointD(340, 460), + "A sticker's caption is a label centered across its box, at the size it shrank to, moved from the picture's corner."); + } + private static void RunArchive() { var folder = Path.Combine(Path.GetTempPath(), "wb-mswb-" + Guid.NewGuid().ToString("N"));