Anchors and highlighting

Coordinates cannot be that address. A Word document is a flow of paragraphs and has no pages until it is laid out, and where they fall depends on the fonts the machine happens to have. So an address names a node of the model, which is what survives a relayout, a different zoom, and a different machine.

apertura:1:9f3ac21b:body/t2/w4/c1/b0/r0+12
└ scheme ┘ │ hash  └ flow ┘└──── steps ────┘└ offset
           version

The eight hex digits are a hash of the file’s bytes, and they are why this can be trusted: an address arriving from a different file is detected rather than quietly resolved to whatever node happens to sit at that path. Silent wrongness is how naive offset schemes fail, and they fail in production rather than in tests.

Three anchors, not one

A locator stops being valid the moment somebody opens the document and presses save, and nobody re-embeds a corpus because a typo was fixed on page four. So an anchor is a list of selectors in decreasing precision, resolved in order until one succeeds. The shape follows the W3C Web Annotation model.

• Exact — the locator. Valid while the bytes are unchanged.
• Portable — character offsets into a named serialisation, for chunks cut by somebody else’s splitter. The profile records which output and which options produced them, so a mismatch is an error rather than a highlight forty characters off.
• Robust — the quoted text with thirty-two characters of context on each side. Survives an edit that moved the passage to another page of another section.

The resolver reports which one matched, so an application can watch its corpus drift before a user reports a highlight in the wrong place.

The loop

import { extract } from '@apertura/extract';

const doc = await extract(bytes);

for (const chunk of doc.chunks({ maxTokens: 512 })) {
  await index.add({ text: chunk.text, meta: chunk.selectors });
}

How the highlight is drawn

Through the CSS Custom Highlight API, with a layer of positioned rectangles as the fallback and for anything a text highlight cannot express. Nothing is inserted into the document: this viewer paginates by measuring what it rendered, so a <mark> would move the text it was marking and repaginate the document on every hit.

A highlight is registered state rather than an operation. Under virtualisation the passage it names is usually not on the page yet, so the registry keeps the intention and paints when the page arrives — which is why the four-hundredth hit is already there when the reader scrolls to it. And a highlight is a list of fragments, never one rectangle: a paragraph split across a page boundary is two groups of rectangles on two pages.