Design notes

Why the non-obvious choices were made, what broke before, and the measurements behind them. Newest thinking is in the relevant docs; this is the record.

The tracers are classical, not a model. The research in research found that no downloadable AI model beats a good classical tracer at the tracing itself: VTracer matches or beats every learned model on icons and clearly beats them on illustrations, at a fraction of a second instead of 10–80 seconds on a GPU. AI belongs around the tracer (background removal, upscaling), which is the next phase.

VTracer was vendored with a rewritten loader, until #53. The official @visioncortex/vtracer package is a wasm-pack --target nodejs build: the wasm is portable, but the glue reads it with fs.readFileSync, so it cannot load in a browser. The community browser build wraps the older 0.8 engine and misses 1.0’s palette and simplification options, so a script rewrote the glue to ESM with an async init(url), asserting every substitution matched. Once Inlay took Poster too, the app stopped shipping VTracer; it’s now a benchmark only, loaded in Node as published (scripts/vtracer.mjs).

Flat art traces each edge once. VTracer traces every colour region on its own and smooths the pixel outline into curves without knowing where the corners are. On the sample badge that turned the straight mountain slopes into slow S-bends and the snow-cap peaks into rounded bumps with two nodes a pixel apart, and none of its settings gave straight lines and sharp corners. Fitting each traced shape separately fixed the lines but opened hairline gaps, because cut-out shapes trace every shared edge twice and the two copies were fitted a fraction of a pixel apart. So Logo, Illustration and Line art now use a tracer of our own, Inlay (packages/inlay/src/inlay.ts), named for inlaid wood, where every piece is cut to fit its neighbours:

Label every pixel with a palette colour (the histogram peaks below). A pixel between two colours (anti-aliasing) takes the nearest colour among its neighbours, so a red/blue edge never grows a purple sliver.
Merge specks into their surroundings.
Walk the cracks between differently labelled pixels. Where three regions meet is a junction; each boundary between junctions is one chain, with one region either side.
Fit each chain once (packages/inlay/src/fit.ts): find corners (two straight sides, fitted a few pixels out, that meet near the outline; a round end’s sides run parallel, so it stays a curve), then a straight line wherever one fits within 1.5× tolerance, otherwise straight stretches as lines and the rest as least-squares Béziers (Schneider) with matching tangents. The line test sees a plain moving average, which removes JPEG’s 16 px colour-block ripple; curves see an average whose window shrinks with local curvature, so small dots keep their size.
Build each region from its chains. Neighbours share the same fitted edge exactly: no gaps, no overlaps, which is also what a vinyl cutter wants.

Scored with npm run score (rendered back at 4× against the clean art, where wobble and rounded corners count):

Sample badge	Wrong pixels at 4×	Nodes	Size
VTracer, previous Logo settings	0.86%	259	7.3 KB
Shared edges, fitted per chain	0.74%	167	5.0 KB
+ corners by their sides, round ends, lines inside curves	0.70%	177	5.4 KB

(Re-measured later for the landing page, VTracer comes out at 0.85% and 271 nodes; see measuring quality.)

Straight edges are single segments and the peaks are single sharp corners.

The third row came from Matt’s testing on a different browser, whose JPEG encoder left different noise: a bump at a snow cap’s tip and hooked ends on the sample’s arc. Sweeping JPEG quality reproduced both, and the fixes were about telling shapes apart rather than tuning numbers. A corner is now two straight sides meeting near the outline (a stroke’s 4 px round end has sides that run parallel, so it stays a curve); smoothing follows local curvature, measured at two scales so JPEG noise can’t fake a tight bend; runs that mix straight and curved parts get lines for the straight stretches and curves joined smoothly between them; and a curve turns at most about 90°, so a circle is four curves. The extra nodes are those round ends and circles. On a 2048 px image Inlay took 1.3 s where VTracer took 17.5 s.

Pixel art moved to Inlay too. Pixel art wants its exact staircase, which Inlay already walks before it fits anything, so a pixel mode skips the clean-up and the fitting and joins straight runs. Its palette is the image’s exact colours, because histogram peaks drop a one-pixel eye for being rare and fold close shades together. On a test sprite both engines draw every pixel exactly; Inlay uses 14% fewer nodes, about the same bytes, and on a 1024 px sprite is 4× faster (0.2 s against 0.85 s). That left Poster on VTracer, since photos have no flat regions or histogram peaks and cut-out shapes cost size there, until #53 (below).

Tuning the presets. Each candidate setting was scored by tracing, rendering the SVG back in Chromium, and measuring the error against the clean source art, not the JPEG. On the built-in sample badge (5 colours, JPEG at q72):

Change	Wrong pixels	Paths	Size
Colour cap 6 (first attempt)	8.75%	346	68 KB
Cap 8	1.39%	373	70 KB
+ `layerDifference: 32`, speckle 8	1.50%	45	13 KB
+ cut-out shapes	0.95%	15	7 KB
+ palette from histogram peaks	0.93% (mean error 3.28 → 1.47)	14	4.3 KB

Two of those were surprises worth remembering:

A cap equal to the number of colours is too low. Anti-aliased edge shades take quantizer slots, so with a cap of 6 on a 5-colour badge the red mountain and green hill merged into one brown. The colour hint in the UI says to raise the cap if that happens.
VTracer’s quantizer drifts flat colours towards their neighbours. The gold sun came out pale peach, plainly visible in the Difference view. Flat art’s real colours are sharp peaks in the histogram, with anti-aliasing a thin spread between them, so packages/inlay/src/palette.ts takes the peaks (5-bit bins, most frequent first, accepted only if perceptually distinct in OKLab) and passed them to VTracer as a fixed palette (Inlay uses them the same way now). That recovered the source colours almost exactly, and on a real illustration cut mean error from 11.1 to 4.3. On a photo it made things worse (9.9 → 15.3), since photos have no peaks, so Poster kept VTracer’s quantizer; it now uses Inlay’s own (quantizePalette).

layerDifference and cut-out shapes follow the same pattern: big wins on flat art, costs on photos, so they are per-preset rather than global.

Small images. A 146×156 logo has eye slits and teeth 2–3 px wide, and VTracer faithfully traces their pixel steps, so the result came out wobbly with the anti-aliasing as an extra grey ink. Sources under 512 px are now upscaled to 1024 px with smoothing and a blur of half the scale factor before tracing; the same logo then traced to clean curves with every detail intact. Mid-size images are left alone: the 640 px sample was already at 0.93% wrong pixels natively.

Light ink on dark. Line art always traces the dark side, so a white logo on black came out as the background with holes. Line art can now invert first (“Light on dark”), and starts that way when the image’s border is dark. On a white-on-black logo that is the whole job in one step: background gone, white shapes as black paths, 6.5 KB. Line art also defaults to a fixed threshold now; adaptive thresholding closed the thin gaps between the logo’s shapes, so it stays as an option for photos of paper.

Only the latest request ever waits. Dragging a slider fires many trace requests a second and a trace can take a second or more. TraceClient holds at most one waiting request per channel (trace, optimise), replacing it when a newer one arrives, so the preview is never more than one trace behind the slider. The worker is created lazily because React StrictMode’s double effect run would otherwise leave the app holding a terminated worker in development.

The numbers shown are the numbers exported. Path, node and byte counts come from the SVGO-optimised output, computed in the background after each change, not from the raw trace. Before this, the panel showed 41 KB for a file that downloaded at 31 KB.

One domain, two halves. The app started as a tool at mattblack.space/tools/img-to-svg/, proxied by the portfolio, so it was built to live at that same path on both hosts. With inlaytrace.com it moved to /app/, beside the landing page at / and the docs at /docs/, all one Vercel project. The old path redirects permanently, with an absolute URL so the portfolio’s proxy passes the redirect through rather than following it. The site builds to dist/ first, because Astro empties its output folder, and the app builds into dist/app after it.

One copy of each doc. docs/*.md is the source: Boards reads it, and scripts/sync-docs.mjs copies it into Starlight at build time (title from the first heading, order to sidebar order, .md links to site URLs). Planning notes such as the launch plan are left out.

The landing page’s evidence is measured. npm run site:assets traces the app’s sample as the app receives it (640 px JPEG) with Inlay and with VTracer as the old Logo preset ran it, scores both against the clean art at 4×, and writes the shapes and numbers the page draws. The close-up is pinned to the left peak: the window where VTracer is most wrong by pixel count is a gentle curve where the two look the same, and the page’s claim is about corners and straight edges. On this sample VTracer’s shared edges stay within 0.06 px of each other, so the page does not claim it leaves gaps.

The CSP is tested, not trusted. There are two, and Vercel applies every matching header block, so they’re scoped never to overlap. The app’s needs a worker allowance (without it the page loads, accepts a drop, and never traces) and, since it runs no WebAssembly, must not allow 'wasm-unsafe-eval'. The site’s is script-src 'self' plus 'wasm-unsafe-eval' for search (Pagefind) and the sha256 of each inline script Starlight writes (theme, sidebar). npm run hosting:check fails the build if any of that goes missing; after the build, --dist hashes every inline script in the site and fails on one the CSP doesn’t list (so a Starlight update fails the build, not the docs) or on any inline script in the app. npm run preview (scripts/serve.mjs) serves dist/ with vercel.json’s redirects, rewrites and headers, and verify fails on any CSP violation in the console.

Untrusted output stays out of the markup. The trace is shown inline (the overlays and hit targets need real DOM paths), so every d, fill and transform is validated against what the tracers emit before it is rebuilt into the document. Recolours are normalised hex or ignored.

Edits sit on top of the trace until they can’t. The document is { inks, doc, background }. Colour edits (inks) are keyed by the traced colour, so they survive re-tracing, as does the background. doc stays null, meaning “whatever the tracer produces”, until the first geometric edit (a node drag, a delete, a mirror) copies the trace into an editable document with fresh ids. From then on a trace setting would throw the edits away, so changing one asks first; going back to the trace is itself an undoable step. Undo/redo stores whole snapshots (cheap at these sizes, capped at 100) and coalesces a drag or a colour-picker sweep into one step.

Path data is normalised once, then round-trips exactly. Editing converts each path to absolute M/L/C/Z nodes with explicit handles (apps/app/src/lib/pathdata.ts). Numbers are written to three decimals and the serialiser is idempotent, so touching one node never nudges the rest of the drawing.

Mirror is two booleans per shape. For each path: intersect it with the kept half-plane, reflect that across the axis, and unite the two. A shape that crossed the axis comes out as one seamless shape rather than two halves that meet; shapes entirely on the discarded side are dropped; paint order and fills are kept. The geometry is paper.js (MIT), loaded only when you mirror: 72 KB gzipped against an 89 KB app. The unit tests run the same core build under jsdom, and verify renders the exported SVG and compares each half against the other (0.0% of pixels differ on a real logo).

Moves and scales are measured from the start of the drag. Each pointer move sends the whole transform since the press, and it is applied to the shapes as they were then. Applying increments instead would compound rounding over a long drag. The starting shapes are captured in the event handler, not inside the state updater: React can run the updater after the handler has already cleared them at the end of the drag, and in an early version that applied the final move twice (a 30 px drag moved 60 px). The browser check drags by a known distance and measures the result.

Smooth nodes stay smooth. A node whose two handles are in line (within 10°) is smooth, which is how the tracer writes curves that meet with matching tangents. Dragging one of its handles swings the other round to stay opposite, keeping its length; ⌥ breaks the pair. Whether a node is smooth is decided at the press, so a drag that bends a handle past the threshold doesn’t suddenly let go of its partner.

Marquees take shapes wholly inside. Touching is the other common rule, but a trace usually has one large shape behind everything, and touching would add it to every selection. A marquee can start in the margin round the image, for when that background shape covers all of it.

PDF and EPS are written by hand. Both are a few path operators per shape (moveto, lineto, curveto, closepath, fill), so a library would add more than it saves. The PDF is five objects with a Flate-compressed content stream and a byte-exact cross-reference table; the tests check every offset, and the file passes qpdf --check and renders back the same as the SVG in Poppler. Coordinates are flipped (both formats put the origin bottom-left) and scaled from the viewBox to points.

Merging colours happens before regions exist. Making two colours the same leaves two shapes and the seam between them. Merging relabels one colour’s pixels as the other’s after palette labelling (so anti-aliased edges were still judged against both) and before specks, components and edges are built, so the seam never exists. It’s a trace setting, so it costs a re-trace and asks first after shape edits.

Overlays never export. The canvas is three layers: the original, the vector exactly as exported, and an overlay svg with hit targets, outlines, nodes, handles and the mirror axis. Nodes and handles are sized in screen pixels (scaled by the current zoom), so they stay grabbable at 8×. A press only becomes a drag after 3 px of travel and keeps the grab offset, so clicking a node selects it without nudging it. Invisible hit targets sit beneath every visible node, so a neighbour’s target can’t steal the node under the pointer.

Zoom is a scale, not a level. Zoom is stored as screen pixels per original pixel (null means fit, which follows the window), so 100% means the same thing however large the image was traced. The slider and the ± steps are logarithmic (√2 a step), and steps stop at fit and at 100% rather than skipping past them. A trackpad pinch over the canvas zooms around the cursor: Chrome, Firefox and Edge send it as ctrl+wheel, Safari as gesture events, and both are handled; ⌘/Ctrl + wheel, ⌘/Ctrl + = − 0 1 work too. Past fit, the sheet gets half a window of padding on each side, which is what lets the point under the cursor stay put (0.1% drift over a 1,100% pinch) even near an edge.

Busy traces stay responsive. A Poster trace of a photo can be 2,500 paths and 60,000 nodes. The node overlay is one <path> and is skipped (with a note) past 20,000 nodes; each shape is a memoised element; path data is validated once per path, not per render; and the stats panel measures only the optimised file. On that trace a node drag went from 117 ms to 25 ms a move, and toggling the overlay from 2.4 s to 0.3 s.

Poster moved to Inlay last, and a bug was the gap. Photos stayed on VTracer because they have no histogram peaks and cut-out shapes cost size on them. Inlay gained a quantized palette (k-means in OKLab, seeded deterministically) and a stacked mode, but the first photo scores were poor: ten times VTracer’s shapes and 2.5 times its size. A larger speck setting made it worse, which pointed at the cause: merging specks ran once, so a speck that joined a small region which then joined a third was stranded as a new speck. Clean art never cascades; photos did thousands of times. Repeating the merge until nothing is left took the photos from 5,150 shapes to 222, and Inlay then beat VTracer on all three photos (mean error 14.3 → 11.8, 57% fewer shapes, 30% smaller), so the app dropped VTracer, its WebAssembly and the 'wasm-unsafe-eval' allowance. Stacking itself saves less than expected on photos (about 3%): most photo regions are small and have few holes. A radius-2 blur before tracing cut error a further 10%, but softens real edges; it’s left for a future setting. At 2048 px Poster takes 3.4–5.1 s on these photos in Node, about 15% longer than VTracer took (2.9–4.5 s), and the SVG is about 30% larger at that size; at the photos’ own size (450–640 px wide) it’s both smaller and closer.