Skip to the text

Docs Guides

Known limits

What the engine deliberately leaves alone, what it knowingly does not do, and what has not been looked at. Nothing here is a surprise to the author; some of it is waiting for a page that needs it.

What the rules leave alone

Rule behaviour to know about

Money and grouped numbers

Fonts and avoid

Opt-in rules

Eleven rules are off until named in enable (number.group is described in the rules page) (rules.md). Their known limits:

HTML mode

Markdown mode

Rehype

Hanging punctuation

Speed

Since 0.2.0 the package is faster than typograf on English text in one piece, and about 3 times slower than tipograph and smartypants, which do much less (smartypants only quotes, dashes and ellipses; neither has abbreviations or ranges). The benchmark (apps/bench/RESULTS.md, generated on the author’s machine) reports, for English text in one piece, text mode, best of three (one run at 1 MB):

Tool 10 KB 100 KB 1000 KB
this package 1 ms 6 ms 63 ms
typograf 1 ms 12 ms 118 ms
typopo 39 ms 2104 ms not run (the file estimates about 210 s)
richtypo 4876 ms not run (the file estimates about 485 s, extrapolating a quadratic) not run
tipograph 0 ms 2 ms 22 ms
smartypants 0 ms 2 ms 18 ms
retext-smartypants 5 ms 52 ms 2075 ms

Every figure here is from one run of the benchmark, so only the ratios between the tools mean anything: an earlier run on the same machine put this package at 33 ms and typograf at 59 at 1 MB, with every other tool about half of what it shows above. That is 0.06 s against 0.12 s for typograf at 1 MB; 0.2.0 took it from 0.32 s (the output is identical: a differential run against the engine as it was before the speed work, over 2.5 million combinations, finds no difference). The cost is linear in the text. What made the old engine slow was a regular expression that opens with a lookbehind, which the engine tries at every position; the rules now find a literal first and read the character before it (src/core/scan.ts), and a rule is skipped when a cheap test (Rule.when) shows its trigger is absent. The quote rule is 5 ms per MB of ordinary prose, 60 to 100 ms on text full of unpaired marks.

Since 0.2.1 no known input makes a rule superlinear: a run of one opening mark (“ repeated 40,000 times) took a minute and now takes about 7 ms, and text full of dialogue or unpaired marks costs about 45 to 70 ms per MB (it was 180 to 200). Every rule is linear on a long run of one character (spaces, digits, x, -, --, ., +-, 1x, !, )), and the opt-in space.before-punctuation, punctuation.double and symbol.degree no longer open with a lookbehind (about 1, 2 and 5 ms per MB when switched on, from 40, 40 and 200). bun scripts/stress.ts times each rule at 20,000, 40,000 and 80,000 characters of each such run and flags a doubling that costs more than 2.5 times; src/core/linear.test.ts holds the loose version of that as a test.

On a JavaScriptCore engine (Bun) a lookbehind with \p{L} or \p{N} does not see letters outside the Basic Multilingual Plane. The scan helpers reproduce whatever the running engine does, so the output equals the old engine’s on the same runtime. One residual difference exists, only for ill-formed UTF-16 (a lone high surrogate directly after a fraction, followed by a character at U+E000 or above) and only with symbol.fraction on: the ½ is skipped. Well-formed text cannot reach it. A second JavaScriptCore quirk is in the original zone regex (src/core/zones.ts): when its email alternative scans a run of word characters, dots and hyphens that holds a www. and an astral character follows, the old engine reported the address one character late (.www.😀)😀 zoned ww.😀)😀). The rewrite finds the address correctly, so on such text, and only there, the output differs from the old engine’s. Speed numbers depend on the machine; re-run the benchmark (see contributing.md) to see your own. src/speed.test.ts is a loose budget (linear scaling, a 1 MB ceiling), not a measurement.

Benchmark caveats

These are repeated at the top of apps/bench/RESULTS.md.

Where the time goes in this package, and what to do about it, is in backlog.md.

Distribution

Something wrong or unclear here? Report an issue.