Traverse allows AI to read and write ink

Abstract

Despite the overwhelming popularity of digital ink for note-taking in academic and professional settings, it largely remains a niche research direction, often limited to narrowly focused models for handwriting recognition.

With the rapidly improving capabilities of large language models, there have been multiple efforts to bring ink into the transformer paradigm, such as the Cursive Transformer and Microsoft’s TrInk. Yet none of these efforts have produced a general-purpose tokenizer that would allow LLMs to understand and generate ink the way text tokenizers allow them to understand and generate text.

Traverse is a tokenizer for digital pen sequences, built as a direct successor to the Cursive Transformer, capable of encoding practically any digital ink sequence at any scale with dynamic precision.

On the IAM On-Line Handwriting Database benchmark, Traverse achieves roughly 2.3x lower reconstruction error than the Cursive Transformer at a comparable token budget, using a single fixed 320-token vocabulary in which precision becomes an inference-time parameter.

Related Work

In early 2025, Sam Greydanus and Zachary Wimpee released a research paper called The Cursive Transformer. They proposed a new tokenization approach for pen sequences into discrete tokens by using polar absolute coordinates.

The tokenizer has a vocabulary of pen tokens, where each movement is encoded as a pair of tokens: an angle token associated with a specific absolute rotation angle $\theta$, and a magnitude token associated with a specific absolute travel distance $d$. Apart from these movement tokens, there are two more tokens to indicate a ‘pen up’ event and a ‘pen down’ event.

In short, given a list of strokes, the tokenizer loops over each point within each stroke, keeping track of its current position on the canvas, and for each step picks the closest angle and magnitude tokens that would bring the current position to the next target point. Pen up and pen down tokens are emitted on transition segments, for example for a space between ‘hello’ and ‘world’, or a gap between letters within a word.

Suppose the vocabulary has 4 rotation bins (0°, 90°, 180°, 270°) and 2 absolute distance bins (5px and 10px), plus dedicated pen-up and pen-down tokens. If you want to draw a 10×20 rectangle with a starting point at (0, 0), both side lengths divide evenly into the available bins:

Now suppose you scale that rectangle to 1.5×, giving a 15×30 rectangle. There is no 15px or 30px bin, so the tokenizer has to approximate each side with multiple tokens, producing a completely different and longer sequence for the same shape:

This illustrates the core issues that make the approach largely impractical for large-scale pen sequence modelling:

• Travel distances are absolute, so the same shape at different scales produces completely different token sequences. This wastes model capacity on learning scale rather than structure, and small strokes become impossible to represent at all.
• The resampling used to normalize sample rate is also scale-dependent, so small strokes get resampled into an uninterpretable mess of straight lines.

Method

Traverse is a scale-invariant tokenizer for digital pen sequences based on the Cursive Transformer work, introducing a number of significant improvements:

• Scale Invariance. Encoded ‘hello’ at 1x and ‘hello’ at 2x will result in identical or near-identical token sequences.
• Inference-time Dynamic Precision. Despite the fixed vocabulary, the tokenizer allows encoding sequences with any precision value, making it possible to adjust the speed/quality ratio in model deployment.
• Adaptive Precision. If the stroke sequence changes its scale within the sequence (for example, you have an equation and the power of some term is written in small print), the precision will automatically adapt to the new scale, preserving the precision value relative to the rest of the handwriting, making it possible to express any stroke sequence.
• Curvature-based resampling. Instead of resampling by point density in absolute terms, the resampling is curvature-based, making it scale invariant.

Traverse operates in two stages: raw strokes are first resampled based on curvature, then the resampled points are encoded into discrete tokens using relative distance bins at a given precision value. The following sections describe each stage in order.

Resampling

One of the key components that made the Cursive Transformer work was resampling, which normalizes the uneven point density that comes from varying capture rates and pen speeds. The problem is that its resampling operates on an absolute distance threshold, so while it handles sample rate, it remains scale-dependent.

For the tokenizer to produce consistent sequences across scales, the resampling itself has to be scale-invariant too. Curvature turned out to be a natural choice here, since it’s a dimensionless geometric property that doesn’t depend on scale.

The resampling works by building a dimensionless ‘complexity’ metric for each stroke from its length and curvature. For each segment $i$between consecutive points, the contribution to the stroke’s total complexity is

Each segment’s share of the budget is proportional to its length, but boosted if it bends sharply. Normalizing curvature by the perimeter is what makes this scale-invariant, which follows from the property of a circle having the same normalized curvature regardless of its radius.

This gives us each stroke’s total complexity. The point budget is then allocated globally, where each stroke receives points proportional to its share of the total complexity across all strokes, scaled by a density hyperparameter $\rho$(with a minimum of 2 points per stroke). Each stroke is then resampled by placing its allocated points at uniform intervals along its cumulative complexity, so that high-curvature regions receive denser sampling than straight regions.

The result is that a small tight curve and a large gentle arc of the same visual complexity receive comparable point counts, giving a consistent representation of strokes regardless of their absolute size.

Tokenization

Since the resampling is scale-invariant, the relative distance between points would be more or less preserved, regardless of the absolute scale. That notion became the foundation of Traverse’s tokenizer.

Instead of having absolute distance bins, Traverse uses relative multiplicative distance bins. The tokenizer keeps track of a base distance $b$that evolves throughout encoding: each token is picked based on its multiplicative distance bin relative to $b$, and then $b$is updated by multiplying it with that bin’s value.

Unlike the Cursive Transformer, tokens in Traverse pack the distance, the rotation, and the pen state. That helps the model be more efficient during inference, and easier to spot patterns in pen up and pen down tokens during training.

Because the multiplicative bins can be both additive (2x, 4x) and subtractive (½x, ¼x), and you’re encoding a distance between two points, you can approach the target point to arbitrary precision: prioritizing quality by spending more ‘refinement’ tokens on reaching the target closer, or prioritizing speed of inference by tolerating larger error, but producing smaller token sequences.

Suppose our vocabulary has 12 rotation bins (30° each) and 3 distance multipliers at $[\frac{1}{2}\times, 1\times, 2\times]$, for both pen-down and pen-up states. Now suppose you draw the same 10×20 and 15×30 rectangles using Traverse. $b$is initialized from the first segment, so the first long side is $1\times$, the short side halves it to $\frac{1}{2}\times$, and the pattern continues with the same multipliers regardless of absolute size:

Since every distance bin is a power of 2, $b$is quantized to the nearest power of 2 as well: $b = 2^{\text{round}(\log_2 b)}$. This keeps $b$on a discrete power-of-2 lattice throughout encoding, so two encoders starting from slightly different initial scales snap to the same lattice point and produce identical token sequences. $b$is also re-snapped to the lattice after each pen-up transition, giving every new stroke a clean starting point.

Precision

This notion of variable precision introduces a question: how do you decide that your error is ‘close enough’ for some given precision value? In the original Cursive Transformer, the ‘close enough’ metric was the absolute distance between the current point and the target point. But absolute distance breaks scale invariance: smaller shapes become less precise.

In Traverse, the precision $p$is a ratio where $0 \leq p < 1$, and the acceptance threshold is $(1 - p) \cdot \tilde{b}$, where $\tilde{b}$is the smoothed base distance. At $p = 0$the threshold equals $\tilde{b}$and each point is reached in a single token. As $p \to 1$the threshold approaches zero and token density grows as $\frac{1}{1 - p}$, so the same finite vocabulary can reproduce a shape to any level of detail.

The initial idea was to set the precision value relative to the current segment length, which would be scale-invariant, but also very sensitive to sudden jumps in precision that would fragment the encoding quality, jumping from very dense and precise points in some areas to extremely low precision in other areas, making them practically illegible.

Instead, the precision anchor is computed as an Exponential Moving Average of $b$in log-space:

The latest distance contributes most, while all the previous distances contribute exponentially less, providing a stable signal to normalize precision against. The delta clamping suppresses sudden spikes in $b$across fragments, which further stabilizes the precision.

Due to a large difference between pen up and pen down distances, it was empirically determined that only refining the precision for pen down steps was optimal for retaining geometric features of the shape the ink represents.

The result is that if the writing includes some elements that are relatively smaller than other elements within the same sequence, the encoder dynamically adapts its precision to ensure uniform quality across the ink sequence, while remaining scale-invariant.

Configuration

The hyperparameters used in the experiments below are summarised in the table. Distance and rotation bins are shared between pen-down and pen-up states; the resampling density $\rho$and curvature weight $w$control the curvature-based resampling stage; the smoothing factor $\alpha$and maximum smoothing deviation control the EMA over the base distance $b$.

Evaluation

To quantitatively evaluate performance, the IAM On-Line Handwriting Database is used as a benchmark.

Vocabulary usage uniformity is comparable across both tokenizers, with Traverse landing within a few percentage points of The Cursive Transformer despite operating on a much smaller vocabulary. Uniformity is reported as the normalized Shannon entropy $H / \log_2 |V|$, where $H = -\sum_i p_i \log_2 p_i$over token probabilities $p_i$and $|V|$is the vocabulary size, reaching $1$when every token is used equally.

To make the comparison between tokenization approaches fair, each sample is uniformly resampled (Traverse’s curvature-based resampling is not used here, so both tokenizers operate on the same point sequence), encoded and decoded with Traverse (5 distance bins at [0.125, 0.5, 1, 2, 4] and 32 uniform rotation bins per pen state, for a vocabulary of 320 pen tokens) and The Cursive Transformer (220 shared angle bins and 151 absolute distance bins per pen state, for a vocabulary of 522 pen tokens) to produce reconstructed stroke sequences, then uniformly resampled again to match the original point count. Reconstruction quality is measured as the Mean Squared Error between the original and reconstructed sequences.

To verify the scale-invariance claim directly, the same handwriting sample is encoded at 1x and 2x scale and the resulting token sequences are compared. Because the relative distances between resampled points are preserved across scales and $b$snaps to the same power-of-2 lattice in both cases, the two encodings collapse to the same (or near-identical) token sequence.

At a comparable token budget, Traverse cuts reconstruction error by roughly 2.3x against The Cursive Transformer (177 tokens at $p = 0.4$vs. 160 tokens, MSE 2.2e-4 vs. 5.0e-4). Crucially, this is achieved with a single fixed 320-token vocabulary: precision becomes an inference-time parameter, letting the same tokenizer trade tokens for fidelity from $p = 0$for fast, coarse output up to $p = 0.8$for reconstructions roughly 5x sharper than The Cursive Transformer.

Limitations

The encoder is greedy: at each step it picks the distance bin that gets closest to the target without overshooting. When even the smallest available bin would still overshoot, it picks that smallest bin anyway and takes the overshoot. The achieved precision can therefore fall short of the requested $p$on segments where the local writing scale is much smaller than the current $b$.

Adaptive precision is built on an exponential moving average, so the smoothed base distance doesn’t snap to the new scale instantly. When the writing scale changes sharply, for example between a large header and small body text underneath it, there is a short transition window where $\tilde{b}$is still converging, and the tokens emitted during that window operate at a less fitting resolution.

Encoding itself doesn’t accumulate drift, since the inner loop keeps emitting tokens until the cursor lands within the precision threshold of the target, so any single overshoot is corrected on the next step. Inference is a different story. Because $b$mutates multiplicatively with every pen-down token, a model trained on Traverse has to implicitly track its own evolving $b$to pick the next token on the right scale. Small prediction errors compound into scale drift that is hard for the model to notice and correct, which is much less of a problem for tokenizers like the Cursive Transformer, where each token lives in an absolute reference frame.

Curvature-based resampling allocates points by each stroke’s share of total complexity, and that complexity is dimensionless by construction. A small component like an i-dot or a punctuation mark can therefore claim a share of the point budget comparable to a long stroke of similar normalized curvature, even though two or three points would represent it adequately. In practice, the resampler often over-resolves these small components at the expense of the larger strokes around them.

Finally, despite the significant improvements in token efficiency over the Cursive Transformer, Traverse still produces relatively long sequences. Given the quadratic attention complexity of transformer models, very long sequences can strain memory and inference latency.

Empirical Validation

The tokenizer evaluation above measures reconstruction quality in isolation. To verify that the format is also learnable by a transformer in practice, a small GPT-2-style decoder-only model was trained purely as a demonstration, not as a competitive handwriting model. It uses a vanilla GPT-2 decoder-only architecture with 8 layers, 8 heads, and 468 embedding size, trained on an NVIDIA RTX A4000 for 16 epochs, with a batch size of 24, and a learning rate of 5e-4.

Dataset

The training set consisted of short snippets of handwritten words, sentences, equations and numbers. The data has been collected from volunteer students who agreed to share their study notes made on an iPad app for notetaking, recorded with Apple Pencil. In total, about 50K samples have been collected across 10 students.

To make the dataset supervised, all the handwriting within samples were transcribed using an open-source OCR model.

Vocabulary

In order to allow the model to learn the connection between text symbols and pen tokens, a character-level tokenizer has been used. The allowed text symbols set was built from ASCII characters list, which includes uppercase and lowercase letters, digits, and a short set of symbols.

Apart from text symbol tokens, all the discrete tokens from Traverse have been added, along with a handful of special tokens for prompt formatting. The specific tokenizer parameters (5 distance bins at [0.125, 0.5, 1, 2, 4] and 32 uniformly spaced rotation bins for both pen-down and pen-up states, yielding a vocabulary of 320 pen tokens) were empirically determined by inspecting the bin usage distribution on the training dataset and selecting the values that produced the most uniform coverage across the vocabulary.

Results

After training, the model produces coherent, human-like handwriting from text prompts, reproducing the variation in slant, spacing, and letterform style present in the training set. The samples below were generated at inference time from short text prompts.

The output sequences decode cleanly through the Traverse tokenizer into stroke trajectories, confirming that the format is learnable end-to-end and that a small transformer is capable of producing valid, stylistically diverse ink without any architectural changes beyond a standard GPT-2 decoder.

Conclusion

Traverse shows that a scale-invariant, relative-distance tokenizer can encode arbitrary digital pen sequences with a small fixed vocabulary, while keeping precision as an inference-time parameter rather than baking it into the bin structure. The combination of curvature-based resampling, multiplicative distance bins, and EMA-smoothed adaptive precision lets the same 320-token vocabulary cover everything from coarse, low-token sketches to high-fidelity reconstructions, without retraining or re-quantization.

The empirical results confirm the approach is viable in practice: Traverse achieves lower reconstruction error than the Cursive Transformer at a comparable token budget, and a small GPT-2-style decoder trained on Traverse tokens learns to generate coherent, stylistically diverse handwriting end-to-end.

Several directions remain open. The current encoder is greedy and can fall short of the requested precision near sharp scale transitions, so a non-greedy or beam-style encoder would likely close that gap. The base distance $b$evolves multiplicatively across the sequence, which is harder for a model to track than an absolute reference frame, so dedicated architectural support for state-tracking, or auxiliary supervision on $b$, is a natural next step. Scaling Graphite well beyond its current demonstration size and broadening the training corpus across writing styles, scripts, and capture devices would test whether the tokenization approach holds up at larger scale.

Acknowledgments

Thanks to the volunteer students who contributed handwritten notes that made the training dataset possible, and to the maintainers of GLM-OCR, the open-source OCR model used for transcription.

References

1. Graves, A. Generating Sequences With Recurrent Neural Networks. 2013. arXiv: 1308.0850 [TODO: cite inline]
2. Aksan, E., Pece, F., & Hilliges, O. DeepWriting: Making Digital Ink Editable via Deep Generative Modeling. CHI 2018. arXiv: 1801.08379 [TODO: cite inline]
3. Greydanus, S., & Wimpee, Z. The Cursive Transformer. 2025. arXiv: 2504.00051
4. Jin, Z., Desai, S., Chen, X., Fang, B., Huang, Z., Li, Z., Gan, C.-X., Tu, X., Mak, M.-W., Lu, Y., & Liu, S. TrInk: Ink Generation with Transformer Network. 2025. arXiv: 2508.21098
5. Wang, D. ScribeTokens: Fixed-Vocabulary Tokenization of Digital Ink. 2026. arXiv: 2603.02805 [TODO: cite inline]
6. Duan, S. et al. GLM-OCR Technical Report. 2026. arXiv: 2603.10910