There is a particular class of hardware project that succeeds not because it solves a previously unsolvable problem, but because it collapses the distance between a concept and its physical instantiation in a way that makes the concept feel new. Riddle, by Maxime Rivest, is one of those projects. It turns a reMarkable Paper Pro into a responsive, handwriting-driven AI interlocutor, styled after Tom Riddle's diary from Harry Potter. You write on the page, the ink fades after a pause, and a reply appears stroke by stroke in a cursive hand. No screen glow, no keyboard, no chat interface.
The conceit is theatrical, but the engineering underneath it is serious. This post examines the system architecture, the signal processing pipeline, the display backend choices, and what the project reveals about the current state of on-device and near-device LLM inference.
System Architecture: From Stylus to Sentence
The pipeline Riddle implements is worth tracing carefully, because each stage involves a non-trivial design decision. Raw pen input is captured via Linux's evdev interface at the full 4096-level pressure resolution the hardware exposes. This is not the processed stroke data that higher-level reMarkable APIs would provide; it is the raw hardware event stream. After a configurable idle period of 2.8 seconds, the accumulated strokes are committed and the current page state is rasterised to a PNG.
That PNG is then passed to what the project calls the oracle, a resident LLM process. The oracle reads the handwriting from the image and streams a reply sentence by sentence. The reply is rendered using the Dancing Script font, rasterised, thinned to single-pixel paths via the Zhang-Suen skeletonisation algorithm, and then replayed as animated strokes on the display. The full chain is:
- Raw evdev input at hardware event rate
- Stroke accumulation and 2.8-second idle detection
- Page rasterisation to PNG
- Vision LLM inference via OpenAI-compatible API or local pi process
- Sentence-by-sentence streaming back to the display layer
- Font rasterisation, Zhang-Suen thinning, stroke tracing, animated replay
The choice to use a vision model rather than an on-device handwriting recognition pipeline is architecturally significant. It sidesteps the considerable complexity of robust HTR (handwriting text recognition) on embedded hardware, at the cost of a network round-trip and dependency on an external API. Given that the reMarkable Paper Pro is an ARM64 device with limited compute, this is a reasonable trade-off, though it does mean the system's responsiveness is bounded by network latency and API availability rather than local hardware.
The Display Backend: quill and the Vendor E-Ink Engine
The most technically interesting component in the repository is quill, the takeover display host. E-ink displays present a well-known challenge for interactive applications: the vendor waveform controllers are proprietary, and the update latency characteristics vary significantly depending on which waveform mode is selected. reMarkable's libqsgepaper.so is the vendor library that manages these waveform sequences, and it is not publicly documented.
Riddle's approach to this is an interposition shim. Rather than attempting to reverse-engineer and reimplement the waveform engine, quill interposes on the QImage constructor from Qt, intercepting calls to the vendor library and exposing a small C ABI: quill_init, quill_buffer, and quill_swap. This is the same general technique used by the epfb-re project, and it is the right approach here. It preserves the vendor's waveform optimisations while giving the application direct control over the framebuffer.
The consequence is that the takeover mode achieves what the documentation describes as "the lowest latency there is" for this device. The system stops xochitl, the vendor UI daemon, and drives the e-ink engine directly. This is a meaningful claim: the animated stroke replay, which is central to the diary illusion, requires sufficiently low display latency that the stroke-by-stroke appearance feels fluid rather than chunky. On a standard e-ink panel with a naive full-refresh waveform, this would be impossible. The partial-update waveform modes that the vendor library manages are what make it work.
There is a second, windowed backend called qtfb that runs inside xochitl via AppLoad without requiring takeover. This is the appropriate choice for users who want a safer installation, and the latency difference is presumably perceptible but not disqualifying for casual use.
Handwriting Synthesis: Zhang-Suen and Stroke Replay
The reply animation deserves specific attention because it is doing something more subtle than simply displaying text. The pipeline rasterises Dancing Script glyphs, applies Zhang-Suen binary thinning to reduce the rasterised strokes to single-pixel skeletal paths, then traces those paths into ordered stroke sequences that can be replayed incrementally on the display.
Zhang-Suen thinning is a classical morphological algorithm from 1984, well-studied and computationally cheap. Its use here is pragmatic: given a rasterised glyph, it produces a skeleton that approximates the pen path a human writer would have followed. The stroke tracing step then converts that skeleton into an ordered sequence of points suitable for animated rendering. This is not the same as having ground-truth stroke order data, which would require either a purpose-built font with stroke metadata or a stroke order prediction model. The approximation is visually convincing for a cursive font like Dancing Script, where the skeleton of a connected glyph maps reasonably well to a plausible pen trajectory.
The sentence-by-sentence streaming from the LLM is architecturally coupled to this animation pipeline in a way that matters for perceived responsiveness. Because the oracle streams replies and the display begins animating as soon as the first sentence arrives, the user sees ink appearing while the model is still generating. The measured first-ink latency of 0.9 to 1.1 seconds is competitive for a round-trip to an external API from an embedded device, and the streaming architecture means the subjective wait time is shorter than the total generation time.
Security and Operational Considerations
The project documentation is admirably candid about the risk surface. The application runs as root, stops the vendor UI, and drives hardware directly. The SSH escape hatch instruction is not boilerplate; it is genuinely necessary advice. Anyone who installs this and then encounters a display lockup without SSH access has a bricked tablet until they can perform a factory reset.
The API key handling is worth noting. Keys are stored in a plaintext oracle.env file on the device. For a personal device used in a home environment this is probably acceptable, but it is worth being explicit: the reMarkable Paper Pro is a networked device running as root, and the API key stored on it has whatever permissions the issuing account grants. Users connecting to OpenRouter or similar aggregators should use keys scoped to the minimum necessary permissions and set spending limits.
The HTTPS implementation is described as pure-Rust with no extra libraries. This is consistent with the project's broader approach of minimising external dependencies on the device, and Rust's TLS ecosystem (rustls, in all likelihood) is mature enough that this is not a concern in practice.
Broader Context: Ambient AI and the Physics of Interaction
Riddle sits at the intersection of two active research areas. The first is ambient or embodied AI interaction, the question of how AI systems can be integrated into physical artefacts in ways that feel natural rather than grafted on. The second is the practical question of how much of an LLM inference pipeline can or should run locally on constrained hardware versus offloaded to remote APIs.
On the first question, Riddle makes a strong argument by demonstration. The Harry Potter framing is not merely whimsical; it is a precise specification of the desired interaction model. The diary metaphor removes the need to explain latency (the diary is thinking), removes the need for a conventional UI, and creates a strong expectation about the pace and register of the interaction. This is good interaction design, and it suggests that themed or metaphor-driven AI interfaces may be underexplored relative to the generic chat paradigm that currently dominates.
On the second question, the project makes a pragmatic choice that will age in an interesting way. Right now, running a capable vision model locally on an ARM64 device with the reMarkable's memory constraints is not feasible. But the trajectory of model compression, quantisation, and hardware capability suggests that within a few years, a small vision-capable model running entirely on-device would be plausible. The architecture Riddle uses, with the oracle abstracted behind an environment variable and a streaming API, is well-positioned to swap in a local inference backend without changing the rest of the system.
The project is available at github.com/MaximeRivest/Riddle under the MIT licence, with the expected caveat that the vendor libraries it interposes are proprietary and must come from your own device. For anyone with a reMarkable Paper Pro and an interest in the intersection of physical media and language models, it is worth the setup time. The demo is genuinely striking, and the codebase is clean enough to learn from.