A persistent tension in language model design sits between retrieval-augmented generation and fully parametric approaches to knowledge. RAG keeps knowledge external and auditable but pays for that with retrieval latency, context overhead, and relatively shallow integration. Parametric methods bake knowledge into weights, which is efficient at inference time but makes updates and cross-model reuse genuinely difficult. A new paper, Cross-Model Memory Transfer via Target-Side Reader Adaptation, asks whether a third approach, Engram-style hashed external memory, can thread this needle by treating a trained memory table as a portable artifact that different backbones can read through lightweight adaptation.
The question is sharper than it might first appear. If a memory table trained alongside one backbone is merely a co-adapted extension of that model, it offers no real modularity advantage over standard fine-tuning. The paper operationalises this as a falsifiable test: freeze the source memory, remove the source backbone entirely, attach the table to a different target model, train only a small reader on the target side, and measure whether the frozen table still provides useful signal. The answer, across a surprisingly broad set of model families and scales, is yes.
What Engram Memory Actually Is
Engram augments a Transformer backbone with an external table addressed by deterministic n-gram hashing rather than by dense retrieval over a document store. For each token position, canonicalised n-grams are hashed to table entries, the retrieved vectors are concatenated, and a small learned reader projects them into the backbone residual stream through a gated mechanism. The key structural property is the clean separation between addressing, storage, and reading. Because addressing depends only on the canonicalised input text rather than on model hidden states, the table can in principle be treated as a standalone artifact once the address space is standardised across tokenisers.
This is the architectural bet the paper exploits. If the addressing scheme can be made tokeniser-agnostic through a canonicalisation pipeline, then the only thing that needs to change when moving the table to a new backbone is the reader that maps retrieved vectors into the target residual stream. Everything else, the table entries themselves, stays frozen.
Methodology and Transfer Protocol
The experimental setup is two-phase. In Phase 1, a source model trains its backbone, memory table, and reader jointly on a language modelling objective. In Phase 2, the memory table is exported and frozen, the source backbone is discarded, and the frozen table is attached to a target model. Only the target-side reader is trained, again on next-token prediction with no task-specific supervision.
The paper tests this across six model families: Pythia, TinyLlama, Qwen3.5, Phi-4-mini, LLaMA 2, and Mistral, spanning different architectures, tokenisers, and parameter counts from 160M to 9B. The main evaluation is a 3x3 transfer matrix covering all combinations of three source memories and three target models, with controls for random memory (same architecture, untrained weights) and no-memory baselines. A more demanding downstream evaluation uses LLaMA-2-7B as source and Mistral-7B-v0.3 as target, a genuinely cross-family 7B setting with distinct tokenisation and representation spaces.
Reader architecture receives serious attention. The paper tests a minimal single-layer reader and a stronger dual-layer, four-branch variant. The latter is motivated by the observation that the gap between same-model and cross-model reuse is partly a reader capacity problem: the target model needs enough expressive power in the reader to align retrieved vectors from a foreign source into its own residual stream.
Results
The headline findings are consistent across the full transfer matrix:
- Frozen transferred memory improves test perplexity over the no-memory baseline in every one of the nine source-target combinations tested, with relative reductions ranging from 1.6% to 15.7%.
- Transferred memory consistently outperforms size-matched random memory under the same reader fitting protocol, confirming that the gains come from learned memory content rather than from the reader or the table architecture alone.
- Permuted-key controls, where the table entries are intact but address integrity is broken, substantially degrade performance, showing that correct addressing matters alongside content.
- On downstream QA tasks (Natural Questions, WebQA, TriviaQA, TruthfulQA, HotpotQA), the dual-layer four-branch reader achieves an average score of 38.8, nearly closing the gap between same-model and cross-model reuse and outperforming MLP Memory baselines.
- Target-side data efficiency is a genuine advantage: transferred memory reaches strong perplexity with substantially fewer Phase 2 training tokens than learning a new memory from scratch on the target side.
The Qwen3.5 scaling experiments (0.8B source to 2B target) reinforce the Pythia results and add a useful clarification. The intrinsic perplexity advantage of transferred memory is stable across target-side token budgets. Downstream accuracy improvements are positive but task-dependent, varying by +0.8 to +2.5 points across BoolQ, RTE, and SciQ. This distinction between intrinsic and downstream transfer is worth taking seriously: a better initialisation does not automatically translate into better task accuracy if the reader cannot route the stored signal in a task-compatible way.
Limitations and Open Questions
The paper is candid about several constraints. The evaluation-set overlap audit finds no meaningful contamination for most tasks, but the pretraining corpora of the underlying foundation models are unavailable, so the QA results should be read as controlled comparisons rather than as clean state-of-the-art claims. The authors are explicit about this.
Downstream gains are task-dependent in ways that the paper begins to diagnose but does not fully resolve. Appendix D attributes some of this to source-corpus specialisation and out-of-domain effects, but the precise conditions under which transferred memory helps on a given task remain unclear. This is an honest limitation rather than a flaw in the experimental design, but it does mean practitioners cannot yet reliably predict whether a given frozen memory will help on a target task without empirical testing.
There is also a question of scale. The largest experiments operate at 7B parameters, and the memory tables themselves are relatively compact. Whether the approach scales to much larger memory tables or much larger backbone models, where the reader capacity and addressing overhead may interact differently with model behaviour, is left open.
The comparison table in the paper positions frozen-memory extraction as uniquely supporting O(1) retrieval, cross-model portability, surgical deletion, and no context overhead simultaneously. That framing is fair given the design, but it elides the fact that the approach currently requires a source model to train the table in the first place, which limits the zero-shot portability story relative to, say, a document store that can be updated without any neural training.
What the paper does establish cleanly is the central methodological point: external memory portability is a testable property, and the test is whether a new backbone can address, align, and extract useful signal from a frozen artifact without retraining the stored table. Reader design turns out to be first-order in determining whether that test passes. That framing is a genuine contribution to how the field should think about evaluating external memory systems, independent of the specific Engram architecture.