When multiple LLM agents collaborate on a task, they typically communicate by generating text: one model decodes its internal state into tokens, and another re-encodes those tokens from scratch. This pipeline is sequential, slow, and lossy. The sharer's rich internal representations get compressed into a discrete token sequence that may not faithfully capture what it has processed. A new paper, Latent Cache Flow (LCF), proposes a more direct channel: transmitting compressed KV cache representations between models rather than generated text, and doing so in a way that does not require the two models to have processed the same input.
This matters because multi-agent LLM workflows are increasingly common. Retrieval-augmented pipelines, tool-using agents, and ensemble reasoning systems all involve models with distinct roles and distinct contexts. The communication bottleneck between them is a practical engineering problem, not just a theoretical one.
The Problem with Prior Cache Transfer
Cache-to-Cache (C2C), the main prior work in this space, addresses text-based communication by training adapter modules that translate one model's KV cache tensors into residual updates for another model's cache. The approach is elegant but has two significant drawbacks. First, C2C is position-wise: it operates by aligning token positions between the sharer and receiver, which only works when both models have processed the same input. Second, the adapters are large. For a 1.1B-parameter model pair, C2C's adapter weighs 956 MB, which is a substantial overhead for a component that sits between two already-large models.
These constraints make C2C poorly suited to the settings where cross-model communication is most useful, namely cases where models have read different documents, used different tools, or are otherwise operating on non-overlapping context.
Key Contributions
LCF addresses both problems through two related ideas:
- Joint key-value compression: Rather than maintaining separate fuser modules for keys and values (as C2C does), LCF routes both through a shared low-dimensional bottleneck. This joint compression reduces the adapter to roughly 4% of C2C's parameter count while preserving or improving downstream accuracy.
- Cross-context cache summarisation (LCF-X): For settings where the sharer and receiver have different inputs, LCF-X pools the sharer's full KV cache into a fixed-size summary tensor using hierarchical learned queries. This summary is then projected into the receiver's cache space, allowing information transfer without any requirement for token-position alignment.
- Structured layer pruning: After training, layers whose learned Gumbel-sigmoid gates are near zero are removed in a three-pass audit: dead layers by gate inspection, harmful layers by single-layer ablation, and then ranking of the remainder by ablation importance. This further reduces adapter size with minimal accuracy loss.
The result is a 13 MB adapter (the pruned LCF-128-9L variant) that outperforms C2C's 956 MB adapter across all four evaluation benchmarks in the shared-context setting.
Experimental Results
The shared-context experiments pair Qwen2.5-0.5B-Instruct as the sharer with Qwen3-0.6B as the receiver, with both base models frozen. LCF is evaluated on MMLU-Redux, ARC-Challenge, OpenBookQA, and MMLU-Pro using zero-shot logit-based scoring.
The results are clear. LCF-256 achieves a weighted average accuracy of 32.88, compared to C2C's 29.13, using a 107 MB adapter. The aggressively pruned 9-layer, 13 MB variant still reaches 31.26 weighted accuracy, beating C2C on all four benchmarks. The non-monotonic behaviour across bottleneck dimensions (LCF-512 underperforms LCF-256) is an interesting finding: the bottleneck appears to act as both a compression mechanism and a regulariser, and simply increasing capacity does not help beyond a point.
One result worth highlighting is that LCF-256 exceeds the optimal routing frontier on ARC-C and MMLU-Pro. The optimal routing frontier is an oracle that selects the correct answer whenever either base model is correct. Surpassing it suggests that cache-level communication is not merely aggregating existing model knowledge but recovering information that neither frozen model could access independently.
The cross-context experiments use partitioned HotpotQA, where the two-hop supporting evidence is split between the sharer and receiver. LCF-X reaches 35.13 F1 and 25.28 exact match on the bridge split, compared to 32.66 F1 and 20.53 EM for the best text-to-text baseline (200 generated tokens). Time-to-first-token drops from 410 ms for T2T-Max-200 to 48 ms for LCF-X, an 8.5x reduction. Both quality and latency improvements are statistically significant under paired tests.
Limitations and Open Questions
The authors are appropriately candid about scope. All experiments use small Qwen models (0.5B and 0.6B parameters), and the shared-context experiments test only a single model pair. C2C has been validated across model families and larger scales; it is not yet known whether LCF's compression gains and pruning patterns generalise similarly. The selected layer indices from pruning are specific to the trained checkpoint, so the method generalises in principle but requires re-running the audit for each new model pair.
The cross-context evaluation, while well-designed, is also deliberately controlled. Partitioned HotpotQA is a clean two-hop benchmark where the information boundary is known. Real agentic workflows involve longer contexts, multi-turn exchanges, tool outputs, and heterogeneous model roles. Whether LCF-X's fixed-size pooling can faithfully compress the sharer's state in those settings is an open question.
There is also a broader architectural question the paper leaves open. The pooling approach in LCF-X commits to a fixed summary size, which trades off coverage against compression. For very long contexts or highly information-dense inputs, it is unclear how the pooling quality degrades. Attention-based pooling with learned queries is a reasonable choice, but the relationship between pool size and downstream accuracy across different context lengths would be worth characterising.
Despite these caveats, LCF makes a credible case that cache-level communication is a viable and practical alternative to text-mediated agent coordination. The adapter size reduction alone is practically significant: a 13 MB projector is something that could plausibly be distributed and loaded at inference time without meaningful overhead, which is not true of a 956 MB module. Combined with the latency improvements in the cross-context setting, this positions LCF as a useful building block for tighter multi-model integration.
The full paper is available at arXiv:2605.22863.