Cache-to-Cache: Direct Semantic Communication Between LLMs (2025)
summary
Current multi-LLM systems rely on text-based communication, which loses rich semantic data and slows down response times due to token generation. Cache-to-Cache (C2C) solves this by allowing models to share internal KV-caches directly via a neural network projection. This method preserves deep semantic information and removes the need for intermediate text. The result is a system that is both more accurate and significantly faster than traditional text-based collaboration.
key points
- The KV-Cache serves as an efficient medium for communication because it contains high-dimensional semantic representations that text cannot capture.
- A learnable gating mechanism and projection network allow the target model to selectively integrate and fuse the source model's cache into its own layers.
- C2C outperforms text-based communication by improving accuracy by 3.1% to 5.4% and reducing latency by approximately 2.5 times.
who should read it
AI researchers and engineers working on multi-model architectures and LLM efficiency. It is specifically useful for those looking to reduce latency in collaborative AI pipelines.
discussion mixed
consensus
Commenters agree that moving beyond lossy text-based communication between models would improve efficiency and semantic richness. There is a shared belief that 'Neuralese' or latent space communication allows for more compact and precise coordination between different model sizes. However, this shift toward non-human-interpretable representations poses a significant challenge for monitoring and safety.
top insight
The idea of 'KV alignment' across a model family allows an expensive reasoning model to delegate subgoals to a smaller model without prefill recompute or handover latency. This would enable a hybrid architecture where models share a common latent representation of intent, eliminating the redundant processing typical of multi-agent orchestration.
Generated by an LLM from the article and thread. Check the sources before relying on it.