Prompt Compression via Activation Aggregation

Prompt compression via learned activation aggregation reduces per-query compute with under 2% accuracy drop

A new arXiv preprint proposes compressing instruction prompts into a single activation vector by taking a learned weighted sum of intermediate-layer activations and re-injecting it at an early layer of the target LLM. The method achieves under 2% accuracy degradation relative to full prompt processing, eliminating the need to reprocess fixed instruction prompts on every query. The work also surfaces structural findings about LLM activation spaces: mid-layer representations transfer to early layers, and a single vector can encode recoverable semantic information.