information-bottleneck-3f017d9f·1 events·first seen Aliases: Information Bottleneck
A new arXiv preprint formalizes the performance difference between multi-agent systems (MAS) and single-agent systems (SAS) using information bottleneck theory. The core insight is that MAS isolates reasoning into local contexts connected by bounded relay messages, creating a compression trade-off: reducing redundant context can improve efficiency but risks losing task-relevant information. The authors derive a parameter β capturing this trade-off and validate it across 18 experiments on five benchmarks at three model scales, finding MAS helps most when relays are near-sufficient and for weaker models, while stronger models gain little or regress. This provides a principled theoretical framework for when to use multi-agent architectures.