OpenDataBox

Systematic evaluation of 12 agent memory systems from a data management perspective

A new arXiv preprint proposes an analytical framework decomposing agent memory into four core modules—representation/storage, extraction, retrieval/routing, and maintenance—and evaluates 12 representative memory systems across five benchmark workloads spanning 11 datasets. The study finds no single architecture dominates across scenarios; effectiveness depends on alignment between memory structure and workload bottleneck. Fine-grained ablation studies quantify effects on retrieval precision, update correctness, and long-horizon stability, and reveal that localized maintenance is more cost-efficient than global reorganization. Code is publicly released.