Entity · paper

Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation

paperactiveprovisional

stage-replay-divergence-follows-the-kv-cache-fixed-prefix-precision-controls-and-bidirectional-cache-transplantation-33d19298

·1 events·first seen 34h ago

Aliases: Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation

Co-occurring entities

Qwen2.5

More like this (12)

replay divergence KV Cache KV Cache Quantization Value-aware Stochastic KV Cache Eviction Self-Generated Replay Hindsight Experience Replay FreqDepthKV Record & Replay Key-Value Cache REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing GitOfThoughts: Version-Controlled Reasoning and Agent Memory You Can Replay, Diff, and Merge Error Certificates for KV-Cache Eviction via Randomized Design

Recent events (1)

5arXiv · cs.CL·34h ago·source ↗

KV cache transplantation and precision controls explain stage-replay divergence in LLM inference

A preprint from arXiv investigates why stage-replay diagnostics for LLMs diverge from live inference, finding that KV cache state—not token identity—is the causally sufficient carrier of divergent trajectories in a Qwen2.5-derived system. A 200-item controlled experiment shows BF16 precision produces disagreements on 166 suffixes while FP32 eliminates decoded disagreements entirely. Bidirectional transplantation of all 48 KV layers causes continuations to follow the cache donor with 100% fidelity across two checkpoints (24/24 and 43/43). The findings have direct implications for reproducibility and debugging of multi-step reasoning systems.

Evaluation and Benchmarking Inference Economics Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation Qwen2.5