pagedweight-777149d5·1 events·first seen Aliases: PagedWeight
PagedWeight is a new runtime weight management system for Mixture-of-Experts LLMs that dynamically quantizes expert weights to balance GPU memory between model weights and KV cache. The method achieves FP16-equivalent accuracy with up to 72% GPU memory savings and 1.94× throughput improvement, and improves quality over static quantization baselines by up to 39.3% at similar memory budgets. The work targets a practical bottleneck in production MoE serving where KV cache growth competes with weight storage for GPU memory.