technique
Entropy-Aware Dense Pruning
techniqueactiveprovisional
entropy-aware-dense-pruning-2f60508a·1 events·first seen 7h agoAliases: Entropy-Aware Dense Pruning
More like this (12)
Layer-Adaptive Expert PruningDense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy DistillationFast Adaptive Semantic EntropyEntropy Regularizationintra-frame entropy-guided sparsificationlayer pruningContinual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMsdepth pruningProbSparse Attentionhybrid dense-sparse retrievalInstruction-Following PruningEntropy-Cut Metropolis-Hastings
Recent events (1)
EADP: Entropy-aware visual token pruning for efficient VLMs under dense instructions
A new arXiv preprint introduces Entropy-Aware Dense Pruning (EADP), a framework for compressing visual tokens in vision-language models (VLMs) that addresses two failure modes in existing methods: textual noise corrupting cross-modal scoring and feature fragmentation from naive Top-K selection. EADP uses statistical entropy to filter textual noise and reformulates token selection as a submodular maximization problem with a spatial prior. The authors report state-of-the-art accuracy-efficiency trade-offs on multimodal benchmarks under strict token budgets.