technique

global attention

techniqueactiveprovisionalglobal-attention-56af744e·1 events·first seen 22d ago

Aliases: global attention

Co-occurring entities

inter-frame token selection visual geometry transformer Good Token Hunting multi-view 3D reconstruction intra-frame entropy-guided sparsification intra-frame token sparsification

More like this (12)

cross-attention bidirectional attention positional attention heads symbolic attention heads reference attention Functional Attention Lightning Attention self-attention sparse attention attention head circuit Grouped-Query Attention Attention-based MIL

Recent events (1)

5arXiv · cs.LG·22d ago·source ↗

Good Token Hunting: Token Selection Framework for Visual Geometry Transformers

This paper introduces a two-stage token selection framework to address the quadratic computational scaling of global attention in visual geometry transformers used for multi-view 3D reconstruction. The approach combines diversity-based inter-frame selection (frame-level) with entropy-guided intra-frame sparsification (token-level within frames). Experiments demonstrate over 85% acceleration for 500-image scenes while maintaining or improving baseline reconstruction quality, offering a favorable speed-accuracy trade-off.

Inference Economics Agent and Tool Ecosystem inter-frame token selection visual geometry transformer global attention +5 more