paper
Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddings
paperactiveprovisional
attention-expansion-enhancing-keyphrase-extraction-from-long-documents-with-attention-augmented-contextualized-embeddings-e6f441cf·1 events·first seen 7d agoAliases: Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddings
More like this (12)
DeepSeek Sparse Attentionsparse attentionTrace Only What You Need: Structure-Aware On-Demand Hypergraph Memory for Long-Document Question AnsweringMassive Text Embedding BenchmarkUncertainty-Aware Hybrid Retrieval for Long-Document RAGWhen Does Mixing Help? Analyzing Query Embedding Interpolation in Multilingual Dense RetrievalReference-Augmented TrainingContextual RetrievalLocality-Sensitive Hashing Attentionsentence embeddingsJoint Embedding Predictive Architecture (JEPA)ProbSparse Attention
Recent events (1)
Attention Expansion mechanism improves keyphrase extraction from long documents without full-context LLMs
Researchers propose an 'attention expansion' mechanism that augments pre-trained language model token representations with information from out-of-context chunks using static word embeddings, enabling more effective keyphrase extraction from long documents. The approach avoids the computational cost of full-document attention or LLM-based inference while expanding the effective contextual scope of PLM-based models. Evaluated across five PLM backbones and five benchmark corpora, the method consistently improves F1 scores over state-of-the-art baselines in both scientific and news domains.