technique

Transformer Grammars

techniqueactiveprovisionaltransformer-grammars-11373244·1 events·first seen 25h ago

Aliases: Transformer Grammars

Co-occurring entities

More like this (12)

Transformer Language Models Graph Transformer transformer architecture Sentence Transformers Recursive Language Models (RLMs)Variable-Width Transformers generative language modeling Sparse Transformer TRL (Transformer Reinforcement Learning)Transformers Pipelines infini-gram Training-Free Looped Transformers

Recent events (1)

3arXiv · cs.CL·25h ago·source ↗

Systematic study of tree traversal methods in Transformer Grammars reveals trade-offs between composition and lookahead

A new arXiv preprint evaluates Depth-First, Breadth-First, and a novel hybrid Production-Rule Traversal strategy for linearizing syntactic trees in Transformer Grammars. The authors test these methods across language modeling, syntactic generalization, and summarization tasks with varying tree configurations and masking strategies. The study reveals inherent trade-offs between nested composition and global lookahead, offering design recommendations for task-aware Transformer Grammars.

Evaluation and Benchmarking Transformer Grammars Production-Rule Traversal