Entity · paper

Rate-Utility Frontiers for Language Encodings: Comparing Tokens, Bytes, and Pixels Under Controlled Linguistic Content

paperactiveprovisional

rate-utility-frontiers-for-language-encodings-comparing-tokens-bytes-and-pixels-under-controlled-linguistic-content-751d9038

·1 events·first seen Jul 20, 2026

Aliases: Rate-Utility Frontiers for Language Encodings: Comparing Tokens, Bytes, and Pixels Under Controlled Linguistic Content

More like this (12)

Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer The Value Axis: Language Models Encode Whether They're on the Right Track Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models How Surprising Is Historical Italian to Language Models? Tokenization Tax, Comprehension Tax, and a Simple Mitigation Content is What Remains: Invariant Speech Tokenization from Parallel Utterances Translation as a Computationally Efficient Bridge: Feasibility of English BERT for Low-Resource Languages Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy Optimization Dynamics Imprint Semantic Specificity in Contrastive Embedding Norms First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers

Recent events (1)

5arXiv · cs.CL·Jul 20, 2026·source ↗

Rate-utility frontiers reveal no dominant encoding across tokens, bytes, and pixels for language models

A new arXiv preprint introduces a controlled framework for comparing text encodings (subword tokens, raw bytes, rendered pixels) by sweeping a shared bottleneck width to trace rate-utility frontiers across 13 languages and 5 scripts. The study separates three often-conflated quantities: input positions, latent capacity, and task-relevant information surviving compression. Evaluated on surface form preservation, cross-lingual alignment, and topic classification, no encoding dominates across all tasks or capacity regimes — pixels excel at surface form, bytes at cross-lingual alignment, and tokens at topic prediction. The findings reframe encoding choice as a task- and capacity-dependent tradeoff rather than a fixed preference.

Evaluation and Benchmarking Multimodal Progress Rate-Utility Frontiers for Language Encodings: Comparing Tokens, Bytes, and Pixels Under Controlled Linguistic Content