4arXiv cs.CL (Computation and Language)·
LCF: Linguistically-grounded code-mixing fingerprints for LLM ownership verification
A new arXiv preprint introduces LCF (Linguistically-Constructed Fingerprints), a unified framework for embedding ownership signals into LLMs via code-mixed triggers that combine low-resource languages under semantic-density substitution and grammar-biased mixing rules. The companion injection method, LCFEdit, uses null-space projection from multilingual representations to embed fingerprints while preserving model utility. The approach addresses two known weaknesses of prior fingerprinting methods: accidental activation of natural-language triggers and perplexity-based filtering of garbled triggers. Evaluations report persistent ownership verification with negligible utility degradation.
Related guides (1)
Related events (8)
Diagnostic framework decomposes LLM difficulty on historical Italian and Russian texts
A new arXiv preprint proposes a four-dimensional framework for measuring LLM difficulty on historical language: tokenization cost, surprisal, semantic robustness, and context sensitivity. Evaluated on 17th-century Italian, 19th-century Italian, and 18th-century Russian texts, the study finds that tokenization penalties (25-30% inflation) are similar across languages but predictive difficulty diverges sharply—early modern Italian is 2.4x more surprising than modern Italian while Russian shows only modest increase. Crucially, embedding similarity remains high (>0.85) even when generation is unstable, and a simple temporal context prompt reduces historical surprisal by ~60%. The findings have practical implications for deploying LLMs in digital library and historical document workflows.
LLM detection tools can paradoxically increase LLM usage and reduce output quality, paper argues
A new arXiv preprint models LLM detectors as behavioral interventions and shows that imperfect detectors can produce counterintuitive outcomes: users may increase LLM usage and produce lower-quality outputs in response to detection pressure. The authors develop a stylized game-theoretic model of strategic user behavior and empirically validate a predicted 'rise-then-fall' pattern in detectable word frequencies using arXiv abstract data. The work identifies systematic failure modes for LLM detection policies deployed in academic, professional, or platform contexts.
Mathematical framework models linguistic monoculture risk from widespread LLM writing assistance
A new arXiv preprint develops a formal mathematical framework to analyze how widespread LLM-assisted writing may reduce population-level linguistic diversity, a phenomenon the authors term 'linguistic monoculture.' The paper models authors and LLMs as distributions over linguistic features and analyzes three interaction mechanisms—fixed shared models, recursively updated shared models, and personalized models—characterizing equilibria and convergence rates for each. A key finding is that individually rational authors may over-conform relative to the social optimum because they do not internalize the value of their own distinctiveness, creating a negative externality. The 'price of monoculture' can grow without bound when distinctiveness dominates authenticity in the utility model.
Systematic study of extrinsic and intrinsic properties for effective code interpreter reasoning in LLMs
Researchers investigate what behavioral properties make LLMs effective at reasoning with a Code Interpreter (CI), identifying two axes: extrinsic 'crucial tokens' and intrinsic 'cognitive behaviors' such as verification, backtracking, and backward chaining. Stronger CI reasoning models consistently exhibit higher prevalence of these properties. The paper shows that appending code-specific crucial tokens at inference time improves performance on mathematical, ordering, and optimization tasks, while augmenting training with cognitive behaviors improves SFT and RL performance in two of three evaluated models. The work also finds these behaviors reduce overthinking in incorrect responses and improve token efficiency.
LLMs susceptible to cognitive heuristics in code vulnerability detection, enabling black-box attacks
A new arXiv paper presents the first systematic study of cognitive heuristics — halo effect, framing effect, and anchoring — in LLM-based code vulnerability detection. Evaluating eight LLMs across three programming languages, the authors find all models susceptible, with framing causing the largest average shift (33.2%), followed by anchoring (23.5%) and halo (18.4%). Critically, the paper demonstrates a proof-of-concept black-box attack that suppresses up to 97% of previously detected vulnerabilities by manipulating surrounding context without changing the code itself. The findings suggest cognitive susceptibility is a systematic and exploitable property of LLM security tooling.
Formal framework for valid extractable memorization claims in LLMs
A new arXiv preprint proposes a principled methodology for making valid extractable memorization claims about LLMs, addressing both over- and under-statement problems in prior work. The core contribution is a 'matched comparison' approach that measures generation probabilities of training sequences against comparable non-training sequences to establish a calibrated baseline for predictability. Two formalizations are offered: a conformal test for population-level claims and a census method for single-document claims. Applied to OLMo 2 32B on Wikipedia and Llama 3.1 70B on books, the framework reveals significant false-positive rates in naive extraction studies and supports memorization claims at probability thresholds as low as 1e-27.
Systematic study reveals effectiveness-fluency trade-offs in LLM conditioning methods
A new arXiv paper systematically evaluates a range of LLM conditioning methods across both concept injection and removal scenarios, finding that efficient steering methods often degrade fluency significantly. A key finding is that activation steering is substantially less effective on instruction-tuned models than on base models, a previously overlooked interaction. Simple prompting and supervised fine-tuning work for concept injection but not removal, and cheap textual metrics are found to correlate well with expensive LLM-as-judge evaluations.
Token probability measurements reveal production-perception asymmetry in LLMs
A new arXiv preprint investigates whether LLMs exhibit a functional analog to the psycholinguistic production-perception distinction, using direct token probability measurements rather than metalinguistic prompting. Using Llama-3.1-8B and four other open-weight models, the authors find that production-perception prompt distances consistently exceed production-production distances by a ratio of ~1.8, with near-ceiling correlations in the production-production control confirming the effect is specific to communicative framing. The effect replicates across five models spanning base and instruction-tuned variants, and temporal analysis shows perception prompts exert strongest influence at sequence beginnings. The findings suggest prompt framing alone induces a production-perception distinction in decoder-only architectures.