Entity · paper

What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations

paperactiveprovisionalwhat-where-and-how-disentangling-the-roles-of-task-language-and-model-in-code-model-representations-76b0503a·1 events·first seen 3d ago

Aliases: What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations

Co-occurring entities

DeepSeek V4 Alibaba Cloud DeepSeek-Coder-V1-6.7B Qwen2.5-Coder

More like this (12)

Reasoning Language Models From Isolated Tasks to Structured Capabilities: A Multilayer Taxonomy for Large Language Models The Value Axis: Language Models Encode Whether They're on the Right Track Verbalizable Representations Form a Global Workspace in Language Models Message Passing Language Models Transformer Language Models Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families Words as Difference Makers: How Large Language Models Determine Causal Structure in Text Contrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures Flow Map Language Models AnyLanguageModel Latent Context Language Models

Recent events (1)

5arXiv · cs.CL·3d ago·source ↗

Study disentangles task, language, and model roles in code model internal representations using circuit analysis

A new arXiv preprint investigates whether independently trained code language models converge on the same internal representations, using a 2x2 factorial design crossing Python/Rust with Qwen2.5-Coder-7B and DeepSeek-Coder-V1-6.7B. The study measures 58 Python and 57 Rust grammatical concepts via concept-circuit extraction across all four conditions. Key findings: which concepts earn dedicated circuits is consistent across models (task-determined), but where and how those circuits are organized differs substantially by model — Qwen processes concepts in late layers (~L17-19), DeepSeek in early layers (L6-7). The work reframes the 'circuit universality' question, concluding universality holds for representational content but not computational organization.

AI Safety Research What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations DeepSeek V4 Alibaba Cloud +2 more