DeepSeek-Coder-V1-6.7B

Study disentangles task, language, and model roles in code model internal representations using circuit analysis

A new arXiv preprint investigates whether independently trained code language models converge on the same internal representations, using a 2x2 factorial design crossing Python/Rust with Qwen2.5-Coder-7B and DeepSeek-Coder-V1-6.7B. The study measures 58 Python and 57 Rust grammatical concepts via concept-circuit extraction across all four conditions. Key findings: which concepts earn dedicated circuits is consistent across models (task-determined), but where and how those circuits are organized differs substantially by model — Qwen processes concepts in late layers (~L17-19), DeepSeek in early layers (L6-7). The work reframes the 'circuit universality' question, concluding universality holds for representational content but not computational organization.