Entity · model

Mixtral-8x7B-v0.1

modelactivemixtral-8x7b-v0-1-c48411df·1 events·first seen Jun 3, 2026

Aliases: Mixtral-8x7B-v0.1

Co-occurring entities

Qwen3-30B-A3B-Base Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models CounterFact

More like this (12)

Mixtral 8x7B Instruct Mixtral 8x22B Mixtral 8x7B Mixtral Mistral-7B-v0.3 RigoChat-7B-v2 Mistral 7B Instruct v0.2 ms-marco-MiniLM-L-6-v2 Mistral Compute RegMix LLaDA-1.5-8B Qwen3VL-8B

Recent events (1)

5arXiv · cs.CL·Jun 3, 2026·source ↗

Expert-aware causal tracing of factual recall in sparse MoE language models

A new arXiv preprint extends causal tracing methodology to sparse mixture-of-experts (MoE) language models, asking which routed experts mediate factual recall rather than just which layers or feed-forward modules. Using CounterFact facts, the authors apply noise-corruption and clean-patch interventions to Qwen3-30B-A3B-Base and Mixtral-8x7B-v0.1, finding that expert-level localization is possible in the former (a single expert at layer 44) but requires multi-expert coalition recovery in the latter. The results indicate that factual localization in MoE models is model- and protocol-dependent rather than universal.

Evaluation and Benchmarking AI Safety Research Qwen3-30B-A3B-Base Mixtral-8x7B-v0.1 Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models +1 more