Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains
JetBrains has released Mellum2, a 12-billion-parameter Mixture-of-Experts model announced via the Hugging Face blog. The model appears to be a successor to JetBrains' earlier Mellum code-focused model. No body content was provided, so specific capability details, benchmarks, or licensing terms are unavailable from this source.
Related guides (3)
Related events (8)
Mixtral 8x7B: Mistral AI Releases Sparse Mixture-of-Experts Open-Weight Model
Mistral AI has released Mixtral 8x7B, a sparse mixture-of-experts (SMoE) model with 46.7B total parameters but only 12.9B active parameters per token, enabling inference speed and cost equivalent to a 12.9B model. Licensed under Apache 2.0, Mixtral outperforms Llama 2 70B on most benchmarks and matches or exceeds GPT-3.5, with support for 32k context, five European languages, and strong code generation. An instruction-tuned variant (Mixtral 8x7B Instruct) achieves 8.3 on MT-Bench, claimed best among open-source models at release. The model is deployed behind Mistral's mistral-small API endpoint and supported via vLLM with Megablocks CUDA kernels.
Welcome Mixtral - a SOTA Mixture of Experts on Hugging Face
Hugging Face published a blog post welcoming Mixtral, Mistral AI's Mixture of Experts (MoE) language model, to the platform. The post covers Mixtral's architecture, which uses 8 experts with 2 active per token, and its integration into the Hugging Face ecosystem including transformers support. Mixtral was positioned as a state-of-the-art open-weights model competitive with much larger dense models.
Mixture of Experts Explained
This Hugging Face blog post provides a technical overview of the Mixture of Experts (MoE) architecture, explaining how sparse gating mechanisms route tokens to subsets of expert feed-forward layers to achieve computational efficiency. The post covers training dynamics, inference considerations, and the tradeoffs between dense and sparse models. It serves as a reference document contextualizing MoE's growing relevance following high-profile model releases using the architecture.
Mistral AI Releases Mixtral 8x22B Under Apache 2.0
Mistral AI has released Mixtral 8x22B, a sparse Mixture-of-Experts model with 141B total parameters but only 39B active parameters, under the permissive Apache 2.0 license. The model features a 64K token context window, native function calling, multilingual support across five European languages, and strong math and coding performance. Mistral claims it outperforms all other open-weight models on standard benchmarks while being faster than dense 70B models due to sparse activation. An instructed version achieves 90.8% on GSM8K maj@8.
Qwen2.5-Max: Large-Scale MoE Model Release by Alibaba's Qwen Team
Alibaba's Qwen team announces Qwen2.5-Max, a large-scale Mixture-of-Experts language model. The post acknowledges that scaling insights for very large MoE models have been limited, citing DeepSeek V3's recent disclosures as a reference point. The model is positioned as a frontier-scale MoE system developed concurrently with ongoing Qwen2 research.
Mistral Small 4: Unified Multimodal, Reasoning, and Coding MoE Model Released Under Apache 2.0
Mistral AI has released Mistral Small 4, a 119B-parameter Mixture-of-Experts model (6B active per token) that unifies capabilities previously split across Magistral (reasoning), Pixtral (multimodal), and Devstral (coding agents) into a single open-weights model. The model features a 256k context window, configurable reasoning effort via a `reasoning_effort` parameter, native text and image input support, and is released under Apache 2.0. Mistral claims 40% latency reduction and 3x throughput improvement over Mistral Small 3, with benchmark results showing competitive performance against GPT-OSS 120B and Qwen models while producing significantly shorter outputs. The release includes day-0 availability as an NVIDIA NIM and support across vLLM, llama.cpp, SGLang, and Transformers.
Meta releases Llama 4 Maverick 17B-128E multimodal model on Hugging Face
Meta released Llama 4 Maverick, a 17B active parameter model with 128 experts (mixture-of-experts architecture), on Hugging Face. The model supports image-text-to-text tasks, making it a multimodal open-weights release. This is part of the Llama 4 generation, representing Meta's latest open-weights frontier push with MoE architecture.
Mixture of Experts (MoEs) in Transformers
A Hugging Face blog post covering Mixture of Experts (MoE) architectures as applied to transformer models. The post likely explains the technical foundations, training considerations, and practical deployment aspects of MoE models. Given the timing in early 2026, it likely contextualizes recent MoE-based frontier models and tooling support within the Hugging Face ecosystem.