Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters
Alibaba's Qwen team releases Qwen1.5-MoE-A2.7B, a mixture-of-experts model with only 2.7 billion activated parameters that claims performance parity with 7B dense models such as Mistral 7B and Qwen1.5-7B. The model activates roughly one-third of its total parameters during inference, offering significant compute efficiency gains. This release follows growing industry interest in MoE architectures sparked by Mixtral, and the model is available on GitHub, HuggingFace, and ModelScope.
Related guides (3)
Related events (8)
Qwen2.5-Max: Large-Scale MoE Model Release by Alibaba's Qwen Team
Alibaba's Qwen team announces Qwen2.5-Max, a large-scale Mixture-of-Experts language model. The post acknowledges that scaling insights for very large MoE models have been limited, citing DeepSeek V3's recent disclosures as a reference point. The model is positioned as a frontier-scale MoE system developed concurrently with ongoing Qwen2 research.
Qwen3 Release: Flagship 235B MoE and Full Model Family Announced
Alibaba's Qwen team has released Qwen3, a new family of large language models including the flagship Qwen3-235B-A22B mixture-of-experts model. The flagship model claims competitive benchmark performance against DeepSeek-R1, OpenAI o1/o3-mini, Grok-3, and Gemini-2.5-Pro on coding, math, and general capabilities. A smaller MoE variant, Qwen3-30B-A3B, reportedly outperforms QwQ-32B despite using only one-tenth the activated parameters, and the 4B model is said to match Qwen2.5's larger models. Models are available across Hugging Face, ModelScope, and Kaggle.
Introducing Qwen1.5: Open-Source Models Across Eight Sizes Including MoE
Alibaba's Qwen team released Qwen1.5, open-sourcing both base and chat models in eight sizes ranging from 0.5B to 110B parameters, plus a Mixture-of-Experts (MoE) variant. The release emphasizes developer experience improvements alongside model quality. Models are available on GitHub, Hugging Face, and ModelScope.
Qwen1.5-32B: Alibaba's 30B-Parameter Capstone for the Qwen1.5 Series
Alibaba's Qwen team released Qwen1.5-32B, a ~30 billion parameter open-weights language model positioned as the capstone of the Qwen1.5 series. The model targets the emerging consensus around 30B parameters as an optimal balance between performance, memory footprint, and inference efficiency. It is released alongside code on GitHub, weights on HuggingFace and ModelScope, and an interactive demo.
Qwen2 Model Family Released: Five Sizes, 128K Context, Multilingual
Alibaba's Qwen team has released Qwen2, an evolution from Qwen1.5, comprising five pretrained and instruction-tuned models ranging from 0.5B to 72B parameters, including a 57B mixture-of-experts variant (57B-A14B). The release highlights training on 27 additional languages beyond English and Chinese, significantly improved coding and mathematics performance, and extended context support up to 128K tokens for the 7B and 72B instruct variants. Benchmark results are claimed to be state-of-the-art across a large number of evaluations.
Qwen releases Qwen3.5-35B-A3B-Base multimodal MoE model on Hugging Face
Qwen has released Qwen3.5-35B-A3B-Base, a 35B-parameter mixture-of-experts image-text-to-text base model on Hugging Face, activating approximately 3B parameters per forward pass. The model supports conversational use and is compatible with Azure deployment endpoints. With over 109K downloads, it represents a notable open-weights multimodal MoE release from the Qwen team.
Qwen releases Qwen3.5-122B-A10B multimodal MoE model on Hugging Face
Qwen has released Qwen3.5-122B-A10B, a 122B-parameter mixture-of-experts image-text-to-text model with 10B active parameters, published on Hugging Face. The model supports conversational use and is compatible with Azure deployment endpoints. High download counts (840K) and likes (564) suggest rapid community uptake shortly after release.
Alibaba releases Qwen3.5 open-weights vision-language model family with MoE architecture across eight sizes
Alibaba released the Qwen3.5 family of eight open-weights vision-language models ranging from 0.8B to 397B parameters, built on a mixture-of-experts architecture with mixed attention and Gated DeltaNet layers. The flagship Qwen3.5-397B-A17B outperforms GPT-5.2, Claude 4.5 Opus, and Gemini-3 Pro on 28 of 44 vision benchmarks, while the 9B model surpasses OpenAI's gpt-oss-120B on most language tasks. Open weights are available under Apache 2.0, with hosted agentic variants (Qwen3.5-Plus, Qwen3.5-Flash) available via Alibaba Cloud. The release is notable for strong small-model efficiency and comes amid reported team departures following the Qwen3 rollout.