Open Weights Progress
open-weights-progress·497 events·last 33h agoOpen-weights model releases (Llama, Mistral, Qwen, DeepSeek, Gemma), the gap to closed frontier models, and the ecosystem around fine-tuning and serving them.
Related entities
Related topics (8)
Guides (1)
Recent events (50)
Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context
IBM released Granite Embedding Multilingual R2, an open-weights (Apache 2.0) multilingual embedding model with 32K context window, claiming best-in-class retrieval quality among sub-100M parameter models. The model is positioned for enterprise RAG and retrieval use cases across multiple languages. It is hosted and announced via Hugging Face.
Latest open artifacts (#21): Open model bonanza — Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1 & others
Interconnects' recurring open-weights roundup covers a dense cluster of recent releases including Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, and GLM-5.1, characterizing the period as a flagship-after-flagship cadence. The piece also includes commentary on CAISI's assessment of DeepSeek V4. As a tier-2 commentary source, this is a synthesis and analysis layer rather than primary announcements.
How Open Model Ecosystems Compound
This Interconnects commentary examines how China's open-first, high-participation AI ecosystem creates compounding advantages over time. The piece reflects on the structural dynamics of open model ecosystems and their strategic implications. It appears to analyze how broad community participation in open-weight model development accelerates capability progress.
Notes from inside China's AI labs
A firsthand account from visits to leading AI labs in China, offering observations on their research culture, capabilities, and strategic direction. The piece provides rare insider perspective on the state of Chinese frontier AI development. Published on Interconnects, a tier-2 commentary source focused on the AI/ML landscape.
EMO: Pretraining Mixture of Experts for Emergent Modularity
AllenAI introduces EMO, a pretraining approach for Mixture of Experts (MoE) models that aims to produce emergent modularity during training. The work explores how MoE architectures can develop specialized expert routing without explicit supervision. Published on the Hugging Face blog, this represents research-level work on improving MoE training dynamics and efficiency.
The Distillation Panic
A commentary piece from Interconnects critiques the framing of 'distillation attacks' as a term for the current trend of training models on outputs from frontier systems. The author appears to argue the terminology is misleading or alarmist. The piece engages with ongoing industry debate about knowledge distillation, model output licensing, and competitive dynamics between AI labs.
Qwen-Image: 20B MMDiT Image Foundation Model with Native Text Rendering
Alibaba's Qwen team has released Qwen-Image, a 20B parameter MMDiT (Multimodal Diffusion Transformer) image generation foundation model. The model claims significant advances in complex text rendering capabilities, including multi-line layouts, paragraph-level semantics, and fine-grained typographic details across alphabetic and other language scripts. It also features precise image editing capabilities and is accessible via Qwen Chat and open-weight repositories on HuggingFace and ModelScope.
Reading today's open-closed performance gap
This commentary from Interconnects analyzes the factors that determine benchmark evaluation scores and the performance gap between open-weight and closed frontier models. It examines how various complex variables contribute to the single evaluation numbers that dominate public discourse, and considers how this gap may evolve over time. The piece is framed as an analytical take on the current state of open vs. closed model competition.
My bets on open models, mid-2026
A Interconnects commentary piece forecasting the trajectory of open-weight models through mid-2026, with a focus on the gap between open and closed frontier models. The author offers predictions about which open-weight developments are most likely to close the capability gap with proprietary systems. As a tier-2 source, this represents informed industry analysis rather than primary reporting.
Qwen3-Coder: 480B MoE Agentic Coding Model Released by Alibaba/Qwen Team
Alibaba's Qwen team has released Qwen3-Coder, a family of code-focused models with the flagship variant being Qwen3-Coder-480B-A35B-Instruct, a 480B-parameter Mixture-of-Experts model with 35B active parameters. It supports 256K native context length and up to 1M tokens via extrapolation. The model claims state-of-the-art results among open-weight models on agentic coding, browser-use, and tool-use benchmarks, with performance described as comparable to Claude Sonnet 4.
Gemma 4 and what makes an open model succeed
A commentary piece from Interconnects analyzing Google's Gemma 4 release and the broader question of what drives success for open-weight models. The piece argues that benchmark scores are not the primary determinant of open model adoption or impact. This is a tier-2 analytical take on the open-weights ecosystem and the strategic dynamics around model releases.
The Future of the Global Open-Source AI Ecosystem: From DeepSeek to AI+
Hugging Face publishes a retrospective and forward-looking commentary marking one year since the 'DeepSeek moment,' examining how DeepSeek's open-weight releases reshaped the global open-source AI ecosystem. The piece analyzes the downstream effects on model development, inference economics, and competitive dynamics between open and closed AI labs. It situates these developments within a broader 'AI+' framing, suggesting a new phase of AI integration across industries.
Google Cloud C4 Brings a 70% TCO Improvement on GPT OSS with Intel and Hugging Face
A collaboration between Google Cloud, Intel, and Hugging Face demonstrates a 70% total cost of ownership (TCO) reduction when running open-source GPT-class models on Google Cloud's C4 instances powered by Intel Xeon processors. The post details inference economics for deploying open-weight LLMs on CPU-based cloud infrastructure rather than GPU instances. This represents a notable data point in the inference cost optimization space, particularly for organizations seeking lower-cost alternatives to GPU-based deployment.
SmolVLA: Efficient Vision-Language-Action Model trained on Lerobot Community Data
Hugging Face introduces SmolVLA, a compact Vision-Language-Action model designed for robotics control, trained on community-contributed data from the LeRobot ecosystem. The model targets efficient deployment on resource-constrained hardware while maintaining competitive manipulation performance. This release represents a continuation of Hugging Face's strategy to democratize robotics AI through open community data pipelines.
Granite 4.1 LLMs: How They're Built
IBM has published a blog post on Hugging Face detailing the construction of its Granite 4.1 language models. The post covers architectural and training decisions behind the new model family. As a tier-2 source with default commentary depth, this provides insight into IBM's continued investment in open enterprise LLMs but lacks the full technical depth of a primary research paper.
The Inevitable Need for an Open Model Consortium
Nathan Lambert at Interconnects argues for the formation of an open model consortium, despite acknowledged skepticism about such organizational structures. The piece appears to make a case that coordinated open-weights AI development requires some form of collective governance or collaboration body. Published April 2026, this reflects ongoing debate about how the open-source AI ecosystem should organize itself relative to frontier closed labs.
Claude Mythos and misguided open-weight fearmongering
A commentary piece from Interconnects critiquing what the author characterizes as unfounded fears around open-weight AI models, likely in the context of Anthropic's Claude and its positioning relative to open-source alternatives. The piece appears to challenge narratives that frame open-weight model releases as uniquely dangerous. As a tier-2 source commentary, it reflects ongoing industry debate about open vs. closed model safety arguments.
Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents
NVIDIA has released Nemotron 3 Nano Omni, a multimodal model targeting long-context understanding across documents, audio, and video modalities. The model is positioned for agentic use cases requiring cross-modal reasoning. It is published via the Hugging Face blog as part of NVIDIA's Nemotron model family. No detailed technical specifications or benchmark results are provided in the available body text.
Latest open artifacts (#20): New orgs! New types of models! With Nemotron Super, Sarvam, Cohere Transcribe, & others
Interconnects' recurring open-weights roundup covers several new model releases and organizations entering the open-artifact space. Highlighted items include Nvidia's Nemotron Super, Indian AI lab Sarvam, and Cohere's Transcribe product. The piece tracks the expanding diversity of organizations and model types contributing to the open-weights ecosystem.
DeepSeek-V4: a million-token context that agents can actually use
A Hugging Face blog post discusses DeepSeek-V4, highlighting its million-token context window as a practically usable capability for agentic applications. The post appears to analyze or announce DeepSeek-V4's long-context features in the context of agent workflows. No article body was available for deeper analysis.
QIMMA: A Quality-First Arabic LLM Leaderboard
TII UAE (Technology Innovation Institute) has launched QIMMA, a leaderboard specifically designed to evaluate large language models on Arabic language tasks with a focus on quality-first assessment. The leaderboard aims to address gaps in Arabic NLP evaluation by providing standardized benchmarks tailored to Arabic linguistic characteristics. This represents a dedicated infrastructure effort for tracking Arabic LLM progress, a historically underserved language in evaluation frameworks.
Qwen3 Embedding: State-of-the-Art Text Embedding and Reranking Models Released
Alibaba's Qwen team has released the Qwen3 Embedding series, a set of open-weights text embedding and reranking models built on the Qwen3 foundation model. The models are designed for retrieval and reranking tasks and claim state-of-the-art performance across multiple benchmarks. They are released under the Apache 2.0 license and are available on Hugging Face and ModelScope.
What comes next with open models
A Interconnects commentary piece examining the next phase of open model development, covering market dynamics, capability trajectories, and the broader industrialization of language models. The piece appears to survey the competitive and technical landscape for open-weight models as they mature. Published in March 2026, it reflects on the state of the open-model ecosystem amid rapid frontier progress.
AI and the Future of Cybersecurity: Why Openness Matters
A Hugging Face blog post argues for the importance of open AI models and research in the cybersecurity domain. The piece likely contends that open-weights models enable better defensive security tooling, red-teaming, and vulnerability research compared to closed alternatives. It addresses the dual-use tension between open access and potential misuse in security contexts.
Dean Ball on open models and government control
A commentary piece from Interconnects examines the legal and policy implications of the Anthropic v. Department of War case for the future of open-weight AI models. The piece, attributed to Dean Ball, argues that the case may set subtle but significant precedents regarding government authority over open model distribution and access. The analysis focuses on how the case's outcome could shape regulatory frameworks affecting open-source AI development.
Qwen3 Release: Flagship 235B MoE and Full Model Family Announced
Alibaba's Qwen team has released Qwen3, a new family of large language models including the flagship Qwen3-235B-A22B mixture-of-experts model. The flagship model claims competitive benchmark performance against DeepSeek-R1, OpenAI o1/o3-mini, Grok-3, and Gemini-2.5-Pro on coding, math, and general capabilities. A smaller MoE variant, Qwen3-30B-A3B, reportedly outperforms QwQ-32B despite using only one-tenth the activated parameters, and the 4B model is said to match Qwen2.5's larger models. Models are available across Hugging Face, ModelScope, and Kaggle.
OLMo Hybrid and Future LLM Architectures
Interconnects covers the latest OLMo hybrid model release and discusses emerging trends in open-source post-training tooling. The piece examines architectural directions for future large language models. As a tier-2 commentary source, it provides analysis rather than primary research findings.
Latest open artifacts (#19): Qwen 3.5, GLM 5, MiniMax 2.5 — Chinese labs' latest push of the frontier
A Interconnects newsletter roundup covering recent open-weight model releases from Chinese AI labs, specifically Qwen 3.5, GLM 5, and MiniMax 2.5. The piece frames these as a continued frontier push from Chinese research organizations. The body content is minimal beyond the title and greeting, suggesting this is either a stub or the full content was not captured.
Qwen2.5-Omni: Alibaba Releases End-to-End Multimodal Model with Real-Time Streaming
Alibaba's Qwen team releases Qwen2.5-Omni, a 7B-parameter end-to-end multimodal model capable of processing text, images, audio, and video simultaneously. The model delivers real-time streaming responses in both text and natural speech synthesis. It is openly available on Hugging Face, ModelScope, DashScope, and GitHub, accompanied by a technical paper.
How much does distillation really matter for Chinese LLMs?
This commentary from Interconnects reacts to Anthropic's post on 'distillation attacks,' examining the role of distillation in the development of Chinese large language models. The piece interrogates how much capability transfer via distillation from frontier models actually explains the progress of Chinese LLMs. It situates the discussion within ongoing debates about knowledge distillation as a competitive and security concern.
Qwen2.5-VL-32B: Reinforcement-Learning-Optimized Vision-Language Model Released
Alibaba's Qwen team has released Qwen2.5-VL-32B-Instruct, a 32-billion-parameter vision-language model built on the Qwen2.5-VL series and further optimized with reinforcement learning. The model is open-sourced under the Apache 2.0 license and available on Hugging Face and ModelScope. It follows the January 2025 launch of the broader Qwen2.5-VL series, positioning the 32B scale as a balance between capability and deployment practicality.
Open Models in Perpetual Catch-Up
A commentary piece from Interconnects examining the structural dynamics between open-weight and closed frontier models, covering topics including the open-closed capability gap, distillation as a catch-up mechanism, innovation timescales, and conditions under which open models can win. The piece also addresses specialized models and gaps in the current open ecosystem. This is a high-level analytical framing of a persistent tension in the AI landscape rather than a report on a specific release or event.
QwQ-32B: Scaling Reinforcement Learning for Enhanced Reasoning
Alibaba's Qwen team releases QwQ-32B, a 32-billion parameter model trained with scaled Reinforcement Learning to improve reasoning capabilities beyond conventional pretraining and post-training methods. The release draws explicit comparison to DeepSeek R1's cold-start and multi-stage RL training approach. The model is available via Qwen Chat, Hugging Face, ModelScope, and a demo interface. This represents Qwen's exploration of RL scalability as a path to enhanced LLM intelligence.
QwQ-Max-Preview Released by Qwen Team
Alibaba's Qwen team has released QwQ-Max-Preview, a preview version of their reasoning-focused model built on top of Qwen2.5-Max. The post is itself generated by the model, serving as a demonstration of its capabilities. As a preview release, it signals an upcoming full model launch in the Qwen series.
Safetensors is Joining the PyTorch Foundation
The safetensors format, developed by Hugging Face as a secure and fast alternative to pickle-based model serialization, is being adopted under the PyTorch Foundation. This move formalizes safetensors as part of the broader PyTorch ecosystem, signaling growing standardization around safe model weight storage. The transition reflects increasing industry concern about supply-chain security in ML model distribution.
Welcome Gemma 4: Frontier Multimodal Intelligence on Device
Google has released Gemma 4, a new open-weights multimodal model family announced via the Hugging Face blog. The release positions Gemma 4 as capable of frontier-level multimodal intelligence while being deployable on-device. As a tier-2 source commentary, the post likely covers model capabilities, availability on Hugging Face Hub, and integration details.
Qwen2.5-Max: Large-Scale MoE Model Release by Alibaba's Qwen Team
Alibaba's Qwen team announces Qwen2.5-Max, a large-scale Mixture-of-Experts language model. The post acknowledges that scaling insights for very large MoE models have been limited, citing DeepSeek V3's recent disclosures as a reference point. The model is positioned as a frontier-scale MoE system developed concurrently with ongoing Qwen2 research.
Falcon Perception: TII Announces Multimodal Perception Capabilities for Falcon
TII (Technology Innovation Institute) has published a blog post on Hugging Face introducing Falcon Perception, a multimodal extension of the Falcon model family. The post appears to detail perception capabilities added to the Falcon series, likely covering vision-language or other sensory modalities. As the body content is empty, specific technical details about architecture, benchmarks, or release scope are unavailable from this source.
Qwen2.5-1M: Open-Source Models with 1M Token Context Window Released
Alibaba's Qwen team has released two open-source models, Qwen2.5-7B-Instruct-1M and Qwen2.5-14B-Instruct-1M, extending context length to 1 million tokens. This follows the earlier upgrade of the proprietary Qwen2.5-Turbo to 1M context two months prior. The release includes inference framework support for deployment, marking the first time Qwen's open-weight models have reached this context length.
Granite 4.0 3B Vision: Compact Multimodal Intelligence for Enterprise Documents
IBM released Granite 4.0 3B Vision, a compact multimodal model targeting enterprise document understanding tasks. The model is hosted on Hugging Face and positioned for deployment in resource-constrained enterprise environments. As a 3B-parameter vision-language model, it competes in the small-but-capable segment increasingly favored for on-premise and edge deployments.
Qwen2.5-VL: Alibaba's New Flagship Vision-Language Model Released in 3B/7B/72B Sizes
Alibaba's Qwen team has released Qwen2.5-VL, their new flagship vision-language model, representing a significant upgrade over Qwen2-VL. The release includes both base and instruct variants in three sizes (3B, 7B, 72B), all open-weighted and available on Hugging Face and ModelScope. The 72B instruct model is also accessible via Qwen Chat. Key capabilities highlighted include enhanced visual understanding, with the model positioned as a major step forward in multimodal performance.
Training mRNA Language Models Across 25 Species for $165
A Hugging Face blog post describes training mRNA language models spanning 25 biological species at a total compute cost of $165. The work demonstrates that biological sequence language models can be trained at extremely low cost, potentially democratizing genomic/transcriptomic AI research. The post likely covers model architecture, training data, and cross-species generalization results.
TRL v1.0: Post-Training Library Built to Move with the Field
Hugging Face has released TRL v1.0, a major milestone for its post-training library focused on reinforcement learning from human feedback and related alignment techniques. The release signals a stabilization of the API and feature set after iterative development tracking the rapidly evolving post-training landscape. TRL is widely used in the open-source community for fine-tuning and aligning language models using methods such as PPO, DPO, and GRPO.
Qwen2.5-Math Process Reward Model for Mathematical Reasoning Supervision
Alibaba's Qwen team introduces a process reward model (PRM) aimed at improving the reliability of mathematical reasoning in LLMs by supervising intermediate reasoning steps rather than only final answers. The work addresses the problem of models producing plausible but flawed intermediate derivations even when reaching correct conclusions. The release includes model weights on HuggingFace and ModelScope alongside a GitHub repository.
QVQ-72B-Preview: Qwen Visual Reasoning Model Release
Alibaba's Qwen team has released QVQ-72B-Preview, a 72-billion parameter multimodal model designed to integrate visual understanding with advanced reasoning capabilities. The model is positioned as an extension of Qwen's language reasoning work into the visual domain. It is available on GitHub, Hugging Face, ModelScope, and Kaggle with a live demo.
QwQ-32B-Preview: Alibaba's Qwen Reasoning Model with Deep Reflection Capabilities
Alibaba's Qwen team has released QwQ-32B-Preview, a 32-billion parameter model designed for deep reasoning across mathematics, code, and general knowledge. The model is positioned as a reasoning-focused system that emphasizes uncertainty and iterative questioning as core design principles. It is available on GitHub, Hugging Face, ModelScope, and via a demo interface.
Qwen2.5-Coder Series Open-Sourced: 32B Model Claims SOTA, Matches GPT-4o on Coding
Alibaba's Qwen team has open-sourced the Qwen2.5-Coder family of code-specialized language models, with the flagship 32B-Instruct variant claiming state-of-the-art performance among open-source code models and parity with GPT-4o on coding benchmarks. The release spans multiple model sizes, expanding on previously released smaller variants. The models are described as combining strong coding ability with general reasoning and mathematical skills.
Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries
A Hugging Face blog post surveys 16 open-source reinforcement learning libraries for LLM training, analyzing their architectural approaches to async and synchronous token generation pipelines. The piece distills practical lessons about throughput, scalability, and design trade-offs across the ecosystem. It serves as a comparative landscape analysis for practitioners building or choosing RL training infrastructure for language models.
Qwen2.5: Large-Scale Open-Source Foundation Model Family Release
Alibaba's Qwen team has released Qwen2.5, described as potentially the largest open-source model release in history, following three months of development after Qwen2. The release encompasses a family of foundation models with improvements in knowledge and reasoning capabilities. The announcement targets developers who have been building on Qwen2 and incorporates feedback from that community.
LeRobot v0.5.0: Scaling Every Dimension
Hugging Face released LeRobot v0.5.0, a major update to its open-source robotics learning library. The release focuses on scaling across multiple dimensions of the robotics ML pipeline. As a tier-2 source with no body content available, specific technical details of the update are not accessible from this item.