Generalizing an LLM from 8k to 1M Context using Qwen-Agent
Alibaba's Qwen team describes an agent built on Qwen2 (8k native context) that processes documents up to 1M tokens by decomposing retrieval and reasoning tasks, reportedly outperforming both RAG pipelines and native long-context models. The agent framework was also used to generate synthetic training data for fine-tuning new long-context Qwen models, creating a self-improvement loop. This positions agent-based context extension as a practical alternative to architectural long-context training.
Related guides (3)
Related events (8)
Qwen2.5-1M: Open-Source Models with 1M Token Context Window Released
Alibaba's Qwen team has released two open-source models, Qwen2.5-7B-Instruct-1M and Qwen2.5-14B-Instruct-1M, extending context length to 1 million tokens. This follows the earlier upgrade of the proprietary Qwen2.5-Turbo to 1M context two months prior. The release includes inference framework support for deployment, marking the first time Qwen's open-weight models have reached this context length.
Qwen2.5-Turbo Extends Context Length to 1M Tokens
Alibaba's Qwen team has released Qwen2.5-Turbo, extending the model's context window from 128K to 1 million tokens (approximately 1 million English words). The update includes optimizations for both model capabilities and inference performance at extreme context lengths. The model is available via API and through HuggingFace and ModelScope demos.
Qwen2 Model Family Released: Five Sizes, 128K Context, Multilingual
Alibaba's Qwen team has released Qwen2, an evolution from Qwen1.5, comprising five pretrained and instruction-tuned models ranging from 0.5B to 72B parameters, including a 57B mixture-of-experts variant (57B-A14B). The release highlights training on 27 additional languages beyond English and Chinese, significantly improved coding and mathematics performance, and extended context support up to 128K tokens for the 7B and 72B instruct variants. Benchmark results are claimed to be state-of-the-art across a large number of evaluations.
Introducing the Qwen Series: Overview of Alibaba's Open-Source LLM Journey
Alibaba's Qwen team published a retrospective introduction to the Qwen series of large language models, four months after the initial Qwen-7B open-source release. The post consolidates links to their paper, GitHub, Hugging Face, and ModelScope repositories, and outlines the team's objectives for the open-source LLM program. It serves as a canonical reference point for the Qwen model family's public positioning.
Qwen2.5-Max: Large-Scale MoE Model Release by Alibaba's Qwen Team
Alibaba's Qwen team announces Qwen2.5-Max, a large-scale Mixture-of-Experts language model. The post acknowledges that scaling insights for very large MoE models have been limited, citing DeepSeek V3's recent disclosures as a reference point. The model is positioned as a frontier-scale MoE system developed concurrently with ongoing Qwen2 research.
Alibaba's Qwen3.7-Max positions as top Chinese LLM with closed weights and agentic focus
Alibaba released Qwen3.7-Max, a closed-weights proprietary model targeting long-running agentic tasks like coding and scientific discovery, with a 1M-token context window and 208 tokens/second output speed. The model ranks fifth to seventh on the Artificial Analysis Intelligence Index, trailing leading U.S. models from OpenAI, Anthropic, and Google but claiming the lowest hallucination rate among frontier models tested—partly by declining to answer over half of prompts. Alibaba's training approach separates task, agentic harness, and verifier components to prevent overfitting to specific setups. The release continues Alibaba's strategic shift from open to closed weights for top-tier models, with leadership changes in the Qwen team suggesting a revenue-focused pivot.
Loong: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selection
Loong is a long document translation agent that uses a 3E memory module (Essence-Exemplar-Entity) to store structured historical context, replacing passive full-context attention with RL-optimized adaptive context selection. The agent learns its context retrieval policy via reinforcement learning on self-sampled reasoning trajectories. Evaluations show average gains of up to 13.0 points across three metrics in English↔Chinese, German, and French translation directions, with strong generalization and robustness to noise in ultra-long documents.
Qwen2.5-LLM: Alibaba releases open-weight language models from 0.5B to 72B
Alibaba's Qwen team releases the Qwen2.5 series of decoder-only dense language models, open-sourcing seven variants spanning 0.5B to 72B parameters. The release targets production use cases in the 10-30B range and mobile deployments at 3B scale. This represents a significant expansion of the open-weights frontier from a Tier 1 Chinese AI lab.