Entity · model
Qwen3
modelactive
qwen3-e6bbc535·37 events·first seen Aliases: Qwen 3, Qwen-3, Qwen3, Qwen3.6, Qwen3.5, Qwen3.7, Qwen 3.8
Merged from
Qwen 3, Qwen-3
Co-occurring entities
AlibabaGRPODeepSeek V4Hugging FaceAnthropicGemma 4ModelScopeQwen3-4BGoogleProximal Policy OptimizationOLMo-3Qwen2.5speculative decodingLlamaAmazon Web ServicesReinforcement LearningAlibaba QwenBeyond Scale and Generation: Understanding Language Model-based Entity MatchingVisual Contrastive Self-DistillationViRL39K
More like this (12)
Guides (1)
Recent events (37)
Controlled factorial study disentangles architecture, model variant, and scale effects in LLM-based entity matching
A new arXiv preprint presents a controlled factorial study of language model-based entity matching across three matcher architectures (bi-encoder, cross-encoder, generative), three model variants, and three model sizes from the Qwen3 family, totaling 1,215 fine-tuning runs on nine datasets. Key findings include: model variant (pretraining objective) is more important than scale for bi-encoders; cross-encoders consistently outperform bi-encoders but larger models narrow the gap; generative matchers only outperform cross-encoders under distribution shift; and larger models are more prone to shortcut learning. The study also evaluates cross-dataset transferability and computational cost, releasing all code and results.
Visual Contrastive Self-Distillation (VCSD) improves multimodal LLM training without external teachers
Researchers propose Visual Contrastive Self-Distillation (VCSD), a training method for vision-language models that creates an on-policy self-distillation signal purely through input conditioning — specifically by contrasting token distributions from an EMA teacher conditioned on the original image versus a content-erased control. The approach eliminates the need for external teachers, privileged answers, visual evidence signals, or reasoning traces. Evaluated on Qwen3-VL and Qwen3.5 models using the ViRL39K dataset, VCSD consistently outperforms matched on-policy self-distillation baselines, with aggregate benchmark gains of up to ~3.75 percentage points at the 8B scale.
Benchmark study reveals how linguistic framing of user beliefs shifts LLM context-following behavior
A new arXiv paper introduces a typology of 17 linguistically motivated expression-of-belief (EoB) types—spanning form, evidentiality, epistemic stance, and tone—to evaluate how phrasing affects whether LLMs defer to user-stated beliefs or their own prior knowledge. The authors benchmark 16 LLMs across Llama 3, Qwen3, and Gemma3 families at scales from 1B to 30B parameters, finding that larger and instruction-tuned models are systematically less context-following than smaller or base models. Specific linguistic framings (e.g., presuppositions, certainty markers) are identified as statistically more persuasive, with implications for prompt robustness and sycophancy research.
Commentary: Kimi K3, Qwen 3.8, and Anthropic's competitive position
A piece from Emerging Trajectories analyzes the competitive dynamics between Moonshot AI's Kimi K3, Alibaba's Qwen 3.8, and Anthropic's strategic position, framing the latter as potentially under pressure. The article surfaced on Hacker News with 248 points and 248 comments, indicating significant community engagement. The framing suggests concern about Anthropic's ability to maintain frontier status as Chinese labs release competitive models.
BIRD: Bootstrapped Iterative Self-Reasoning Distillation reduces LLM chain-of-thought length while improving accuracy
Researchers introduce BIRD (Bootstrapped Iterative Self-Reasoning Distillation), a two-stage method for compressing chain-of-thought reasoning in large language models without sacrificing accuracy. The approach first fine-tunes a model on brevity-instructed correct traces to warm-start the rollout distribution, then applies on-policy reverse-KL distillation against a concise self-teacher. On Qwen3-8B, BIRD improves MATH-500 accuracy from 86.2% to 92.0% while cutting average response length from 3,099 to 1,115 tokens, outperforming cold-start on-policy distillation baselines.
Alibaba Qwen releases Qwen 3.8 model
Alibaba's Qwen team announced Qwen 3.8, a new model in the Qwen 3 series. The announcement generated significant community engagement on Hacker News with 416 points and 314 comments. Details on capabilities and benchmarks are not available from this source snippet alone, but the community response suggests notable interest in the release.
Graded entity-familiarity probes in LLMs enable refusal steering and cross-language robustness
Researchers probe activations at the final prompt token across twelve instruction-tuned models (Bielik, PLLuM, Gemma-4, Qwen3 families) to assess whether LLMs encode graded familiarity with named entities before generating answers. Using a new dataset of 1,440 Polish entities across popularity deciles plus fabricated controls, they find familiarity probes reliably separate real from fabricated entities and track popularity in Polish-adapted models. A key finding is that a single one-dimensional familiarity direction injected at one layer in Gemma-4-12B can steer refusal rates monotonically from 0.24 to 1.00 (or 0.73 to 0.00), revealing a separation between representational familiarity and the policy that converts it into abstention behavior.
CARE-PPO: PPO-based RL framework for joint quantitative prediction and confidence estimation in LLMs
Researchers introduce CARE-PPO, a reinforcement learning fine-tuning framework that jointly trains LLMs for numerical prediction accuracy and calibrated confidence estimation. The approach repurposes the PPO critic as a confidence estimator at inference time, using a Confidence-Aligned Reward for Estimation derived from prediction error. Evaluated on healthcare and finance tasks with Qwen-3 4B and 8B models, CARE-PPO outperforms logit-based and verbalized confidence baselines and shows improved out-of-distribution generalization. The work addresses the hallucination and overconfidence problems that limit LLM deployment in high-stakes quantitative domains.
Auditing LLM-as-Judge reliability: judge upgrades are not interchangeable across model families
A new arXiv paper investigates measurement validity problems in LLM-as-judge evaluation, finding that swapping evaluator models changes scores even when candidate responses are fixed. Across four judgment datasets, the authors compare Qwen3 dense judges (1.7B–32B) and MiniMax M2/M2.7 API releases, finding that only the Qwen3 1.7B→4B upgrade yields robust adjacent gains while MiniMax adjacent releases do not. Stronger judges reduce but do not eliminate position and verbosity bias, and repeated-sample juries add little when errors are correlated. The paper argues for standardized reporting requirements including dataset slices, bias probes, error-dependence estimates, and protocol audit trails.
DeLS-Spec: Decoupled long-short context speculative decoding improves drafting efficiency
DeLS-Spec is a new speculative decoding method that combines a fixed block-parallel draft model (DFlash) as a long-context expert with a lightweight locally-trained short-context head, avoiding joint training with the target model. The approach introduces intra-block causal conditioning at low training cost and is modular across DFlash checkpoints. Experiments on Qwen3 models show consistent speedup and acceptance-length improvements over DFlash on math, code, and dialogue benchmarks.
Agon: Competitive cross-model RL uses rival models as implicit reasoning graders
Agon is a new reinforcement learning framework where two competing models grade each other implicitly by attempting the same problems in alternating roles — one drafts a solution, the other reads it while solving, and each is rewarded for out-solving the rival. This sidesteps the need for process labels or a reward model, and because both models are jointly optimized, each faces a progressively stronger opponent. On the hard split of DeepMath with Qwen3, Agon doubles GRPO's pass@1, roughly eight times the gain of an untrained Mixture-of-Agents baseline, with results replicating on competitive programming and across model families.
Scaling laws study finds LLM social simulation fidelity mostly improves with compute, with notable exceptions
A new arXiv preprint investigates whether scaling compute improves the fidelity of LLM-based social simulations across three domains: opinion modeling, behavioral simulation, and longitudinal forecasting. Using 85 Qwen3-architecture models trained under fixed-compute budgets from 10^18 to 10^20 FLOPs, plus 35 larger open-weight models up to 70B parameters, the authors find strong scaling in most settings. However, longitudinal forecasting, underrepresented populations, and specific cognitive bias calibration tasks (e.g., risk aversion) scale poorly, with fine-tuning failing to close gaps from 0.5B to 8B parameters. The work provides empirical grounding for where scaling will and will not suffice for social simulation research.
LOCOS: Logit-Contribution Scoring identifies non-literal retrieval heads in long-context LLMs
A new arXiv preprint introduces Logit-Contribution Scoring (LOCOS), a method for identifying attention heads responsible for non-literal retrieval in long-context LLMs — cases where models synthesize answers from meaning rather than copying tokens verbatim. Existing detectors fail at this task because they rely on a literal-copy criterion that misses the output-value (OV) circuit mechanism. Evaluated across Qwen3, Gemma-3, and OLMo-3.1, LOCOS outperforms prior attention-based detectors on the NoLiMa benchmark, with ablation of 50 heads on Qwen3-8B collapsing ROUGE-L from 0.401 to 0.000 while the best baseline retains 0.292. The identified heads are retrieval-specific, leaving parametric recall and arithmetic reasoning unaffected.
Single transformer layer training can match full-parameter RL post-training in LLMs
A new arXiv paper challenges the assumption that all transformer layers contribute equally during RL post-training, finding that training a single layer can recover most or all of the gains from full-parameter RL. The authors introduce a 'layer contribution' metric and evaluate across seven models from the Qwen2.5 and Qwen3 families, three RL algorithms (GRPO, GiGPO, Dr. GRPO), and tasks including math reasoning, code, and agentic decision-making. A consistent structural pattern emerges: high-contribution layers concentrate in the middle of the transformer stack, and this ranking is stable across datasets, tasks, and algorithms.
Theory of acceptance criteria in speculative decoding for greedy and relaxed regimes
A new arXiv preprint develops a theoretical framework for speculative decoding acceptance criteria beyond the standard stochastic, distribution-preserving setting. The authors characterize rejection regions for greedy decoding, additive/multiplicative relaxed acceptance, top-m criteria, and entropy-thresholded acceptance in terms of KL divergence and margin-based bounds. The framework is extended to greedy tree decoding and validated empirically on Qwen3 models, showing relaxed and tree-based criteria substantially expand certified acceptance regions. The work fills a gap between existing theory and practical inference systems that use non-exact acceptance rules.
LLM embedding spaces partially recover expert-defined symptom structure in mental health language
A new arXiv preprint investigates whether LLM embedding geometry aligns with expert-defined symptom structure in mental health language, using 28 Reddit communities as a testbed. The authors compare pretrained and fine-tuned Qwen3 embeddings (0.6B and 4B) against an expert symptom matrix via representational similarity analysis, with controls for affective, stylistic, and topic confounds. Results show measurable but level-dependent alignment: fine-tuning strengthens it at fine-grained category levels, and larger scale improves both zero-shot alignment and fine-tuning gains. The paper argues that classification accuracy alone is insufficient to validate embedding geometry against domain knowledge.
HydraHead: Head-level hybridization of full and linear attention for long-context efficiency
Researchers introduce HydraHead, an architecture that hybridizes Full Attention (FA) and Linear Attention (LA) at the head level rather than the conventional layer level, motivated by interpretability findings showing functional heterogeneity among heads within the same layer. An interpretability-driven selection strategy preserves FA only for retrieval-critical heads, achieving a 7:1 LA-to-FA ratio while matching the long-context performance of a 3:1 layer-wise hybrid. Trained on only 15B tokens, HydraHead achieves over 69% improvement over the baseline at 512K context length, approaching Qwen3.5's performance despite that model having a native 256K context window. The work suggests head-level hybridization is a significantly underexplored and high-potential design axis for efficient long-context models.
LLMs predict dementia and depression severity from clinical interview transcripts in zero-shot and feature-extraction settings
Researchers evaluate three open-weights LLMs (Mistral 3.1, DeepHermes, Qwen3) for predicting dementia and depression severity from speech transcripts of 154 German-speaking patients in standardized clinical interviews. The study introduces a new observer-based Global Depression Scale (GDS-D) and tests both zero-shot prediction and LLM-based feature extraction for Support Vector Regression. Zero-shot performs well for depression (MAE 0.60), while structured feature extraction reduces dementia assessment error by up to 35%; pause-enriched automatic transcripts match human transcription quality, suggesting viable fully-automated screening pipelines.
ZPPO: Teacher-in-prompt training method outperforms distillation and GRPO for small vision-language models
Researchers introduce Zone of Proximal Policy Optimization (ZPPO), a training method inspired by Vygotsky's zone of proximal development that embeds teacher guidance in prompts rather than policy gradients or logit imitation. On hard questions where student rollouts fail, ZPPO constructs Binary Candidate-included Questions (BCQ) and Negative Candidate-included Questions (NCQ) to help the student discriminate correct from incorrect responses, with a replay buffer that recirculates hard questions until mastered. Evaluated on the Qwen3 family (0.8B–9B) with a 27B teacher across a 31-benchmark suite covering VLM, LLM, and video tasks, ZPPO outperforms both distillation and GRPO baselines, with the largest gains at the smallest model scale. The method addresses a known failure mode of RL training where zero-reward rollouts produce no gradient signal.
Expert Tying reduces MoE LLM memory footprint by ~2x with minimal quality loss
Researchers introduce Expert Tying, an architectural modification for Mixture-of-Experts LLMs that shares expert parameters across consecutive transformer layers while keeping routing and attention layer-independent. Evaluated on OLMoE, Qwen3, and DeepSeek-style MoE architectures, the method achieves nearly 2x memory reduction with negligible perplexity or downstream quality degradation. The approach exploits parameter redundancy in MoE pathways to improve the compute-to-memory trade-off for training and inference.
ms-swift: ModelScope framework for fine-tuning 600+ LLMs and 300+ MLLMs
ms-swift is an open-source Python framework from ModelScope supporting PEFT and full-parameter fine-tuning methods (CPT, SFT, DPO, GRPO) across 600+ LLMs and 300+ multimodal LLMs, including Qwen3, DeepSeek, Llama4, and others. The project has accumulated 14,487 GitHub stars and was accepted at AAAI 2025. It serves as a broad-coverage training harness for the current generation of open-weights frontier models.
Bebop: MTP with rejection sampling and TV loss achieves 1.8x RL training speedup
Researchers introduce Bebop, a framework for integrating Multi-Token Prediction (MTP) into large-scale RL training pipelines for LLMs. The work identifies that MTP acceptance rates degrade during RL due to entropy fluctuations, and proposes probabilistic rejection sampling plus a novel end-to-end Total Variation (TV) loss that directly optimizes multi-step acceptance rates, achieving up to 95% acceptance rates and 25% extra inference throughput gains. Applied to Qwen3.5, Qwen3.6, and Qwen3.7 models, the method yields up to 1.8x end-to-end acceleration in async RL training. The approach eliminates the need for costly online MTP updating by using pre-RL MTP training with the proposed objectives.
DeepSeek releases R1-0528-Qwen3-8B distilled reasoning model on Hugging Face
DeepSeek released DeepSeek-R1-0528-Qwen3-8B, an 8B parameter text-generation model on Hugging Face, combining the R1-0528 reasoning capabilities with a Qwen3 base. The model has accumulated over 306K downloads and 1K likes shortly after release, indicating strong community uptake. This appears to be a distilled version of the R1-0528 reasoning model targeting smaller-scale deployment.
Study finds thinking mode in LRMs shifts instruction-following errors by constraint type rather than uniformly degrading performance
A new arXiv paper investigates how enabling built-in chain-of-thought reasoning ('Thinking ON/OFF') in Qwen3 and Hunyuan models affects instruction following on IFEval. Aggregate pass-rate changes are small but 10-20% of prompts switch outcomes, with 'Planning' constraints (global counting, structure) improving under thinking while 'Precision' constraints (exact local form) consistently worsen. Activation patching and trace-relevance analyses reveal an execution gap: thinking traces engage with Planning constraints but fail to translate that engagement into compliance, while Precision failures are more mechanistically recoverable. The findings have practical implications for when to enable reasoning modes in instruction-following applications.
VaSE: Value-Aware Stochastic KV Cache Eviction improves reasoning model efficiency
A new arXiv preprint introduces Value-aware Stochastic KV Cache Eviction (VaSE), a training-free method for compressing KV caches in long-chain-of-thought reasoning models. The authors identify two key failure modes in prior eviction approaches — catastrophic repetition loops caused by evicting high-magnitude value states, and low cache diversity — and address both with targeted protections and stochastic eviction. On six reasoning tasks with Qwen3 models at 4x compression, VaSE outperforms the current best selection-based sparse attention method and exceeds the strongest eviction baseline by over 4%, while supporting FlashAttention2 and maintaining a static memory footprint.
The Batch Issue 345: Iranian Drone Attacks on AWS Data Centers, Qwen3.5, DeepSeek-Huawei, and AI Job Insecurity
Andrew Ng's weekly newsletter covers several significant AI-adjacent developments: Iranian drones struck at least three Amazon Web Services data centers in Bahrain and the UAE, disrupting cloud services and raising concerns given U.S. military use of AWS to run Anthropic Claude; the issue also previews Qwen3.5 model releases across multiple sizes and DeepSeek's reported moves involving Huawei hardware. Ng also addresses widespread job insecurity across skill levels amid rapid AI advancement, citing geopolitical risks including the Iran war, Taiwan uncertainty, and rare-earth metal supply chains as compounding factors.
Alibaba releases Qwen3.5 open-weights vision-language model family with MoE architecture across eight sizes
Alibaba released the Qwen3.5 family of eight open-weights vision-language models ranging from 0.8B to 397B parameters, built on a mixture-of-experts architecture with mixed attention and Gated DeltaNet layers. The flagship Qwen3.5-397B-A17B outperforms GPT-5.2, Claude 4.5 Opus, and Gemini-3 Pro on 28 of 44 vision benchmarks, while the 9B model surpasses OpenAI's gpt-oss-120B on most language tasks. Open weights are available under Apache 2.0, with hosted agentic variants (Qwen3.5-Plus, Qwen3.5-Flash) available via Alibaba Cloud. The release is notable for strong small-model efficiency and comes amid reported team departures following the Qwen3 rollout.
Data Points: OpenAI and Microsoft sever their exclusive relationship
This edition of The Batch covers several major AI industry developments: OpenAI has revised its partnership with Microsoft, ending exclusivity while retaining Microsoft as primary cloud partner through 2032 and gaining freedom to deploy on AWS and Google Cloud. DeepSeek released V4 model weights featuring 1M-token context and Huawei Ascend chip optimization, though it trails leading open and closed models on aggregate benchmarks. Google and Amazon are deepening investments in Anthropic with up to $40B and $25B respectively in funding-for-compute deals, and an agentic AI system autonomously designed a functional RISC-V CPU from a 219-word spec in 12 hours.
SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Tasks
SCOPE is a data-free self-play framework for training language models on open-ended tasks without external supervision or frontier-model judges. It co-evolves two policies—a Challenger that generates document-grounded tasks and a Solver that answers via multi-turn retrieval—using a frozen copy of the initial model as a self-judge that writes task-specific rubrics. Across three 7-8B models (Qwen2.5, Qwen3, OLMo-3), SCOPE achieves up to +10.4 points on eight open-ended benchmarks and +13.8 points on seven held-out short-form QA benchmarks, matching or exceeding GRPO trained on ~9K curated prompts. Ablations identify rubric generation quality as the primary bottleneck for self-judging.
OpenPipe ART: Agent Reinforcement Trainer for Multi-Step Agents via GRPO
OpenPipe has released ART (Agent Reinforcement Trainer), an open-source Python library for training multi-step agents on real-world tasks using GRPO (Group Relative Policy Optimization). The framework supports multiple model families including Qwen3, GPT-OSS, and Llama. With nearly 10k GitHub stars and 66 gained today, it is gaining notable community traction as a practical RL fine-tuning tool for agentic workflows.
The 4 Things Qwen-3's Chat Template Teaches Us
A Hugging Face blog post performs a deep dive into the chat template design of Qwen-3, examining the technical choices made in its prompt formatting and conversation structure. The analysis surfaces lessons about how chat templates encode model behavior, reasoning modes, and tool-use conventions. As a tier-2 commentary piece, it provides practical implementation guidance for developers integrating Qwen-3 into applications.
EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
EnvFactory is a fully automated framework for training tool-use LLM agents via Agentic Reinforcement Learning, addressing two key bottlenecks: scalable execution environments and realistic multi-turn training data. It autonomously constructs stateful, executable tool environments from authentic resources and synthesizes natural trajectories with implicit human intents via topology-aware sampling. Using only 85 verified environments across 7 domains, it generates 2,575 SFT and RL trajectories and improves Qwen3-series models by up to +15% on BFCLv3, +8.6% on MCP-Atlas, and +6% on conversational benchmarks, outperforming prior approaches that use 5x more environments.
Unsloth: Web UI and Library for Efficient Fine-tuning of Open Models
Unsloth is an open-source Python library and web UI (Unsloth Studio) for efficient fine-tuning and local inference of open-weight models including Gemma 4, Qwen3, DeepSeek, and GPT-OSS variants. The project has accumulated over 64,000 GitHub stars with continued daily growth (+139 today), indicating strong community adoption. It targets practitioners who want to train and run large models locally with reduced memory and compute requirements.
Qwen 3.7 Preview Announced by Alibaba
Alibaba's Qwen team has announced a preview of Qwen 3.7, the next iteration in their Qwen 3 model series. The announcement appeared on Twitter/X and generated notable community discussion on Hacker News with 179 points and 67 comments. Specific capability details and model specifications are not available from this source alone.
Qwen3 Embedding: State-of-the-Art Text Embedding and Reranking Models Released
Alibaba's Qwen team has released the Qwen3 Embedding series, a set of open-weights text embedding and reranking models built on the Qwen3 foundation model. The models are designed for retrieval and reranking tasks and claim state-of-the-art performance across multiple benchmarks. They are released under the Apache 2.0 license and are available on Hugging Face and ModelScope.
Qwen-MT Turbo: Alibaba Releases Specialized Translation Model Supporting 92 Languages
Alibaba's Qwen team has released qwen-mt-turbo, a specialized machine translation model built on Qwen3 and trained on trillions of multilingual and translation tokens. The model supports 92 languages and dialects covering over 95% of the global population. It incorporates reinforcement learning techniques to improve translation accuracy and linguistic fluency, and is available via the Qwen API.
Qwen3Guard: Real-time Safety Guardrail Model for Token Stream Classification
Alibaba's Qwen team has released Qwen3Guard, the first dedicated safety guardrail model in the Qwen family, built on Qwen3 foundation models and fine-tuned for safety classification. The model performs real-time safety detection on both prompts and responses, providing risk levels and categorized classifications for content moderation. Qwen3Guard claims state-of-the-art performance on major safety benchmarks across English, Chinese, and multilingual settings.