Entity · model
Qwen3.5-0.8B
modelactive
qwen3-5-0-8b-509cb72f·6 events·first seen Aliases: Qwen3.5-0.8B, Qwen3-0.6B, Qwen3-0.6B-Instruct
Co-occurring entities
More like this (12)
Recent events (6)
Relay-OPD addresses prefix failure in on-policy distillation via teacher-student trajectory handoff
Researchers introduce Relay On-Policy Distillation (Relay-OPD), a training method that addresses 'prefix failure' in on-policy knowledge distillation, where student models compound early reasoning errors throughout a trajectory. The approach detects divergence points where teacher and student continuations asymmetrically diverge, then briefly hands generation to the teacher to produce a corrective 'relay leg' before the student resumes. Evaluated on eight mathematical reasoning benchmarks using Qwen3-4B-Instruct-2507 as teacher and Qwen3-0.6B/1.7B as students, Relay-OPD outperforms standard OPD by +5.73% and the strongest baseline FastOPD by +1.49% on average for the 1.7B model, while also reducing training trajectory length by over 50%.
SCOPE-RL framework densifies reward signals in RLVR to improve reasoning accuracy and token efficiency
SCOPE-RL introduces a two-stage reinforcement learning framework that addresses sparse reward limitations in RLVR by adding prefix-decomposed sub-question rewards before success and correctness-gated process-shape rewards after success. Applied to Qwen3-8B-Instruct on math reasoning datasets, the method improves average accuracy by up to 11.2 percentage points and reduces reasoning tokens by up to 27.1% over outcome-only GRPO. The gains generalize across GSPO and a smaller Qwen3-0.6B model, suggesting reward-signal densification is broadly complementary to existing RLVR advances. Code and data are publicly released.
TF-Engram: Train-free phrase-level memory injection for LLMs using SSD-backed storage
TF-Engram is a train-free memory system for LLMs that constructs phrase-specific semantic memory offline from external corpora and stores it across a GPU–DRAM–SSD hierarchy, avoiding the hash-collision problems of prior engram-style approaches. The system uses Early-Exit Guided Predictive Prefetching to hide external-memory latency during autoregressive decoding. Evaluated on Qwen3-0.6B, it improves average downstream score from 57.6 to 59.4, outperforming both the frozen backbone and a parameter-matched LoRA baseline, while substantially reducing GPU memory demand.
Program-as-Weights: compiling natural-language specs into compact neural adapters for local execution
Researchers introduce 'fuzzy-function programming' and its instantiation Program-as-Weights (PAW), a paradigm where a 4B compiler model converts natural-language function specifications into parameter-efficient adapters for a frozen lightweight interpreter. A 0.6B Qwen3 interpreter running PAW programs matches the performance of direct prompting with Qwen3-32B while using ~1/50th the inference memory and running at 30 tokens/s on a MacBook M3. The approach reframes large foundation models as one-time 'tool builders' rather than per-input solvers, targeting tasks like log triage, JSON repair, and intent-based ranking that resist rule-based implementation. The authors also release FuzzyBench, a 10M-example training dataset.
BitNet Text Embeddings: Ternary-weight LLM encoders with multi-precision output vectors
BITEMBED is a new framework that converts pretrained LLM backbones (tested on Qwen3-0.6B and Gemma3-270M) into BitNet-style text embedding encoders using ternary weights, quantized activations, and lightweight normalization refinement. The system applies continual contrastive pre-training and supervised fine-tuning with similarity-distribution and attention-relation distillation from a full-precision teacher. Evaluated on MMTEB (English v2), BITEMBED achieves performance largely comparable to full-precision teacher embedders while supporting flexible output embedding precisions to trade off storage cost. The work targets the dual bottleneck of inference compute and vector index storage in large-scale retrieval systems.
Qwen releases Qwen3.5-0.8B multimodal model on Hugging Face
Alibaba's Qwen team released Qwen3.5-0.8B, a small-scale image-text-to-text model, on Hugging Face. The model supports conversational use and is compatible with Azure deployment endpoints. With over 2.7 million downloads and 562 likes, it has seen substantial community uptake for a sub-1B parameter multimodal model.