6arXiv cs.CL (Computation and Language)·
SLAI T-Rex: Full-parameter post-training of DeepSeek-V4 family on Ascend NPU SuperPOD achieves 34% MFU
Researchers present SLAI T-Rex, an end-to-end optimization framework for full-parameter post-training of trillion-parameter MoE models on Huawei Ascend NPU SuperPOD infrastructure, using the DeepSeek-V4 model family as the target workload. The system achieves 34.22% Model FLOPs Utilization, a 2.93x improvement over the open-source baseline, through hierarchical optimizations spanning model parallelism, communication orchestration, and kernel execution. Building on this infrastructure, the team develops a domain-specialized CPT and SFT pipeline for Operations Research tasks using DeepSeek-V4-Flash, producing a model that achieves 71.81% zero-shot Pass@1 on OR benchmarks, outperforming GPT-5.4-Mini by ~4 percentage points. The work is notable both as a non-GPU large-scale training system report and as a demonstration of domain specialization for complex mathematical reasoning.
Related guides (3)
Related events (8)
DeepSeek V4 Preview Release: 1.6T-param Pro and 284B Flash Models with 1M Context, Open-Sourced
DeepSeek has released DeepSeek-V4 as an open-weights preview, comprising two MoE variants: V4-Pro (1.6T total / 49B active parameters) and V4-Flash (284B total / 13B active parameters). Both models support 1M token context by default, enabled by a novel Token-wise compression and DeepSeek Sparse Attention (DSA) architecture. V4-Pro claims open-source SOTA on agentic coding benchmarks and world-class math/STEM/coding performance rivaling top closed-source models, while V4-Flash offers near-parity reasoning at lower cost and latency. The API is live today with OpenAI and Anthropic compatibility, and legacy model endpoints will be retired in July 2026.
Mach-Mind-4-Flash: 35B MoE agentic model matching 100B-class performance via post-training optimization
Mach-Mind-4-Flash is a 35B-parameter Mixture-of-Experts model with only 3B activated parameters that achieves performance comparable to 100B-class models through post-training techniques alone. The pipeline combines a unified RL/OPD training infrastructure with multi-teacher scheduling, parallel domain-specific RL experts fused via Multi-Teacher On-Policy Distillation (MOPD), and Hybrid Median-length Policy Optimization (HMPO) which compresses reasoning chains 19-46% with minimal accuracy loss. Benchmark results include 92.70 on AIME'26, 82.82 on IFBench, and 75.80 on BFCL-v4, claiming to lead or match models 10-30x its activated size at a fraction of inference cost. The work is notable for demonstrating that post-training optimization can close large gaps in activated parameter count for agentic tasks.
DeepSeek-V3: 671B MoE Open-Source Model with 3x Speed Improvement
DeepSeek releases V3, a 671B parameter Mixture-of-Experts model with 37B activated parameters, trained on 14.8T tokens. The model runs at 60 tokens/second (3x faster than V2) and is fully open-source with weights and paper released. API pricing is set at $0.27/M input tokens and $1.10/M output tokens starting February 8, positioning it as a low-cost frontier alternative. DeepSeek signals future multimodal capabilities in the ecosystem.
DeepSeek Releases V3.2-Exp with Sparse Attention Architecture and 50%+ API Price Cut
DeepSeek has released DeepSeek-V3.2-Exp, an experimental model built on V3.1-Terminus that introduces DeepSeek Sparse Attention (DSA), a fine-grained sparse attention mechanism designed to improve long-context performance and reduce compute costs during training and inference. Benchmarks indicate V3.2-Exp performs on par with V3.1-Terminus while achieving efficiency gains. The release is accompanied by a 50%+ API price reduction effective immediately, open-weights release on Hugging Face, a technical report, and GPU kernel code in TileLang and CUDA.
DeepSeek-R1 Release: Open-Source Reasoning Model on Par with OpenAI o1
DeepSeek has released DeepSeek-R1, a reasoning-focused large language model claiming performance parity with OpenAI o1 on math, code, and reasoning benchmarks. The model is fully open-source under the MIT License, including weights and outputs, enabling distillation and commercial use. Six distilled smaller models (up to 32B and 70B) are also released, with the 32B and 70B variants reportedly matching OpenAI o1-mini. API access is live at significantly lower pricing than comparable frontier models ($0.55/M input tokens, $2.19/M output tokens).
Nvidia Nemotron 3 Ultra: hybrid Mamba-transformer open-weights model targeting agentic workloads
Nvidia released Nemotron 3 Ultra, a 550B parameter (55B active) hybrid Mamba-transformer mixture-of-experts model with a 1M token context window, publishing weights, training data, and RL environments under an open license. The model ranks as the highest-scoring U.S. open-weights model on the Artificial Analysis Intelligence Index (47.7-48.2) and is approximately three times faster than comparable open-weights rivals, though it trails leading Chinese models like Kimi K2.6 and DeepSeek V4 Pro on intelligence benchmarks. Nvidia used a novel Multi-Teacher On-Policy Distillation approach with 10+ specialized teacher models and trained using NVFP4 quantization. The release is strategically motivated by Nvidia's interest in a healthy open-weights ecosystem that drives AI semiconductor adoption.
DeepSeek-V3.1 Release: Hybrid Think/Non-Think Model with Agent-Focused Upgrades
DeepSeek has released V3.1, a hybrid inference model supporting both thinking and non-thinking modes in a single model, positioned as their first step toward the agent era. The model features improved tool use and multi-step agent task performance, with benchmarks showing gains on SWE-bench and Terminal-Bench, and faster thinking efficiency compared to DeepSeek-R1-0528. The base model received 840B tokens of continued pretraining for long-context extension, a new tokenizer, and open-source weights are available on HuggingFace. API updates include 128K context for both modes, Anthropic API format compatibility, and strict function calling support in beta.
DeepSeek-V2.5: Merged Open-Source Model Combining General and Coding Capabilities
DeepSeek has released DeepSeek-V2.5, an open-source model that merges DeepSeek-V2-Chat-0628 and DeepSeek-Coder-V2-0724 into a single unified model. The release improves general conversational capabilities, coding performance, instruction-following, and writing tasks while also strengthening safety properties—raising the overall safety score from 74.4% to 82.6% and reducing safety spillover rate from 11.3% to 4.6%. The model is available via backward-compatible API endpoints (deepseek-chat and deepseek-coder) and on HuggingFace, retaining features like Function Calling, FIM completion, and JSON output. Benchmark results show improvements on HumanEval Python and LiveCodeBench, though SWE-verified performance remains an acknowledged weak area.