4arXiv cs.CL (Computation and Language)·
ELMOD: Efficient 2.7B German-first language model optimized for on-device inference
Researchers present ELMOD, a 2.7B-parameter German language model trained on 55k H100 GPU hours using exclusively public data, targeting resource-constrained on-device deployment. The work introduces German-specific data preprocessing pipelines handling morphological variation, compounding, and orthographic conventions, plus a quality filtering and rephrasing step to improve instructional data quality. ELMOD claims state-of-the-art performance in the sub-3B size class for German, reportedly matching 7B-parameter models on German benchmarks.
Related guides (2)
Related events (8)
MobileMoE: Scaling Mixture-of-Experts for Sub-Billion Parameter On-Device Deployment
MobileMoE introduces a family of on-device MoE language models with 0.3–0.9B active parameters and 1.3–5.3B total parameters, targeting mobile deployment under memory and compute constraints. The authors derive an on-device MoE scaling law identifying a sweet spot of moderate sparsity with fine-grained and shared experts, then train models through a four-stage recipe including quantization-aware training on open-source data. Across 14 benchmarks, MobileMoE matches or exceeds leading dense on-device LLMs with 2–4× fewer inference FLOPs, and delivers 1.8–3.8× faster prefill and 2.2–3.4× faster decode than dense baselines on commodity smartphones at comparable INT4 memory.
Soofi S 30B-A3B: Sovereign German-English MoE Hybrid Mamba Transformer Released Open-Source
Researchers present Soofi S 30B-A3B, a Mixture-of-Experts hybrid Mamba Transformer foundation model for German and English, activating only 3B of 30B parameters per token with near-constant inference cache for long-context efficiency. Pretrained on ~27 trillion tokens with up-weighted German data, it claims top aggregate scores among fully open models in both languages, outperforming OLMo 3 32B and Apertus 70B, and surpasses all European sovereign baselines tested. The model was built on Deutsche Telekom's sovereign HPC infrastructure in Munich and will be released with weights, checkpoints, hyperparameters, training code, and full data accounting under permissive terms.
Qwen2.5-LLM: Alibaba releases open-weight language models from 0.5B to 72B
Alibaba's Qwen team releases the Qwen2.5 series of decoder-only dense language models, open-sourcing seven variants spanning 0.5B to 72B parameters. The release targets production use cases in the 10-30B range and mobile deployments at 3B scale. This represents a significant expansion of the open-weights frontier from a Tier 1 Chinese AI lab.
Optimizing your LLM in production
A Hugging Face blog post covering practical techniques for optimizing large language models in production environments. The post likely addresses inference efficiency methods such as quantization, batching, caching, and hardware utilization strategies. It serves as a practitioner-oriented guide for deploying LLMs at scale.
Fast Inference on Large Language Models: BLOOMZ on Habana Gaudi2 Accelerator
This Hugging Face blog post covers deploying BLOOMZ, a large multilingual language model, on Intel's Habana Gaudi2 accelerator for inference. It benchmarks throughput and latency performance on Gaudi2 as an alternative to GPU-based inference. The post is part of ongoing work to demonstrate non-NVIDIA hardware options for large model deployment.
Introducing Gemma 3 270M: The compact model for hyper-efficient AI
Google DeepMind has released Gemma 3 270M, a 270-million parameter compact language model added to the Gemma 3 family. The model is positioned as a highly specialized, hyper-efficient tool for resource-constrained deployments. This extends the Gemma 3 lineup into the sub-billion parameter range, targeting edge and on-device use cases.
SmolLM: Hugging Face Releases Blazingly Fast Small Language Models
Hugging Face introduces SmolLM, a family of small language models designed for on-device and edge deployment with high speed and competitive performance. The models are positioned as efficient alternatives for resource-constrained environments. The release includes model weights and associated tooling on the Hugging Face Hub.
Mistral AI Releases Ministral 3B and 8B Edge Models
Mistral AI has introduced two new small language models, Ministral 3B and Ministral 8B, targeting on-device and edge computing use cases. Both models support up to 128k context length and claim state-of-the-art performance in the sub-10B parameter category, outperforming comparable models from Google and Meta on internal benchmarks. Ministral 8B features an interleaved sliding-window attention mechanism for memory-efficient inference and is priced at $0.1/M tokens via API, while Ministral 3B is priced at $0.04/M tokens. Weights for Ministral 8B Instruct are available for research use, with commercial licensing available on request.