4GitHub Trending (AI/LLM filtered)·
Rapid-MLX: High-performance local AI inference engine for Apple Silicon
Rapid-MLX is an open-source Python project claiming 4.2x faster inference than Ollama on Apple Silicon, with 0.08s cached time-to-first-token and full tool-calling support via 17 parser implementations. It offers prompt caching, reasoning separation, cloud routing, and a drop-in OpenAI-compatible API, with stated compatibility with Claude Code, Cursor, and Aider. The project has accumulated 3,355 GitHub stars with modest daily growth (+18).
Related guides (3)
Related events (8)
omlx: LLM inference server with continuous batching and SSD caching for Apple Silicon
omlx is an open-source Python project providing an LLM inference server optimized for Apple Silicon, featuring continuous batching and SSD caching managed via a macOS menu bar interface. The project has accumulated nearly 16,000 GitHub stars with strong daily momentum. It targets local inference on Apple hardware, a growing niche as consumer-grade silicon becomes increasingly capable for running open-weights models.
mlx-lm: LLM inference library for Apple MLX framework trending on GitHub
mlx-lm is an open-source Python library for running LLMs using Apple's MLX framework, designed for Apple Silicon hardware. The repository has accumulated 5,817 stars with 43 new stars today, indicating steady community interest. It represents a key piece of the Apple-native ML inference ecosystem.
K-Search framework extended to translate CUDA kernel expertise to Apple Silicon MLX via AI-driven evolutionary search
Researchers at UC Berkeley extended K-Search, an evolutionary AI-driven kernel optimization framework, with a CUDA-to-MLX translation layer that adapts existing CUDA kernels to Apple Silicon rather than rebuilding from scratch. The system uses an LLM (Gemini 3.5 Pro Preview) to maintain a structured 'world model' decision tree and iteratively generate, compile, and benchmark candidate MLX kernels. Results show 0.97x performance relative to the native MLX Attention kernel and up to 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel. The approach is framed as generally applicable to any ecosystem where CUDA expertise is transferable, not just MLX.
WWDC 24: Running Mistral 7B with Core ML
This Hugging Face blog post covers running Mistral 7B on Apple devices using Core ML, likely demonstrated or announced around WWDC 2024. It addresses on-device inference of a 7B parameter open-weights model using Apple's ML framework. This represents a practical deployment pattern for running capable open-weights LLMs locally on Apple Silicon hardware.
Data Points: PrismML fits 27B model on iPhone; Cognition SWE-1.7, Nvidia Audex, Anthropic language-value study
A newsletter digest covers four notable AI developments: PrismML (a Caltech/Khosla spinout) compressed Alibaba's Qwen 27B model to under 4 GB via ternary/binary quantization for on-device iPhone inference; Cognition released SWE-1.7 (trained on Kimi K2.7), jumping from 9.4% to 42.3% on FrontierCode 1.1 Main with novel RL and infrastructure techniques; Nvidia introduced Audex, a 30B unified audio-text transformer trained on 157B audio tokens; and Anthropic published research showing Claude's expressed values shift measurably by language across 309,815 conversations. Each item represents a distinct technical development across on-device inference, coding agents, multimodal models, and model behavior analysis.
Simon Willison highlights Nativ: a local AI model runner for Mac
Simon Willison links to Nativ, a tool for running AI models locally on macOS. The post is a brief pointer from a respected AI practitioner, signaling community interest in local inference tooling for Apple hardware. No detailed technical analysis is provided in the body.
The Batch: Jalapeño inference chip, Fugu multi-agent system, Claude Tag, Robin bio-agent, and Getty-OpenAI deal
OpenAI and Broadcom announced Jalapeño, OpenAI's first custom inference chip, designed in nine months with AI-assisted design and showing better performance-per-watt than current accelerators; engineering samples are already running GPT-5.3-Codex-Spark with datacenter deployment planned by end of 2026. Sakana AI released Fugu, a multi-agent routing system that scored 73.7% on SWE-Bench Pro, outperforming Claude Opus 4.8 and GPT-5.5 while remaining below the inaccessible Fable 5. Additional items cover Anthropic's Claude Tag Slack integration for async team collaboration, Seedance 2.5 video model improvements, the Robin autonomous biology research agent that identified a novel drug candidate, and a Getty Images licensing partnership with OpenAI.
Releasing Swift Transformers: Run On-Device LLMs in Apple Devices
Hugging Face released Swift Transformers, a Swift library enabling on-device LLM inference on Apple hardware (iOS, macOS) via Core ML. The library provides a pipeline abstraction for text generation and supports models converted to Core ML format. This extends the Hugging Face ecosystem to Apple's native development environment, lowering the barrier for deploying LLMs on Apple Silicon devices.