6Berkeley AI Research (BAIR) Blog·
K-Search framework extended to translate CUDA kernel expertise to Apple Silicon MLX via AI-driven evolutionary search
Researchers at UC Berkeley extended K-Search, an evolutionary AI-driven kernel optimization framework, with a CUDA-to-MLX translation layer that adapts existing CUDA kernels to Apple Silicon rather than rebuilding from scratch. The system uses an LLM (Gemini 3.5 Pro Preview) to maintain a structured 'world model' decision tree and iteratively generate, compile, and benchmark candidate MLX kernels. Results show 0.97x performance relative to the native MLX Attention kernel and up to 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel. The approach is framed as generally applicable to any ecosystem where CUDA expertise is transferable, not just MLX.
Related guides (3)
Related events (8)
Rapid-MLX: High-performance local AI inference engine for Apple Silicon
Rapid-MLX is an open-source Python project claiming 4.2x faster inference than Ollama on Apple Silicon, with 0.08s cached time-to-first-token and full tool-calling support via 17 parser implementations. It offers prompt caching, reasoning separation, cloud routing, and a drop-in OpenAI-compatible API, with stated compatibility with Claude Code, Cursor, and Aider. The project has accumulated 3,355 GitHub stars with modest daily growth (+18).
mlx-lm: LLM inference library for Apple MLX framework trending on GitHub
mlx-lm is an open-source Python library for running LLMs using Apple's MLX framework, designed for Apple Silicon hardware. The repository has accumulated 5,817 stars with 43 new stars today, indicating steady community interest. It represents a key piece of the Apple-native ML inference ecosystem.
Custom CUDA Kernels for All from Codex and Claude
A Hugging Face blog post describes using AI coding agents (Codex and Claude) to automatically generate custom CUDA kernels, lowering the barrier to GPU kernel development. The piece demonstrates agent-assisted GPU programming as a practical workflow for ML practitioners. This represents a concrete application of AI coding tools to the specialized domain of CUDA/GPU optimization.
omlx: LLM inference server with continuous batching and SSD caching for Apple Silicon
omlx is an open-source Python project providing an LLM inference server optimized for Apple Silicon, featuring continuous batching and SSD caching managed via a macOS menu bar interface. The project has accumulated nearly 16,000 GitHub stars with strong daily momentum. It targets local inference on Apple hardware, a growing niche as consumer-grade silicon becomes increasingly capable for running open-weights models.
Using Stable Diffusion with Core ML on Apple Silicon
Hugging Face published a guide on running Stable Diffusion models via Apple's Core ML framework on Apple Silicon hardware. The post covers converting diffusion model weights to Core ML format and integrating them into the Diffusers library for on-device inference. This represents an early effort to enable efficient local image generation on consumer Apple hardware without requiring cloud GPU resources.
From Zero to GPU: A Guide to Building and Scaling Production-Ready CUDA Kernels
Hugging Face published a guide on building and scaling production-ready CUDA kernels, covering the full workflow from development to deployment. The post targets ML engineers who need to write custom GPU kernels for inference optimization and production workloads. It addresses practical concerns around kernel compilation, testing, and integration with existing ML frameworks.
ktransformers: heterogeneous LLM inference/fine-tune optimization framework gains traction on GitHub
ktransformers is an open-source Python framework for heterogeneous LLM inference and fine-tuning optimizations, developed by kvcache-ai. The project has accumulated 18,236 GitHub stars with 328 added in a single day, indicating significant community interest. It targets KV-cache and mixed-hardware inference optimization, a key area for reducing LLM serving costs.
Creating Custom Kernels for the AMD MI300
A Hugging Face blog post details the process of writing custom GPU kernels targeting the AMD MI300 accelerator. The post covers practical techniques for optimizing AI workloads on AMD hardware, contributing to the growing ecosystem of non-NVIDIA GPU support for ML inference and training. This is relevant to the broader trend of diversifying AI infrastructure beyond CUDA-dominant workflows.