mlx-lm: LLM inference library for Apple MLX framework trending on GitHub
mlx-lm is an open-source Python library for running LLMs using Apple's MLX framework, designed for Apple Silicon hardware. The repository has accumulated 5,817 stars with 43 new stars today, indicating steady community interest. It represents a key piece of the Apple-native ML inference ecosystem.
Related guides (1)
Related events (8)
omlx: LLM inference server with continuous batching and SSD caching for Apple Silicon
omlx is an open-source Python project providing an LLM inference server optimized for Apple Silicon, featuring continuous batching and SSD caching managed via a macOS menu bar interface. The project has accumulated nearly 16,000 GitHub stars with strong daily momentum. It targets local inference on Apple hardware, a growing niche as consumer-grade silicon becomes increasingly capable for running open-weights models.
vLLM: High-Throughput LLM Inference and Serving Engine Trending on GitHub
vLLM is an open-source Python library providing high-throughput and memory-efficient inference and serving for large language models. The project has accumulated over 80,500 GitHub stars with 98 new stars today, indicating continued strong community interest. It is a widely adopted inference backend in the AI/ML ecosystem, supporting PagedAttention and various optimization techniques for LLM deployment.
MLflow trending on GitHub as open-source AI engineering platform
MLflow, an open-source platform for managing AI/ML workflows, is trending on GitHub with 26,442 total stars and 22 new stars today. The project supports agents, LLMs, and traditional ML models, offering debugging, evaluation, monitoring, and optimization capabilities for production AI applications. It is a mature, widely-used tooling platform in the MLOps space.
Releasing Swift Transformers: Run On-Device LLMs in Apple Devices
Hugging Face released Swift Transformers, a Swift library enabling on-device LLM inference on Apple hardware (iOS, macOS) via Core ML. The library provides a pipeline abstraction for text generation and supports models converted to Core ML format. This extends the Hugging Face ecosystem to Apple's native development environment, lowering the barrier for deploying LLMs on Apple Silicon devices.
Langfuse: Open Source LLM Engineering Platform Trending on GitHub
Langfuse is an open-source LLM engineering platform providing observability, metrics, evaluations, prompt management, and dataset tooling. It integrates with OpenTelemetry, LangChain, OpenAI SDK, and LiteLLM. The project has accumulated 28,075 GitHub stars with 89 new stars today, indicating sustained community traction. Backed by Y Combinator (W23), it represents a notable entry in the LLM ops/tooling ecosystem.
LiteLLM AI gateway trending: 50K stars, unified interface for 100+ LLM APIs
LiteLLM is a Python SDK and proxy server providing a unified OpenAI-compatible interface to 100+ LLM APIs including Bedrock, Azure, OpenAI, VertexAI, Anthropic, and others. It includes cost tracking, guardrails, load balancing, and logging. The project is trending on GitHub with ~50K total stars and 141 new stars today, signaling continued strong adoption as an AI gateway layer.
Introducing AnyLanguageModel: One API for Local and Remote LLMs on Apple Platforms
Hugging Face has introduced AnyLanguageModel, a unified Swift API that abstracts over both local on-device LLMs and remote LLM endpoints on Apple platforms (iOS, macOS). The library aims to simplify developer integration by providing a single interface regardless of whether inference runs locally or via a cloud API. This is positioned as a tooling release targeting the Apple developer ecosystem for AI-powered app development.
WWDC 24: Running Mistral 7B with Core ML
This Hugging Face blog post covers running Mistral 7B on Apple devices using Core ML, likely demonstrated or announced around WWDC 2024. It addresses on-device inference of a 7B parameter open-weights model using Apple's ML framework. This represents a practical deployment pattern for running capable open-weights LLMs locally on Apple Silicon hardware.