4GitHub Trending (AI/LLM filtered)·
LitGPT: Lightning AI's multi-LLM pretraining, finetuning, and deployment framework
LitGPT is an open-source Python framework from Lightning AI supporting 20+ high-performance LLMs with recipes for pretraining, finetuning, and large-scale deployment. The repository has accumulated 13,563 GitHub stars with modest daily momentum (+39). It serves as a practitioner-oriented tooling layer for working with open-weights models across the full training and serving lifecycle.
Related guides (2)
Related events (8)
vLLM: High-Throughput LLM Inference and Serving Engine Trending on GitHub
vLLM is an open-source Python library providing high-throughput and memory-efficient inference and serving for large language models. The project has accumulated over 80,500 GitHub stars with 98 new stars today, indicating continued strong community interest. It is a widely adopted inference backend in the AI/ML ecosystem, supporting PagedAttention and various optimization techniques for LLM deployment.
LiteLLM AI gateway trending: 50K stars, unified interface for 100+ LLM APIs
LiteLLM is a Python SDK and proxy server providing a unified OpenAI-compatible interface to 100+ LLM APIs including Bedrock, Azure, OpenAI, VertexAI, Anthropic, and others. It includes cost tracking, guardrails, load balancing, and logging. The project is trending on GitHub with ~50K total stars and 141 new stars today, signaling continued strong adoption as an AI gateway layer.
Making LLMs lighter with AutoGPTQ and transformers
Hugging Face announces native integration of AutoGPTQ into the transformers library, enabling 4-bit quantized inference for large language models. The integration allows users to load and run GPTQ-quantized models directly through the standard transformers API with minimal code changes. This lowers the hardware barrier for deploying LLMs by significantly reducing VRAM requirements while maintaining competitive performance.
Accelerating LLM Inference with TGI on Intel Gaudi
Hugging Face's Text Generation Inference (TGI) framework has added a backend for Intel Gaudi accelerators, enabling LLM inference on Intel's AI hardware. The integration allows users to deploy large language models on Gaudi hardware using TGI's serving infrastructure. This expands the hardware ecosystem for LLM inference beyond NVIDIA GPUs, offering an alternative accelerator option for enterprise deployments.
MLIR-based compilation method for LLM inference on specialized hardware
Researchers present an MLIR-based compiler pipeline for deploying large language models on AI accelerators, using two dialect layers (TopOp for framework-agnostic graph representation and TpuOp for hardware-specific lowering). The method splits each Transformer layer into three static compilation stages (prefill, prefill_kv, decode) to handle the distinct computational profiles of prompt processing and autoregressive generation. The approach is implemented in the open-source TPU-MLIR compiler and LLM-TPU project, supporting Qwen, Llama, InternVL, and MiniCPM-V families with GPTQ, AWQ, and AutoRound quantization.
GLM-5.1 Open-Weights Model Targets Long-Running Agentic Tasks; Andrew Ng on Coding Agent Acceleration by Software Domain
Z.ai released GLM-5.1, an open-weights mixture-of-experts LLM (754B total / 40B active parameters) designed for sustained agentic coding tasks lasting up to eight hours, featuring iterative planning-execution-evaluation loops with thousands of tool calls. The model claims top open-weights performance on Artificial Analysis Intelligence Index and SWE-Bench Pro, available under MIT license via HuggingFace. The accompanying editorial by Andrew Ng offers a tiered framework for how much coding agents accelerate different software work categories—frontend most, then backend, infrastructure, and research least—with practical implications for team organization. A secondary item references data-center opposition and LLM helpfulness failure modes.
ktransformers: heterogeneous LLM inference/fine-tune optimization framework gains traction on GitHub
ktransformers is an open-source Python framework for heterogeneous LLM inference and fine-tuning optimizations, developed by kvcache-ai. The project has accumulated 18,236 GitHub stars with 328 added in a single day, indicating significant community interest. It targets KV-cache and mixed-hardware inference optimization, a key area for reducing LLM serving costs.
torchtune: PyTorch Native Post-Training Library for LLMs
Meta's PyTorch team introduces torchtune, a PyTorch-native library for post-training LLMs that emphasizes modularity, hackability, and direct access to underlying PyTorch components. The library supports fine-tuning, experimentation, and deployment-oriented workflows across distributed training settings. Benchmarked against popular frameworks Axolotl and Unsloth, torchtune demonstrates competitive performance and memory efficiency while maintaining flexibility for research iteration. The paper presents design principles, model builders, training recipes, and distributed training stack details.