Accelerate a World of LLMs on Hugging Face with NVIDIA NIM
NVIDIA NIM microservices are being integrated with Hugging Face to enable optimized inference deployment for a broad range of LLMs hosted on the Hub. The partnership allows developers to deploy Hugging Face models via NIM's containerized inference stack, leveraging NVIDIA's TensorRT-LLM and other optimizations. This expands the ecosystem of models accessible through NIM beyond NVIDIA's own catalog to the wider Hugging Face model repository.
Related guides (4)
Related events (8)
Serverless Inference with Hugging Face and NVIDIA NIM
Hugging Face and NVIDIA have partnered to offer serverless inference via NVIDIA NIM microservices on DGX Cloud infrastructure. The integration allows developers to run optimized model inference without managing GPU infrastructure, combining Hugging Face's model hub with NVIDIA's inference optimization stack. This represents an expansion of the existing Hugging Face–NVIDIA partnership into managed inference services.
Optimum-NVIDIA: One-Line LLM Inference Acceleration via TensorRT-LLM
Hugging Face's Optimum-NVIDIA integration wraps NVIDIA's TensorRT-LLM backend to enable high-performance LLM inference with minimal code changes. The library targets developers who want near-peak GPU throughput without manually configuring TensorRT-LLM pipelines. It positions as a bridge between the Hugging Face ecosystem and NVIDIA's optimized inference stack.
Accelerating Hugging Face Transformers with AWS Inferentia2
Hugging Face published a blog post detailing how to accelerate Transformer model inference using AWS Inferentia2, Amazon's second-generation ML inference chip. The post covers integration patterns between the Hugging Face ecosystem and the Neuron SDK for deploying models on Inferentia2 hardware. This represents a practical guide for enterprise and cloud-based inference deployment using dedicated AI accelerators.
NVIDIA Llama Nemotron Nano VLM Released on Hugging Face Hub
NVIDIA has released the Llama Nemotron Nano VLM on Hugging Face Hub, a compact vision-language model built on the Llama architecture. The model is part of NVIDIA's Nemotron family targeting efficient multimodal inference. This release makes the model accessible to the broader research and developer community through Hugging Face's model hosting infrastructure.
Deploy LLMs with Hugging Face Inference Endpoints
Hugging Face published a guide on deploying large language models using their Inference Endpoints service. The post covers how to set up scalable, production-ready LLM deployments with minimal infrastructure overhead. It targets developers looking to move from experimentation to hosted inference without managing raw compute.
Accelerating over 130,000 Hugging Face Models with ONNX Runtime
Hugging Face and Microsoft have integrated ONNX Runtime (ORT) to accelerate inference for over 130,000 models on the Hugging Face Hub. The integration enables optimized deployment across CPU and GPU hardware without requiring users to manually export or configure ONNX models. This represents a significant expansion of ORT's reach within the open-weights model ecosystem, lowering the barrier to production-grade inference optimization.
Falcon LLM Integrated into Hugging Face Ecosystem
Hugging Face announced the integration of the Falcon language models (Falcon-7B and Falcon-40B) into its ecosystem, including model hosting, inference APIs, and tooling support. Falcon, developed by the Technology Innovation Institute (TII), had recently topped the Open LLM Leaderboard at the time of release. The post covers usage patterns, fine-tuning guidance, and deployment options within the Hugging Face stack.
Bringing Serverless GPU Inference to Hugging Face Users via Cloudflare Workers AI
Hugging Face and Cloudflare have partnered to bring serverless GPU inference to Hugging Face users through Cloudflare Workers AI. The integration allows developers to run Hugging Face models on Cloudflare's global edge network without managing GPU infrastructure. This represents an expansion of serverless inference options for the Hugging Face ecosystem, lowering the barrier to deploying ML models at scale.