Hugging Face blog: Profiling PyTorch nn.Linear toward a fused MLP implementation
A Hugging Face blog post (Part 2 of a profiling series) walks through optimizing PyTorch's nn.Linear layers toward a fused MLP kernel. The post covers profiling methodology and kernel fusion techniques relevant to inference and training efficiency. This is a practical deep-dive into low-level PyTorch optimization for ML practitioners.
Related guides (3)
Related events (8)
How Hugging Face Accelerate Runs Very Large Models Thanks to PyTorch
This Hugging Face blog post explains the technical mechanisms behind the Accelerate library for running large models that exceed single-GPU memory, leveraging PyTorch features such as device maps, CPU/disk offloading, and sharded checkpoints. It describes how models can be distributed across multiple GPUs, CPU RAM, and disk storage transparently. The post serves as both documentation and a technical explainer for practitioners working with large-scale inference and deployment.
Improving Hugging Face Training Efficiency Through Packing with Flash Attention 2
Hugging Face published a blog post describing a technique for improving training efficiency by packing multiple short sequences into a single batch using Flash Attention 2. The approach reduces padding waste and improves GPU utilization during LLM fine-tuning. This is a practical infrastructure optimization relevant to practitioners training models on datasets with variable-length sequences.
Optimization story: Bloom inference
This Hugging Face blog post documents practical inference optimization techniques applied to the BLOOM large language model. It covers strategies for reducing latency and memory footprint during deployment, likely including quantization, tensor parallelism, and batching approaches. The post serves as a technical case study for serving very large open-weights models efficiently.
Hugging Face on PyTorch / XLA TPUs
This Hugging Face blog post covers the integration of Hugging Face Transformers with PyTorch/XLA for training on Google TPUs. It describes how users can leverage TPU hardware through the XLA compiler backend to accelerate transformer model training. The post serves as a technical guide for the ecosystem connecting Hugging Face's model library with Google's TPU infrastructure.
Accelerating PyTorch Transformers with Intel Sapphire Rapids - Part 2
This Hugging Face blog post covers inference optimization techniques for PyTorch Transformer models on Intel Sapphire Rapids (4th Gen Xeon) CPUs. It likely demonstrates performance gains using hardware-specific features such as AMX (Advanced Matrix Extensions) and BF16 support. The post is part of a series focused on making transformer inference more efficient on Intel server hardware without requiring GPU acceleration.
Accelerating Hugging Face Transformers with AWS Inferentia2
Hugging Face published a blog post detailing how to accelerate Transformer model inference using AWS Inferentia2, Amazon's second-generation ML inference chip. The post covers integration patterns between the Hugging Face ecosystem and the Neuron SDK for deploying models on Inferentia2 hardware. This represents a practical guide for enterprise and cloud-based inference deployment using dedicated AI accelerators.
How Hugging Face Sped Up Transformer Inference 100x for API Customers
Hugging Face describes engineering optimizations that achieved up to 100x speedups in transformer inference for their hosted API customers. The post covers techniques applied to accelerate model serving at scale. This is a 2021 article documenting early inference optimization work at Hugging Face's inference API product.
Optimizing Stable Diffusion for Intel CPUs with NNCF and Hugging Face Optimum
This Hugging Face blog post details techniques for optimizing Stable Diffusion inference on Intel CPUs using Neural Network Compression Framework (NNCF) and the Optimum library. The workflow covers quantization and other compression methods to reduce latency and memory footprint on CPU hardware. This is relevant to the inference-economics and enterprise-deployment threads as it addresses running diffusion models without dedicated GPU hardware.