4Hugging Face Blog·
Hugging Face blog: profiling attention mechanisms in PyTorch (Part 3)
A Hugging Face blog post (Part 3 of a series) covers profiling attention operations in PyTorch, targeting practitioners who want to understand and optimize attention layer performance. The post is focused on inference/training infrastructure tooling relevant to LLM workloads. It is a practical technical guide rather than a novel research finding.
Related guides (3)
Related events (8)
Hugging Face blog: Profiling PyTorch nn.Linear toward a fused MLP implementation
A Hugging Face blog post (Part 2 of a profiling series) walks through optimizing PyTorch's nn.Linear layers toward a fused MLP kernel. The post covers profiling methodology and kernel fusion techniques relevant to inference and training efficiency. This is a practical deep-dive into low-level PyTorch optimization for ML practitioners.
Improving Hugging Face Training Efficiency Through Packing with Flash Attention 2
Hugging Face published a blog post describing a technique for improving training efficiency by packing multiple short sequences into a single batch using Flash Attention 2. The approach reduces padding waste and improves GPU utilization during LLM fine-tuning. This is a practical infrastructure optimization relevant to practitioners training models on datasets with variable-length sequences.
Visualize and Understand GPU Memory in PyTorch
A Hugging Face blog post explains how to visualize and analyze GPU memory usage during PyTorch model training. The post covers tools and techniques for understanding memory allocation patterns, helping practitioners diagnose and reduce memory bottlenecks. This is practical infrastructure knowledge relevant to training large models efficiently.
How Hugging Face Accelerate Runs Very Large Models Thanks to PyTorch
This Hugging Face blog post explains the technical mechanisms behind the Accelerate library for running large models that exceed single-GPU memory, leveraging PyTorch features such as device maps, CPU/disk offloading, and sharded checkpoints. It describes how models can be distributed across multiple GPUs, CPU RAM, and disk storage transparently. The post serves as both documentation and a technical explainer for practitioners working with large-scale inference and deployment.
Hugging Face blog compares fine-tuning techniques beyond LoRA
A Hugging Face blog post examines whether alternative parameter-efficient fine-tuning (PEFT) methods can outperform LoRA, currently the dominant fine-tuning technique. The post likely benchmarks or analyzes competing approaches such as DoRA, IA3, or other PEFT variants against LoRA baselines. This is relevant for practitioners choosing fine-tuning strategies for LLMs.
Hugging Face on PyTorch / XLA TPUs
This Hugging Face blog post covers the integration of Hugging Face Transformers with PyTorch/XLA for training on Google TPUs. It describes how users can leverage TPU hardware through the XLA compiler backend to accelerate transformer model training. The post serves as a technical guide for the ecosystem connecting Hugging Face's model library with Google's TPU infrastructure.
How Hugging Face Sped Up Transformer Inference 100x for API Customers
Hugging Face describes engineering optimizations that achieved up to 100x speedups in transformer inference for their hosted API customers. The post covers techniques applied to accelerate model serving at scale. This is a 2021 article documenting early inference optimization work at Hugging Face's inference API product.
Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers
A Hugging Face blog post discusses inference optimization techniques derived from OpenAI's gpt-oss codebase that can be applied within the Hugging Face Transformers library. The post appears to cover practical tricks for improving transformer inference speed or efficiency. As a tier-2 source with commentary depth, this is a practitioner-oriented technical guide bridging OpenAI's internal methods and the open-source ecosystem.