Convert Transformers to ONNX with Hugging Face Optimum
Hugging Face published a guide on converting Transformer models to ONNX format using the Optimum library. The post covers the tooling workflow for exporting models from the Transformers ecosystem into ONNX for optimized inference deployment. This is a practical infrastructure topic relevant to production ML deployment patterns.
Related guides (3)
Related events (8)
Optimum + ONNX Runtime: Faster Training for Hugging Face Models
Hugging Face's Optimum library integrates with Microsoft's ONNX Runtime Training to accelerate fine-tuning of transformer models. The integration aims to reduce training time and memory usage with minimal code changes for practitioners using the Hugging Face ecosystem. This tooling update targets enterprise and research users looking to optimize training efficiency on existing hardware.
Introducing Optimum: The Optimization Toolkit for Transformers at Scale
Hugging Face announced Optimum, an optimization toolkit designed to accelerate Transformers models on various hardware backends. The toolkit aims to bridge the gap between Transformers model development and hardware-specific optimizations from partners. It provides a unified interface for quantization, pruning, and hardware-accelerated inference across different accelerators.
Accelerated Inference with Optimum and Transformers Pipelines
Hugging Face announced integration between the Optimum library and the Transformers Pipelines API, enabling hardware-accelerated inference with minimal code changes. The integration targets deployment on specialized hardware backends such as ONNX Runtime, allowing users to swap in optimized inference engines transparently. This lowers the barrier to production-grade inference optimization for practitioners using the Hugging Face ecosystem.
Accelerate your models with Optimum Intel and OpenVINO
Hugging Face's Optimum Intel library integrates with Intel's OpenVINO toolkit to accelerate inference of transformer models on Intel hardware. The post covers how to export models to OpenVINO IR format and run optimized inference pipelines. This targets deployment efficiency for NLP and vision models on CPU and other Intel accelerators.
Accelerating over 130,000 Hugging Face Models with ONNX Runtime
Hugging Face and Microsoft have integrated ONNX Runtime (ORT) to accelerate inference for over 130,000 models on the Hugging Face Hub. The integration enables optimized deployment across CPU and GPU hardware without requiring users to manually export or configure ONNX models. This represents a significant expansion of ORT's reach within the open-weights model ecosystem, lowering the barrier to production-grade inference optimization.
Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers
A Hugging Face blog post discusses inference optimization techniques derived from OpenAI's gpt-oss codebase that can be applied within the Hugging Face Transformers library. The post appears to cover practical tricks for improving transformer inference speed or efficiency. As a tier-2 source with commentary depth, this is a practitioner-oriented technical guide bridging OpenAI's internal methods and the open-source ecosystem.
Optimize and Deploy with Optimum-Intel and OpenVINO GenAI
Hugging Face's Optimum-Intel library integrates with Intel's OpenVINO runtime to enable optimized inference of generative AI models on Intel hardware. The post covers quantization, model export, and deployment workflows using OpenVINO GenAI APIs. This targets edge and CPU-based inference scenarios where reducing model size and latency is critical.
Transformers v5: Simple model definitions powering the AI ecosystem
Hugging Face has announced Transformers v5, a major version update to its flagship open-source library. The release focuses on simplified model definitions and architectural improvements to the codebase. As one of the most widely used ML libraries in the ecosystem, this update has broad implications for researchers and practitioners building on top of the Transformers framework.