Universal Image Segmentation with Mask2Former and OneFormer
Hugging Face published a blog post introducing Mask2Former and OneFormer, two universal image segmentation architectures now available in the Transformers library. These models unify panoptic, instance, and semantic segmentation tasks under a single framework. The post covers model capabilities, usage examples, and integration into the HuggingFace ecosystem.
Related guides (3)
Related events (8)
The State of Computer Vision at Hugging Face
Hugging Face published a survey of the computer vision ecosystem available through its platform as of early 2023, covering supported model architectures, tasks, datasets, and tooling. The post reviews progress in image classification, object detection, segmentation, and multimodal vision-language models integrated into the Transformers library. It serves as a reference for practitioners on what CV capabilities are accessible via the Hugging Face hub and APIs.
Zero-shot image segmentation with CLIPSeg
This Hugging Face blog post introduces CLIPSeg, a model that performs zero-shot image segmentation by leveraging CLIP-based text and image prompts. The approach allows segmentation of arbitrary image regions without task-specific training, using natural language or reference images as queries. The post likely covers integration into the Hugging Face ecosystem and practical usage examples.
Sentence Transformers Joins Hugging Face
Sentence Transformers, a widely-used library for generating sentence embeddings and semantic similarity, is officially joining Hugging Face. This integration brings the popular embedding framework under the Hugging Face ecosystem, likely enabling tighter integration with the Hub, datasets, and other HF tooling. The move consolidates a key component of the NLP/embedding pipeline within the dominant open-source AI platform.
Transformers v5: Simple model definitions powering the AI ecosystem
Hugging Face has announced Transformers v5, a major version update to its flagship open-source library. The release focuses on simplified model definitions and architectural improvements to the codebase. As one of the most widely used ML libraries in the ecosystem, this update has broad implications for researchers and practitioners building on top of the Transformers framework.
The Transformers Library: Standardizing Model Definitions
Hugging Face published a blog post outlining their approach to standardizing model definitions within the Transformers library. The post addresses how the library structures and maintains model code to ensure consistency, reproducibility, and ease of integration across a wide range of architectures. This is a tooling and ecosystem development relevant to practitioners building on or contributing to the Transformers framework.
NVIDIA NeMo Megatron-Bridge: Bidirectional Hugging Face Conversion for Megatron-Based Training
Megatron-Bridge is an NVIDIA NeMo training library for Megatron-based models that supports bidirectional conversion between Megatron and Hugging Face formats. The repository has accumulated 670 stars with modest daily growth (+5). It addresses a practical interoperability gap between the high-performance Megatron training stack and the broader HuggingFace ecosystem.
Introducing Decision Transformers on Hugging Face
Hugging Face introduces support for Decision Transformers, a framework that casts offline reinforcement learning as a sequence modeling problem using transformer architectures. The blog post covers the conceptual basis of Decision Transformers and their integration into the Hugging Face ecosystem. This represents an early step in bringing RL-based model paradigms into the standard ML tooling stack.
Graphcore and Hugging Face Launch New Lineup of IPU-Ready Transformers
Graphcore and Hugging Face announced a collaboration to make transformer models compatible with Graphcore's Intelligence Processing Unit (IPU) hardware. The partnership expands the set of Hugging Face models that can run natively on IPU infrastructure. This represents an effort to broaden the hardware ecosystem available for transformer model inference and training beyond GPUs.