4Hugging Face Blog·
Hugging Face introduces FFASR Leaderboard for real-world ASR benchmarking
Hugging Face has launched the FFASR Leaderboard, a new benchmark designed to evaluate automatic speech recognition (ASR) systems under real-world conditions. The leaderboard aims to address gaps in existing ASR evaluations by testing models on more challenging, naturalistic audio. This is a community-facing evaluation resource relevant to practitioners building or comparing speech recognition systems.
Related guides (2)
Related events (8)
Open ASR Leaderboard: Trends and Insights with New Multilingual & Long-Form Tracks
Hugging Face has updated its Open ASR Leaderboard to include new multilingual and long-form audio transcription evaluation tracks. The post analyzes trends across submitted automatic speech recognition models, providing comparative benchmarking data across languages and extended audio contexts. This expands the leaderboard's coverage beyond English short-form ASR to better reflect real-world deployment scenarios.
Hugging Face introduces Real World VoiceEQ benchmark for human-quality voice AI evaluation
Hugging Face has released Real World VoiceEQ, a new benchmark designed to measure the human-perceived quality of voice AI systems. The benchmark targets a gap in existing evaluation frameworks by focusing on naturalistic, real-world voice interaction quality rather than purely technical metrics. This is relevant to the growing voice AI ecosystem where subjective quality dimensions are difficult to quantify.
Adding Benchmaxxer Repellant to the Open ASR Leaderboard
Hugging Face describes measures taken to prevent benchmark gaming ('benchmaxxing') on the Open ASR Leaderboard by introducing private or held-out evaluation data. The post addresses the integrity of automatic speech recognition benchmarks, where models may be overfitted or tuned specifically to public test sets. This is part of a broader effort to maintain meaningful leaderboard rankings as ASR model submissions increase.
Bringing the Artificial Analysis LLM Performance Leaderboard to Hugging Face
Hugging Face is hosting the Artificial Analysis LLM Performance Leaderboard, which tracks inference performance metrics such as latency, throughput, and cost across multiple LLM providers. The leaderboard provides a standardized comparison of how different models perform in production deployment contexts rather than purely capability benchmarks. This collaboration brings infrastructure and deployment performance data into the Hugging Face ecosystem.
FunASR: Industrial-Grade Speech Recognition Toolkit with 170x Realtime Performance
FunASR is an open-source speech recognition toolkit from ModelScope supporting 50+ languages, speaker diarization, emotion detection, and streaming inference at 170x realtime speed. It exposes an OpenAI-compatible API, positioning it as a drop-in alternative for production ASR workloads. The repository has accumulated 16,317 stars with modest daily momentum (+42 today).
Object Detection Leaderboard on Hugging Face
Hugging Face has launched an object detection leaderboard to benchmark and compare models on standard detection tasks. The leaderboard provides a centralized evaluation platform for tracking progress in object detection across the community. This follows the pattern of Hugging Face expanding its evaluation infrastructure for specific ML subdomains.
Guide to Setting Up a Hugging Face Leaderboard: Vectara Hallucination Leaderboard as Example
This Hugging Face blog post provides an end-to-end tutorial on creating custom leaderboards on the Hugging Face platform, using Vectara's hallucination leaderboard as a concrete example. It covers the technical setup process for hosting evaluation leaderboards, which are increasingly important infrastructure for tracking model capabilities. The post bridges tooling and evaluation concerns by showing how third-party organizations can publish standardized benchmarks on HF.
Arabic Leaderboards: Introducing Arabic Instruction Following, Updating AraGen, and More
Hugging Face introduces new Arabic-language evaluation infrastructure, including an Arabic Instruction Following benchmark and updates to the AraGen leaderboard. The post covers evaluation methodology for Arabic LLM capabilities, expanding the ecosystem of non-English benchmarks. This is part of a broader effort to track model performance on Arabic language tasks beyond standard English-centric evaluations.