Introducing BLOOM: The World's Largest Open Multilingual Language Model
Hugging Face and the BigScience workshop released BLOOM, a 176-billion parameter open-access multilingual language model trained on 46 natural languages and 13 programming languages. The model was developed collaboratively by over 1,000 researchers and represents a significant milestone in open-weights large language model development. BLOOM was designed to be freely accessible to researchers and practitioners, in contrast to proprietary models of similar scale.
Related guides (4)
Related events (8)
The Technology Behind BLOOM Training
This Hugging Face blog post details the infrastructure and training methodology used to train BLOOM, a 176-billion parameter open-access multilingual language model. It covers the use of Megatron-DeepSpeed for distributed training across hundreds of GPUs, including tensor parallelism, pipeline parallelism, and data parallelism strategies. The post also discusses hardware setup, memory optimization techniques, and lessons learned during the large-scale training run.
Fast Inference on Large Language Models: BLOOMZ on Habana Gaudi2 Accelerator
This Hugging Face blog post covers deploying BLOOMZ, a large multilingual language model, on Intel's Habana Gaudi2 accelerator for inference. It benchmarks throughput and latency performance on Gaudi2 as an alternative to GPU-based inference. The post is part of ongoing work to demonstrate non-NVIDIA hardware options for large model deployment.
Optimization story: Bloom inference
This Hugging Face blog post documents practical inference optimization techniques applied to the BLOOM large language model. It covers strategies for reducing latency and memory footprint during deployment, likely including quantization, tensor parallelism, and batching approaches. The post serves as a technical case study for serving very large open-weights models efficiently.
2023, Year of Open LLMs
Hugging Face's year-in-review post surveys the major open-weight large language model releases and milestones of 2023. The piece covers the proliferation of open models from various labs and the ecosystem developments that made them accessible. It serves as a retrospective on how open-source LLMs matured and competed with proprietary systems throughout the year.
Falcon 180B Released: New Open-Weights Frontier Model
Technology Innovation Institute (TII) has released Falcon 180B, a 180-billion parameter open-weights language model announced via Hugging Face. At the time of release, it was positioned as the largest publicly available open-weights model, trained on 3.5 trillion tokens. The model is available on Hugging Face Hub for research and commercial use under a custom license.
Very Large Language Models and How to Evaluate Them
This Hugging Face blog post from October 2022 discusses approaches to zero-shot evaluation of large language models hosted on the Hub. It covers methodologies for benchmarking LLMs without task-specific fine-tuning, addressing the practical challenges of evaluating very large models at scale. The post situates evaluation tooling within the broader ecosystem of open model hosting and assessment.
The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare
Hugging Face has launched the Open Medical-LLM Leaderboard, a public benchmark for evaluating large language models on healthcare and medical tasks. The leaderboard aggregates performance across multiple medical question-answering datasets to enable standardized comparison of open-weight models in clinical and biomedical domains. This initiative aims to accelerate progress in medical AI by providing transparent, reproducible evaluation infrastructure.
Sumi: First open 7B uniform diffusion language model pretrained from scratch at scale
Researchers introduce Sumi, a fully open 7B uniform diffusion language model (UDLM) pretrained from scratch on 1.5 trillion tokens — the first UDLM at both large parameter scale and large token budget. Sumi performs competitively with autoregressive models on knowledge, reasoning, and coding benchmarks, though underperforms on commonsense tasks, attributed partly to an education-heavy data mixture. Model weights, checkpoints, and full training recipe including data mixture specification are released publicly. The work fills a gap in the diffusion language model landscape, providing a reference point for studying scaling behavior and generation dynamics in uniform diffusion.