Almanac
technique

Xet

techniqueactivexet-451311f8·2 events·first seen 28d ago

Aliases: Xet

Co-occurring entities

More like this (12)

Recent events (2)

5Hugging Face Blog·28d ago·source ↗

Xet Storage Integration on Hugging Face Hub

Hugging Face has integrated Xet, a chunk-based deduplication storage backend, into the Hub to improve large model file storage and transfer efficiency. The integration aims to reduce redundant data storage and speed up uploads/downloads for large model weights by splitting files into content-addressed chunks. This is an infrastructure improvement relevant to the open-weights ecosystem where multi-gigabyte model files are common.

5Hugging Face Blog·28d ago·source ↗

Migrating the Hugging Face Hub from Git LFS to Xet

Hugging Face is migrating its model and dataset hosting infrastructure from Git LFS to Xet, a content-addressed storage system designed for large binary files. The migration aims to improve upload/download speeds, deduplication, and storage efficiency for the large model weights and datasets hosted on the Hub. This represents a significant infrastructure change affecting how millions of AI artifacts are stored and accessed by the community.