technique

Data Parallelism

techniqueactivedata-parallelism-b7501ac9·1 events·first seen 28d ago

Aliases: Data Parallelism

Co-occurring entities

N-Dimensional Parallelism tensor parallelism pipeline parallelism Accelerate Hugging Face

More like this (12)

tensor parallelism N-Dimensional Parallelism pipeline parallelism Parallel-Synthesis Data Is Better Together Adaptive Parallel Reasoning Ulysses Sequence Parallelism CUDA near-deduplication Temporal Simultaneity distributed training PyTorch DDP

Recent events (1)

5Hugging Face Blog·28d ago·source ↗

Accelerate ND-Parallel: A Guide to Efficient Multi-GPU Training

Hugging Face published a guide on N-dimensional parallelism for multi-GPU training using the Accelerate library. The post covers combining data parallelism, tensor parallelism, pipeline parallelism, and other strategies to efficiently scale model training across GPU clusters. This is a practical technical resource aimed at practitioners working with large-scale distributed training setups.

Training Infrastructure Agent and Tool Ecosystem N-Dimensional Parallelism tensor parallelism pipeline parallelism +3 more