EPFL

MODUS: Decoder-only architecture for any-to-any multimodal modeling without modality-specific heads

Researchers at EPFL introduce Modus, a decoder-only model that treats all modalities symmetrically for any-to-any prediction without modality-specific heads, losses, or task pipelines. Unlike prior any-to-any models that use encoder-decoder or diffusion architectures trained from scratch, Modus leverages pre-trained decoder-only models as a prior. The model supports chained generation through intermediate modalities and cross-modal self-verification, achieving competitive performance with specialist and multitask baselines across benchmarks. All materials are open-sourced.