MuseNet: OpenAI's Transformer-Based Multi-Instrument Music Generation System
OpenAI released MuseNet, a deep neural network capable of generating 4-minute musical compositions across 10 instruments and multiple styles. The system uses the same large-scale transformer architecture as GPT-2, trained on hundreds of thousands of MIDI files to predict the next token in a sequence. MuseNet discovered patterns of harmony, rhythm, and style without explicit musical programming, demonstrating the generality of the GPT-2 unsupervised approach beyond text.
Related guides (3)
Related events (8)
OpenAI Jukebox: Neural Music Generation with Singing as Raw Audio
OpenAI introduced Jukebox, a neural network capable of generating music including rudimentary singing as raw audio across various genres and artist styles. The model operates directly on raw audio rather than symbolic representations like MIDI. OpenAI released model weights, code, and a sample exploration tool alongside the announcement.
Image GPT: Transformer Models Applied to Pixel Sequences for Image Generation and Classification
OpenAI demonstrates that a large transformer model trained autoregressively on pixel sequences can generate coherent image completions and samples, analogous to text generation. The work establishes a correlation between generative sample quality and downstream image classification accuracy. The best generative model achieves features competitive with top convolutional networks in the unsupervised setting, suggesting shared representational principles across modalities.
Generative modeling with sparse transformers
OpenAI introduced the Sparse Transformer, a deep neural network using a modified sparse attention mechanism to model sequences up to 30x longer than previously feasible with standard transformers. The approach sets new benchmarks on text, image, and audio generation tasks. The key algorithmic contribution is factorized sparse attention patterns that reduce the quadratic complexity of full self-attention.
Meta Introduces Muse Spark: First Closed-Weights Model from Superintelligence Labs
Meta released Muse Spark, its first AI model in roughly a year and the debut product of its Superintelligence Labs, marking a significant departure from its open-weights Llama strategy. The natively multimodal reasoning model supports tool use and multi-agent orchestration, achieves fourth place on the Artificial Analysis Intelligence Index, and claims notable token efficiency—matching Llama 4 Maverick with over 10x less training compute. Meta withheld parameter count, architecture, and training details, positioning Muse Spark as a closed commercial product competing with OpenAI, Google, and Anthropic. The release introduces 'thought compression' via RL and a parallel multi-agent 'contemplating' mode, while showing gaps in coding and agentic benchmarks.
Meta Introduces Muse Spark: First Model from Meta Superintelligence Labs with Multimodal Reasoning and Multi-Agent Orchestration
Meta has launched Muse Spark, the first model from its newly formed Meta Superintelligence Labs, positioned as a natively multimodal reasoning model with tool-use, visual chain-of-thought, and multi-agent orchestration capabilities. The model introduces 'Contemplating mode,' which runs multiple agents in parallel to compete with frontier reasoning modes, achieving 58% on Humanity's Last Exam and 38% on FrontierScience Research. Meta claims a greater than 10x compute efficiency improvement over Llama 4 Maverick through a rebuilt pretraining stack, and describes predictable scaling across pretraining, RL, and test-time reasoning axes. Muse Spark is available at meta.ai with a private API preview, and is framed as the first step on a scaling ladder toward 'personal superintelligence.'
Magenta RealTime 2: open-weights live music generation model
Magenta RealTime 2 is an open-weights model for real-time live music generation, released as a GitHub repository under the Magenta project. The repository has accumulated 1,440 stars with 86 added in a single day, indicating notable community interest. This represents a multimodal generative AI release in the audio/music domain.
Meta Pivots to Closed Weights with Muse Spark; The Batch Issue 349 Roundup
Meta introduced Muse Spark, its first AI model in roughly a year and the first product from its Superintelligence Labs, marking a pivot away from its open-weights strategy toward a closed model. Muse Spark is a natively multimodal reasoning model supporting tool use and multi-agent orchestration, with three reasoning modes and a novel 'thought compression' post-training technique using RL to penalize excessive reasoning tokens. The model ranks fourth on the Artificial Analysis Intelligence Index and matches Llama 4 Maverick's capabilities with over an order of magnitude less training compute, though it trails in coding and agentic benchmarks. The issue also covers broader industry themes including AI-native software engineering team structures, big pharma AI adoption, and regulatory developments.
Better language models and their implications
OpenAI announced GPT-2, a large-scale unsupervised language model capable of generating coherent multi-paragraph text and achieving state-of-the-art performance on language modeling benchmarks. The model demonstrated zero-shot capability across reading comprehension, machine translation, question answering, and summarization without task-specific fine-tuning. OpenAI notably withheld the full model release citing misuse concerns, marking an early high-profile instance of staged/responsible release policy.