MuseNet, developed by OpenAI, is a deep neural network that has the ability to generate 4-minute musical compositions using 10 different instruments. It can combine various styles ranging from country to Mozart to the Beatles. Unlike traditional music composition programs, MuseNet does not rely on explicit programming based on human understanding of music. Instead, it has learned patterns of harmony, rhythm, and style by predicting the next token in hundreds of thousands of MIDI files. This is made possible by using the same general-purpose unsupervised technology as GPT-2, a large-scale transformer model trained to predict the next token in a sequence, whether it is audio or text.
MuseNet offers both simple and advanced modes of music generation. In the simple mode, users can explore the variety of musical styles the model can create by listening to random uncurated samples that have been pre-generated. In the advanced mode, users can directly interact with the model, although the completions may take longer. This allows users to create entirely new musical pieces.
One of the notable features of MuseNet is its ability to blend different musical styles in novel ways. For example, users can provide a few notes from a Chopin Nocturne and ask the model to generate a piece in a pop style with piano, drums, bass, and guitar. MuseNet is able to convincingly blend the two styles, with the full band joining in at a specified time. This flexibility allows musicians and non-musicians alike to experiment and create new compositions.
MuseNet has some limitations, such as the use of composer and instrumentation tokens to give more control over the generated samples. During training, these tokens were prepended to each sample to guide the model in making note predictions. Additionally, MuseNet utilizes the recompute and optimized kernels of Sparse Transformer to train a 72-layer network with 24 attention heads, enabling it to remember long-term structure in a piece. The training data for MuseNet was collected from various sources, including ClassicalArchives, BitMidi, and the MAESTRO dataset.
OpenAI encourages users to share their creations and provides a tweet button in the MuseNet demo to facilitate sharing. They are also open to suggestions for improving the MuseNet demo and welcome those interested in composing with MuseNet or contributing MIDI files to the training set.
