Procedural music generation techniques [closed]

Question

I've been putting a lot of thought into procedural generation of content for a while and I've never seen much experimentation with procedural music. We have fantastic techniques for generating models, animations, textures, but music is still either completely static or simply layered loops (e.g. Spore).

Because of that, I've been thinking up optimal music generation techniques, and I'm curious as to what other people have in mind. Even if you haven't previously considered it, what do you think will work well? One technique per answer please, and include examples where possible. The technique can use existing data or generate the music entirely from scratch, perhaps on some sort of input (mood, speed, whatever).

Answer 1

Cellular Automata - read.

You can also try it out here.

Edit:

rakkarage has supplied another resource: http://www.ibm.com/developerworks/java/library/j-camusic/

Answer 2

The most successful system will likely combine several techniques. I doubt you'll find one technique that works well for melody, harmony, rhythm and bass sequence generation across all genres of music.

Markov chains, for instance, are well suited for melodic and harmonic sequence generation. This method requires analysis of existing songs to build the chain transition probabilities. The real beauty of Markov chains is that the states can be whatever you want.

• For melody generation, try key-relative note numbers (e.g. if the key is C minor, C would be 0, D would be 1, D# would be 2 and so on)
• For harmony generation, try a combination of key-relative note numbers for the root of the chord, the type of the chord (major, minor, diminished, augmented, etc.) and the inversion of the chord (root, first or second)

Neural networks are well suited to time series prediction (forecasting), which means they're equally suited to 'predicting' a musical sequence when trained against existing popular melodies/harmonies. The end result will be similar to that of the Markov chain approach. I can't think of any benefit over the Markov chain approach other than reducing the memory footprint.

In addition to pitch you will need duration to determine the rhythm of the generated notes or chords. You can choose to incorporate this information into the Markov chain states or neural network outputs, or you can generate it separately and combine the independent pitch and duration sequences.

Genetic algorithms can be used to evolve rhythm sections. A simple model could use a binary chromosome in which the first 32 bits represent the pattern of a kick drum, the second 32 bits a snare, the third 32 bits a closed hi hat and so on. The downside in this case is that they require continuous human feedback to assess the fitness of the newly evolved patterns.

An expert system can be used to verify sequences generated by the other techniques. The knowledge base for such a validation system can probably be lifted from any good music theory book or website. Try Ricci Adams' musictheory.net.

Answer 3

There are over 50 years of research into these techniques, often overlooked by developers not familiar with the history of computer music and algorithmic composition. Numerous examples of systems and research that address these issues can be found here:

http://www.algorithmic.net

Answer 4

An easy and somewhat effective algorithm is to use 1/f noise aka "pink noise" to select durations and notes from a scale. This sounds sort of like music and can be a good starting point.

A better algorithm is to use "markov chains".. scan some example music and build a table of probabilities. In the simplest case, it would be something like C is 20% likely to follow A. To make this better, look at the sequence of the past few notes, for example "C A B" is 15% likely to be followed by B, and 4% likely to be followed by a Bb, etc. Then, just pick notes using the probabilities of the previously chosen notes. This remarkably simple algorithm generates pretty good results.

Markov chains for music generation

Answer 5

Dmitri Tymoczko has some interesting ideas and examples here :

http://music.princeton.edu/~dmitri/whatmakesmusicsoundgood.html

Answer 6

My software uses applied evolutionary theory to "grow" music. The process is similar to Richard Dawkins' The Blind Watchmaker program - MusiGenesis adds musical elements randomly, and then the user decides whether or not to keep each added element. The idea is to just keep what you like and ditch whatever doesn't sound right, and you don't have to have any musical training to use it.

The interface blows, but it's old - sue me.

Answer 7

I have always liked the old Lucasarts games that used the iMuse system, which produced a never-ending, reactive soundtrack for the game and was very musical (because most of it was still created by a composer). You can find the specs (including the patent) here: http://en.wikipedia.org/wiki/IMUSE

Nintendo seems to be the only company to still use an approach similar to iMuse to create or influence the music on the fly.

Unless your project is very experimental, I would not abandon the use of a composer - a real human composer will produce much more musical and listenable results than an algorythm.

Compare it to writing a poem: You can easily generate nonsene poems which sound very avant-garde, but to replicate shakespeare with an algorythm is difficult, to put it mildly.

Answer 8

Have you taken a look at SoundHelix (http://www.soundhelix.com)? It's an Open-Source Java framework for algorithmic random music creation that produces pretty neat music. You can use SoundHelix as a standalone application, as an applet embedded in a webpage, as an JNLP-based applet or you can include it into your own Java program.

Examples generated with SoundHelix can be found here: http://www.soundhelix.com/audio-examples

Answer 9

Research on non-boring procedural music generation goes way back. Browse old and new issues of Computer Music Journal http://www.mitpressjournals.org/cmj (no real domain name?) This has serious technical articles of actual use to music synthesis tinkerers, soldering iron jockeys, bit herders and academic researchers. It's ot a fluffy reviews and interviews rag such as several of the mags you can find in major bookstores.

Answer 10

Such a big subject. You could take a look at my iPad app, Thicket, or my Ripple software at morganpackard.com. In my experience, most of the academic approaches to dynamic music generation come up with stuff that sounds, well, academic. I think the more successful stuff is found on the fringes of the club/electronica world. Monolake is my hero in this respect. Very listenable stuff, very much computer-generated. My own music isn't bad either. Paul Lansky's "Alphabet Book" is a nice example of extremely listenable algorithmic music, especially considering that he's an academic guy.

Answer 11

The technique I've been considering is to create small musical patterns, up to a bar or so. Tag these patterns with feeling identifiers such as 'excitement', 'intense', etc. When you want to generate music for a situation, pick a few patterns based on these tags and pick an instrument you want to play it with. Based on the instrument, figure out how to combine the patterns (e.g. on a piano you may be able to play it all together, depending on hand span, on a guitar you may play the notes in rapid succession) and then render it to PCM. In addition, you could change key, change speed, add effects, etc.

Answer 12

The specific technique you're describing is something Thomas Dolby was working on ten or fifteen years ago, though I can't remember now what he called it so I can't give you a good search term.

But see this Wikipedia article and this Metafilter page.

Answer 13

The book Algorithmic Composition is a good tour of the several methods used:

"Topics covered are: markov models, generative grammars, transition networks, chaos and self-similarity, genetic algorithms, cellular automata, neural networks and artificial intelligence."

It is a good starting point on this wide topic, however it does never describe in depth how each method works. It provides a good overview of each, but will not be enough to if you do not already have knowledge about them.

Answer 14

Back in the late 90's, Microsoft created an ActiveX control called the "Interactive Music Control" which did exact what your looking for. Unfortunately, they seem to have abandon the project.

Answer 15

Not quite what you're after, but I knew someone who looked at automatically generating DJ sets called Content Based Music Similarity.

Answer 16

If you're into deeper theories about how music hangs together, Bill Sethares site has some interesting twists.

Answer 17

Ive been looking into doing this project proposal - "8.1" from the "Theory and praxis in programming language" research group from the University of Copenhagen - department of CS:

8.1 Automated Harvesting and Statistical Analysis of Music Corpora

Traditional analysis of sheet music consists of one or more persons analysing rhythm, chord sequences and other characteristics of a single piece, set in the context of an often vague comparison of other pieces by the same composer or other composers from the same period.

Traditional automated analysis of music has barely treated sheet music, but has focused on signal analysis and the use of machine learning techniques to extract and classify within, say, mood or genre. In contrast, incipient research at DIKU aims to automate parts of the analysis of sheet music. The added value is the potential for extracting information from large volumes of sheet music that cannot easily be done by hand and cannot be meaningfully analysed by machine learning techniques.

This - as I see it - is the opposite direction of your question the data generated - I imagine - could be used in some instances of procedural generation of music.

Answer 18

My opinion is that generative music only works when it goes through a rigorous selection process. David Cope, an algorithmic music pioneer, would go through hours of musical output from his algorithms (which I think were mostly Markov Chain based) to pick out the few that actually turned out well.

I think this selection process could be automated by modeling the characteristics of a particular musical style. For instance, a "disco" style would award lots of points for a bassline that features offbeats and drum parts with snares on the backbeats but subtract points for heavily dissonant harmonies.

The fact is that the music composition process is filled with so many idiomatic practices that they are very difficult to model without specific knowledge of the field.

Answer 19

I've been working on a Python module for procedural music. I just programmed out what I know about notes, scales, and chord construction, then have been able to let it randomly generate content from those constraints. I'm sure there's more theory and patterns a system like that could be taught, especially by someone who understands the subject better. Then you can use those systems as constraints for genetic algorithms or randomized content generation.

You can go over my implementation here, especially the randomly generated lead example may be useful to you. Someone with a solid understanding of chord progressions could create a song structure from techniques like that and implement constrained random melodies like this over it. My knowledge of music theory does not extend that far.

But basically, you'll need to encode the theory of the kind of music you want to generate, and then use that as a constraint for some algorithm for procedurally exploring the range of that theory.