Algorithmic composition - style mimic

[Musicologo]

When you think of Algorithmic composition techniques in order to mimic a style you can have at least two approaches: rule based or intelligence.

Rule based, you give the rules for computer to imitate a style. You have to know the rules yourself.

Intelligence you provide a database with songs and the computer has to infer the rules and patterns from the corpus and then tries to generate new examples similar to those ones.

Now,

If your goal is to mimic a complex style or practice that has simple rules with lots of variants and a large corpus, say, Polish Polkas, Argentinian Tangos or Jamaican Reggaes, What would you do?

1. Rule based or intelligence aproach?

2. What language would be the best to implement your system? C++, Open Frameworks, Max/MSP, Nyquist, Lisp, OpenMusic, any other?...

3. What is the best language to work with MiDi files, if you have a database full of midis?

I've been reading David Cope, Eduardo Reck Miranda, Nierhaus and still haven't find a consensus. Do you have any thoughts on this matter?

P.S. Don't ask me "and WHY do you want to do that?", I just have to

[Ogg Vorbis]

I find myself stumbling over the term, "rules." It seems like the best you could do would be to have a database full of polkas, say, and have the software examine all the dimensions of the music to look for statistics on how many times the phrase structure of the main melody is 32 bars before it goes to the "B" section, or how rapid the harmonic movement is, etc.

From this statistical hierarchy, you could ask the computer to generate something new that had a similar proportion of these dimensional occurrences.

But "rules" are more like..."the system shall dispense the correct number of notes to create an 'A' section," which would not really work for music (even polkas!).

It would be like trying to extract the "rules" for weather.

But computers can be really fast at collecting statistical occurrences of things from a set, so I'd go with that approach. And I'd guess that just about any computer language could be used to do that.

[kbaccki]

But "rules" are more like..."the system shall dispense the correct number of notes to create an 'A' section," which would not really work for music (even polkas!).

I was thinking more along the lines of: "a basic rock song shall utilize some predefined combinations of the I, ii, IV, V, and vi chords"... and "a basic jazz song shall utilize at least one ii-V-I resolution, and contain at least two min7b5 chords"...

On the programming language end I know that Java as of 1.4 has support for reading/manipulating/writing MIDI files... http://java.sun.com/j2se/1.4.2/docs/api ... mmary.html

I would say it's at least a good idea to consider prototyping in Java given the breadth of runtime/API features at your disposal and the relative simplicity of the language and APIs compared to say writing a windows app in C++...

Oh, and I forgot to weigh in on rule vs. intelligence... the former approach is like 10,000 times simpler than the latter. Considering most syles of music literally regurgitate the idioms and structures of what came before (except perhaps for really "complex" stuff, like lengthy, evolving classical pieces for example), a hefty rule system could get you 95% of the way there... the hard part is "improv", and the intelligence (consciousness, understanding, analysis, decision making, etc.) of making music... for that you would need a complex learning system, right? For the project sponsors we call that AI -- Additional Investment.

[jancivil]

It's either intelligence or it's doomed to fall very short of what a person will like to hear. "Music" is too subtle to break down to that (infinitely) 'simpler' level.

Even if you got rid of 'pitch' content altogether, real rhythm (eg., rock, jazz) REQUIRES intelligence. Unless you're building a techno/electro/tarnce/roboto kind of deal, which doesn't really.

I would most strongly advise against beginning with a premise like: 'it's all regurgitated from some previous model'. It is but it isn't. You're going to have too-narrow a model to work I think.

[kbaccki]

I suppose. It comes down to a simple problem, IMO, not related to music in any way: how much time and money do you have to build XYZ? In this case XYZ = intelligence system. If you're not bound by time and money, then go for the full monty. If you're working under constraints just keep in mind that building intelligence is exceedingly complex. Sometimes (most times?) the end effect is more important than how you got there.

Think about photorealism or physics simulation (or team/foe/bystander/whoever AI, for that matter) in computer games... Back in the day you had "2.5D" graphics... it looked like 3D but it wasn't actually 3D. A physics simulation may get you a semblance of the physical world, but in the end it's just a rough approximation with lots of limitations relative to a "generalized solution", due to intrinsic constraints on CPU, code complexity, etc. The point is to make the model, rendering, simulation, music creation process, or whatever give the end user an effective representation in the least amount of complexity, even if the implementation is not "perfect" from a purist perspective. If you have a deadline, then I'd say at some point you have to avoid the "if I can first model the human brain, then I could..." approach.

[phlake]

I think that Markov Chains would be interesting to apply to this kind of topic. Java has a pretty straightforward MIDI file implementation; you might look at that.

[deastman]

My brother once wrote me a proof-of-concept algorithmic melody generator in Java. If you're limiting yourself to MIDI, the implementation is extremely simple and straightforward. Personally, I would go for the rule-based approach, just because you're more likely to get useful results in this lifetime.

I'm a big fan of David Cope's work in this area, but he's been at this for decades. I think you need to set realistic goals for what you can expect to accomplish.

For prototyping purposes, you might also check out ArtWonk/MusicWonk from Alogrithmic Arts. I've worked with an older version of their product, SoftStep, and it was built specifically as a sandbox for exploring concepts in generative music.
http://algoart.com/

[xh3rv]

I've been reading David Cope, Eduardo Reck Miranda, Nierhaus and still haven't find a consensus. Do you have any thoughts on this matter?

Ya, there's actually no distinction between the rules-based and intelligence-based. Even with machine learning, we really cannot escape the fact that it's only learning what it was programmed to learn. IMO the most exciting things happening related to any kind of machine learning are approaches which explicitly recognize that machines and humans are good at different things. [e] So like, a computer that manages style templates, does some numeric analysis of a corpus of music, etc. could greatly enhance the process of constructing a set of rules for a style.

The real question is, how deeply and specifically which abstractions are we going to use to model musical style? This is what there won't be consensus on, and that's actually a good thing - different choices will lead to different output which can be valuable in different contexts.

Language choice is irrelevant without a sense of what the abstracted model of style is - actually coding something here isn't really a challenge, relatively, the music-oriented languages like Max/MSP probably won't help much for the harder problems (well, Max/MSP is all I've seen dug into), what they will have a lot of handy utility. The one notable thing I would say here is that LISP or something similar can be a pretty cool tool for experimenting with the abstracted model of (music or anything) if it's not completely well-defined. It's certainly not the only language used in AI research but it is a notable one and not for no reason.

[jancivil]

... the hard part is "improv", and the intelligence (consciousness, understanding, analysis, decision making, etc.) of making music... for that you would need a complex learning system, right? For the project sponsors we call that AI -- Additional Investment.

My post was based on an assumption, which may not be right, that the goal was a machine which could, for real, write some music. Writing music, begins with improvisation. If the goal is a facsimile of, or... 'generating something which might seem like somebody made it, at first, until you realize...' The key word which will be missing outside of AI is 'understanding'.

that's a lesser problem to solve by far, but something someone might suss along the way to a greater goal?

[Musicologo]

Humm... Thanks for so many enlighted replies.

I'd have to say that really time and resources will be a key issue here. So, to same extent they will condition the final result and the approach, because I need to present results, whatever they are in a certain amount of time. I can't just make it for the sake of it.

On the other hand, till now, I was more worried with the writing than the performance.

So, It is of no shock for me if the machine just writes scores and midis.
The improvisation part and the performance would then be another different problem.

Let's say: first we need to mimic the composer, the creative.
Only then we need to create a "humanizer" to perform the midis in a more convincing way.

Does that makes sense? Or you think it should be the same package? The process of composition should embbed in itself the subtelities of a human performance, swing, etc...?

[deastman]

Does that makes sense? Or you think it should be the same package? The process of composition should embbed in itself the subtelities of a human performance, swing, etc...?

Taking this in the traditional sense of a classical composer writing a composition, they would only hear their finished composition in its full rendition when it was performed by a human orchestra. If your intention is ultimately to hear a convincing rendition of the composition, I think you'll have to take performance aspects into consideration at some point.

[Ogg Vorbis]

Does that makes sense? Or you think it should be the same package? The process of composition should embbed in itself the subtelities of a human performance, swing, etc...?

Putting it all in the "same package" would probably be limiting and not as sensible for the following reasons:

1) The "rules of composition" (man, I still have a problem with that) are different than the principles of performance. Let's say that a composer writes a nice arppegiation for the violas. The use of such a figure is consistent with the computer's understanding of the style, etc.

But a human viola section will add "life" to such a figure automatically. They will do a small cresc. when the figure rises and dim. when it falls. They will add a little bit more bow weight to the top of the figure as it ascends and maybe even very subtly alter the note duration to bring out the top of the phrase.

In repeated notes, performers will automatically reduce the intensity of the subsequent notes so avoid the machine-like "sawing" that young inexperienced players will often do.

So there are performance parameters independent of the composer's intentions, so it makes sense to segment those functions.

2) Interpretation of the same piece varies from performer-to-performer. And that's usually a GOOD thing. There's more than one way to perform a piece. Add to this, that a performer will never play the same piece the same way night after night.

Audiences appreciate that the performer brings their own interpretive expertise to the music and their own personal touches. Composers usually appreciate that as well. It's nice to know that a piece of music is not just a static realization, but a living, breathing organic entity.

So create a computer program which will emulate performance seems quite daunting to me. How many "rules" are there to human performance? Well, again, "rules" aren't going to cut it. There are too many exceptions to any rule that can be layed down.

A computer program will have to be able to look at a phrase and see it as a whole, rather than a collection of individual, discrete note events. It will have to look at the context of that phrase and be able to make spur-of-the-moment decisions on how to approach performing that phrase. And then, how will the phrase connect to the next phrase so that it makes musical sense?

And so on.

I think that computer performance would be more than twice as difficult as computer composition. Maybe you could just focus on algorithms for composition and ask humans to play what it produces?

[TiUser]

Referring to your definition of the process I would say "Intelligent".

The idea you suggest has already being practically proven to work for composition.

What you call "rule based" is what KI tried for decades but finally always failed because you never can find a complete set of perfect rules describing everything that may be relevant.

Maybe the reason why I think so is that the brain itself works "intelligent" too - quite close to the process you suggest for music composition but for almost everything. The brain is not able to store the loads of everything straight forward like a database does. So it creates rules - a kind of compression while gaining application versatility at the same time.

The problem with the brain is that it applies rules all the time without telling our conscience it does, so it's hard to identify all the rules we apply. But in some areas of brain research this has also being verified - for instance with linguistic rules.

Yes, people will deny that your "intelligent" approach can work because you offend something fundamental with your question - the human as a creative individual being with self conscience and some uniqueness in the universe. Some of the tricks our brains have to prevent us from getting mad...

[Ogg Vorbis]

If you assigned note values based on the quantification of receding arctic sea ice, you'd get "Al Gore Rhythmic Composition."

Ha ha ha Ha ha ha Ha ha ha Ha ha ha Ha ha ha Ha ha ha

[jancivil]

Humm...

Let's say: first we need to mimic the composer, the creative.
Only then we need to create a "humanizer" to perform the midis in a more convincing way.

Does that makes sense? Or you think it should be the same package? The process of composition should embbed in itself the subtelities of a human performance, swing, etc...?

My point was that to mimic the composer, that's 'understanding'. The process tends to be bits of "improvisation" edited into some kind of shape, or even 'form'. I wouldn't separate out these rhythmic characteristics from that. There is 'baroque' rhythm, even baroque swing, which is part and parcel of the 'harmonic rhythm'. In western music, that right there is crucial. In Bach pere and fils, there are timings in the ornamention - at cadence points, ok - that are part of the design characteristic; your goals are approached and prepared; these ornaments are shaped by performance practice.
In jazz, if it ain't got that swing it don't mean a thing, and that's the truth. This is style; style distinguishes 'a music' from other musics.

As far as a "humanizer", I've never seen one that works, in this after-the-fact scenario. I think there are reasons for this. I think it's fascinating...

I am personally quite skeptical it can be done well at this point in human history. That isn't because the idea offends me, as far as I know we could be virtual musicians in some higher intelligence's computer simulation and given limited intelligence, just enough for the level we're on. I have no elevated idea of my, or our place in the univers3.

I think to design a thing like this that makes beautiful music is another step in evolution to be honest.
It's fascinating, I commend you for your efforts regardless.