Audio & Educational Technology

I believe that a discussion about musical “style” would be incomplete without at least a mention of Pandora.  Pandora comes to us as a result of the Music Genome Project, by Tim Westergren.   Pandora comes to us as a result of musical analysis of “tens of thousands” of artists.  As Tim describes it, they analyzed,

“the musical qualities of each song one attribute at a time. This work continues each and every day as we endeavor to include all the great new stuff coming out of studios, clubs and garages around the world.”

Tim goes to describe how their team attempted to describe music according to unique “attributes” that they call “genes”.

“We ended up assembling literally hundreds of musical attributes or “genes” into a very large Music Genome. Taken together these genes capture the unique and magical musical identity of a song – everything from melody, harmony and rhythm, to instrumentation, orchestration, arrangement, lyrics, and of course the rich world of singing and vocal harmony.”

Professor Pennycook & I were discussing the mechanism they likely used to do this, which almost certainly involves meta-data rather than musical pattern-matching.   Pandora lets users create stations based on their favorite songs or artists, and will deliver songs/artists that match the attributes of the artist/song used to create the station.  You can also vote in favor of the selection played or against it.  In this way, you can “train” Pandora to match the songs that you think the station should play.  Pandora will actually send you an email reminding you to do this periodically.  One quesiton we had is whether our votes influenced the way the database tags the music.  One thing that we found curious was why in some cases when we created a station, the first song that plays is often not by the artist for which the station was named.  This happened with Dr. Pennycook with Mahler and with me when creating a station for Beethoven.

In any case, Pandora has clearly been designed around a taxonomy of musical attributes that are, at least, populated by its creators.  My question is–does this taxonomy also imply a “folksonomy” (re: Thomas Vanderwal)?

Does this represent “generative music”?  No–but perhaps it’s another step in a direction to consider what it means to adapt to listener “style”.  I use Pandora regularly, and think it’s tres cool.  Frankly, I’ve heard some people describe it as a ‘killer app’‘.  This is debatable, but I do know that Pandora & their “musical genome project” is significant in that it’s the closest app that I’ve found that tries to give me my tunes, my way.

Well, I took the next step in my exploration of experimental performance interfaces & Pd and decided to install Pd for Windows & tried the Wiimote.  I spent some time looking for a PC Wii to midi driver and eventually found one that worked called Glovepie.  I also installed midiYoke as a midi “patch bay” driver that links everything together.  I then used Pd to receive & process the midi data from the Wiimote.

In the process, I took a script for an oscillator that came bundled with Pd & modified it to accept midi channel data, using a piece of the test script that also comes bundled with Pd.   I then had to use & modify a script in Glovepie to have a different starting note value & made sure that Pd used midi yoke (channel 2) as an input device.   I’m only succeeding in making the wiimote increment & decrement pitches in a *very* rough way.  The process is akin, I think, to learning how to steer a tank.   However, it does seem to work.

I captured a short video with my phone so that you can see it for yourself.  this time I exported the movie as a .mov & posted it  @ http://www.adaptablearts.com/media/jason-wii-pd.mov .

Here’s the process that I used:

I recently attended a workshop on iphone app development.  In light of that presentation and our research of generative music in Spore, I thought this new iphone app developed by Brian Eno called “Bloom” was interesting.  I downloaded the app for the iTouch (I’m on Verizon, so I don’t use the iPhone).

http://createdigitalmusic.com/2008/10/08/hands-on-with-bloom-new-generative-iphone-app-by-eno-and-chilvers/

Here’s a Youtube clip that demos the app

The music the app produced was interesting–amazingly it sounded a *lot* like the stuff in Spore..  The app lets you either set a random setting to play music based on some basic choices, or you can interact by tapping the screen to make music.  Tapping the bottom of the screen generates lower pitches, while tapping higher generates higher pitches.

Some points:

• Bloom has a 4 octave range
• There are 9 different “moods” that can be selected to provide a different textural experience
• Each “mood” changes the tonic of the scale that plays.  One “mood” may have a range from C2 – C6, while another mood ranges from G2 – G6, etc.
• When you touch the iTouch, it generates pitches that then repeats a pattern using a delay.  There’s a setting called “evolve when idle” that I think takes a pattern & develops it so that something plays when you’re not.
• For example, the mood “Neroli” is in the key of G, and I’m pretty sure in a mixolydian mode.  The lower areas of the touch pad isn’t as responsive to changes in pitch as the middle & upper registers.

Listen to a sample of me playing the iTouch with the  “Neroli” mood!

• You’ll hear ambient background stuff that Bloom generates.  Also, the application was set to randomize the “mood” during playback (not sure what this does), and the delay was set a pretty short interval
• The first part of the clip is me trying to play a scale from the bottom up.  You can hear me playing the same notes even as I’m moving up the touchpad.  I actually moved from I-V-I for the first octave before moving up the scale.
• The later part of the clip is me “playing” the app.  Funny thing, last weekend I was in the middle of my music composition lesson when my tutor & I got into a jam session with Bloom.  He was on the guitar & we were just doing some ambient stuff.  I’ve *never* jammed with an ipod before.  That was a first.

Something arguably even more interesting is info on an iphone app called RjDj : http://www.rjdj.me .  It uses Pd and the iphone accelerometer & mic to sample audio.  Since I don’t have an iPhone, I can’t try this myself, but more info can be found here on this blog:
http://createdigitalmusic.com/2008/09/02/iphonetouch-roundup-control-art-snow-patrol-visualizers-recording-one-for-india/ .

I decided to explore the options given to us by Noatikl, by Intermorphic, for generative music first (since I’m waiting for my copy of Spore to arrive).  (By the way, in case you were wondering “Noatikl” is pronounced: “No-ti-cul”, emphasis on the first syllable.) What I found interesting about this software was it’s apparent ability to serve as both a stand-alone application as well as a VSTi (Virtual Studio Technology instrument) in a Digital Audio Workstation (DAW).  Noatikl apparently allows for Cubase, Garage Band, or Logic users to load Noatikl as a VSTi.   Although VSTi integration is standard for virtual instruments, what’s interesting about Noatikl is how it works as a VSTi.  When you instantiate Noatikl, you then have the ability to set the midi input of your tracks to come from Noatikl, using Cubase’s input transformer to filter for the correct channel.  There’s a brief tutorial on this integration @ http://www.vimeo.com/kantudok/.

Noatikl apparently also works as a “hyperinstrument” that can receive text/mouse input as well as midi cc events.  Noatikl claims to harmonize its voices based on midi data.  See: http://www.intermorphic.com/tools/noatikl/doc/noatiklUG_scripting_2.html for more of an explanation.

Although I have Cubase 4 running on Windows, my initial attempt at installing Noatikl was problematic.  Cubase doesn’t recognize the Noatikl VSTi, nor does the stand-alone version recognize incoming midi data.  I’ve reported the problem to Intermorphic, who has responded in a timely fashion, so I’m hoping to get the technical issues resolved when I can return to the problem.

Although Pd is certainly occupied my recent interest, there are other tools that support generative music that are also on my list.  These include:

Intermorphic’s Noatikl: the original progenetor of this software was Koan–which was pioneered by Brian Eno and written by SSEYO in 1996.  Intermorphic eventually acquired the rights to Koan.  Noatikl is promising since it works as a VSTi.

Intermorphic’s Mixtikl:  although this software is not yet released, it promises to offer support for mobile devices–including the iPhone.

Cycling 74’s Max/MSP: Max/MSP is a commercial analog of Pd, and has a long history in computer music creation.

Musigenesis: A fun-looking, shareware application that lets people of non-musical background create music.

Nodal: This promising music creation tool generates music in midi form, and is designed for the Apple.  The screenshots display a ‘circuit-diagram’ view of musical “events” and the “connection between events”.  This software appears to also let multiple people play music simultaneously.

The more I learn about Pd, the more I learn about its flexibility–and of course, the more that I also learn about its technical requirements.  Although Pd is Windows compatible, its preferred OS is linux.  I did manage to get Pd to run in a virtual install of Ubuntu on a WinXP machine–but of course Ubuntu did not recognize the audio hardware of the machine I was using.  My next step is to try the same process on a much faster machine with a more industry-standard audio device.  Since it’s an EMU audio card, I might have some luck.  The goal is to get a completely working version of Pd running in a way that supports looking at the work that’s been done with the tool–and to eventually learn the scripting that makes Pd go.

It would seem that Pure Data (Pd) has been used to build the dynamic soundtrack behind the ’soon-to-be-smash-hit’ Spore.  Brian Eno led this effort (as he led the generative music movement in the mid-90’s with Koan) and designed a tool that he calls, “The Shuffler” in Pd.  Apparently, this technology ensures that the player will never hear the same composition twice. In fact, we take a look at this Gamespy article, we can see a photo of the Pd script used in the game!

According to an MTV blog posting & video demo, as the player ‘builds’ parts of their environment (in the demo they show the building of a spaceship), the user’s choices drive the music they hear.  The team is calling this kind of music “procedural music”.  The presenter even references how user choices influence the use of counterpoint.  Take a look at a demo on the MTV blog to hear more.

Spore comes out next week–so of course my research will compel me to check it out.

Much has been done in the field of generative music.  The early work done with Pd in the mid-90’s continues to this day, and is one pillar of the technology systems that support generative music and algorithmic composition.

Although the technology that drives this next example is unclear, one example of a site that demonstrates algorithmic composition is Wolfram Tones, which takes computational algorithms (from Mathematica) and combines them with music theory to produce an interface that lets people “make” music in a variety of styles.  It would appear that Wolfram takes visual patterns as represented by “cellular automation”, flips the pattern on its side and represents the relationship between cells through musical ideas such as pitch.  This is, of course a gross-oversimplification, so please consult Wolfram for the details.

One interesting thing about Wolfram is that users can use the tool to create their own midi–which can either be saved on the site, emailed as a midi file–and even sent directly to your cell phone as ringtone.  How’s that for sharp?

Another interesting feature is that Wolfram builds in compositional controls that include not only pitch-mapping, instrumentation, meter & tempo, but also a “generator” that allows users to change the computational “rules” used to derive music from the pattern seen.

Of course, the next best feature of the software is that it’s free.

I will begin by examining style-based automated music composition, and how the ability to create this form of music can be applied to designing immersive educational technology.  I will be examining technologies such as Pd (Pure Data) and Max/MSP with the hope of understanding enough of the mechanics to assess their transportability to virtual environments.