Audio & Educational Technology

Well, I took the next step in my exploration of experimental performance interfaces & Pd and decided to install Pd for Windows & tried the Wiimote.  I spent some time looking for a PC Wii to midi driver and eventually found one that worked called Glovepie.  I also installed midiYoke as a midi “patch bay” driver that links everything together.  I then used Pd to receive & process the midi data from the Wiimote.

In the process, I took a script for an oscillator that came bundled with Pd & modified it to accept midi channel data, using a piece of the test script that also comes bundled with Pd.   I then had to use & modify a script in Glovepie to have a different starting note value & made sure that Pd used midi yoke (channel 2) as an input device.   I’m only succeeding in making the wiimote increment & decrement pitches in a *very* rough way.  The process is akin, I think, to learning how to steer a tank.   However, it does seem to work.

I captured a short video with my phone so that you can see it for yourself.  this time I exported the movie as a .mov & posted it  @ http://www.adaptablearts.com/media/jason-wii-pd.mov .

Here’s the process that I used:

I recently attended a workshop on iphone app development.  In light of that presentation and our research of generative music in Spore, I thought this new iphone app developed by Brian Eno called “Bloom” was interesting.  I downloaded the app for the iTouch (I’m on Verizon, so I don’t use the iPhone).

http://createdigitalmusic.com/2008/10/08/hands-on-with-bloom-new-generative-iphone-app-by-eno-and-chilvers/

Here’s a Youtube clip that demos the app

The music the app produced was interesting–amazingly it sounded a *lot* like the stuff in Spore..  The app lets you either set a random setting to play music based on some basic choices, or you can interact by tapping the screen to make music.  Tapping the bottom of the screen generates lower pitches, while tapping higher generates higher pitches.

Some points:

• Bloom has a 4 octave range
• There are 9 different “moods” that can be selected to provide a different textural experience
• Each “mood” changes the tonic of the scale that plays.  One “mood” may have a range from C2 – C6, while another mood ranges from G2 – G6, etc.
• When you touch the iTouch, it generates pitches that then repeats a pattern using a delay.  There’s a setting called “evolve when idle” that I think takes a pattern & develops it so that something plays when you’re not.
• For example, the mood “Neroli” is in the key of G, and I’m pretty sure in a mixolydian mode.  The lower areas of the touch pad isn’t as responsive to changes in pitch as the middle & upper registers.

Listen to a sample of me playing the iTouch with the  “Neroli” mood!

• You’ll hear ambient background stuff that Bloom generates.  Also, the application was set to randomize the “mood” during playback (not sure what this does), and the delay was set a pretty short interval
• The first part of the clip is me trying to play a scale from the bottom up.  You can hear me playing the same notes even as I’m moving up the touchpad.  I actually moved from I-V-I for the first octave before moving up the scale.
• The later part of the clip is me “playing” the app.  Funny thing, last weekend I was in the middle of my music composition lesson when my tutor & I got into a jam session with Bloom.  He was on the guitar & we were just doing some ambient stuff.  I’ve *never* jammed with an ipod before.  That was a first.

Something arguably even more interesting is info on an iphone app called RjDj : http://www.rjdj.me .  It uses Pd and the iphone accelerometer & mic to sample audio.  Since I don’t have an iPhone, I can’t try this myself, but more info can be found here on this blog:
http://createdigitalmusic.com/2008/09/02/iphonetouch-roundup-control-art-snow-patrol-visualizers-recording-one-for-india/ .

The more I learn about Pd, the more I learn about its flexibility–and of course, the more that I also learn about its technical requirements.  Although Pd is Windows compatible, its preferred OS is linux.  I did manage to get Pd to run in a virtual install of Ubuntu on a WinXP machine–but of course Ubuntu did not recognize the audio hardware of the machine I was using.  My next step is to try the same process on a much faster machine with a more industry-standard audio device.  Since it’s an EMU audio card, I might have some luck.  The goal is to get a completely working version of Pd running in a way that supports looking at the work that’s been done with the tool–and to eventually learn the scripting that makes Pd go.

It would seem that Pure Data (Pd) has been used to build the dynamic soundtrack behind the ’soon-to-be-smash-hit’ Spore.  Brian Eno led this effort (as he led the generative music movement in the mid-90’s with Koan) and designed a tool that he calls, “The Shuffler” in Pd.  Apparently, this technology ensures that the player will never hear the same composition twice. In fact, we take a look at this Gamespy article, we can see a photo of the Pd script used in the game!

According to an MTV blog posting & video demo, as the player ‘builds’ parts of their environment (in the demo they show the building of a spaceship), the user’s choices drive the music they hear.  The team is calling this kind of music “procedural music”.  The presenter even references how user choices influence the use of counterpoint.  Take a look at a demo on the MTV blog to hear more.

Spore comes out next week–so of course my research will compel me to check it out.

I will begin by examining style-based automated music composition, and how the ability to create this form of music can be applied to designing immersive educational technology.  I will be examining technologies such as Pd (Pure Data) and Max/MSP with the hope of understanding enough of the mechanics to assess their transportability to virtual environments.