Audio & Educational Technology

This has been a very enjoyable semester.  I feel like I’ve covered a lot of ground.  With Dr. Pennycook’s guidance, I’ve done a survey of the world of generative music–and even though I feel as if I’ve learned much, I also feel like I’ve just scratched the surface.   I think I’ve come to the place where the next step is one that takes the theories and knowledge provided by the people I’ve studied so far–namely Cope (and Hoffsteader), Clarke, (just a bit) of Lerdahl, and of course Dr. Pennycook, and venture into the exciting world of generative music technology through Max/MSP.

In addition to taking an experimental performance interfaces course, I will concurrently be doing a project for Dr. Liu’s multimedia class, where a team of us will be designing a micro-simulation using the Unity 3d game engine. Here’s a few questions I came up with to begin this exploration…

• How do we apply the principles of telling stories to designing with Max/MSP?
• How does Max/MSP work exactly?
• How do we build end-user control in generative music?
• How does Max/MSP take in end-user data from Unity?
• How will Max/MSP output music through Unity?
• How do we build sound design in Unity?
• How does Unity record and handle end-user data?
• What coding will need to be done through Unity to allow these systems to interact?
• In short, how will these 2 work and play together?

And yes, the reading will continue and I plan to take a sneak peak at these systems between now & the end of January…So, there’s more to come…

Where’s the link?  Let me begin by revisiting the idea of musical style.  Cope has concretely established that it’s possible to reproduce musical style.  Pandora has demonstrated that it can cue into listener style.  What about someone who is playing the role of a learner?  Can we take the notion of ’style’ and apply it to learning? Can the notion of generative music be connected to the notion of choices that a learner makes?  What if we could identify key ‘decision points’ that identify with the idea of ‘progress’ through a learning event–and let the learner ‘create’ the sonic environment in which they learn?  What if we take this one step further–and build the music and audio in such a way that as the learner progresses through the learning experience they are given subtle cues and shifts in what they hear that impact their perception of the learning experience?

How can we begin to address these questions–and what can we use as a technical basis for this examination?

Lessons from Spore & Bloom

One important facet that perhaps sheds light on this question is our examination of Eno’s audio system in Spore & Bloom.  Spore is successful at providing the user with basic “control” over what they hear in key “building” stages of the game.  In a sense, the user–without musical knowledge–can create music.

The same is true in Bloom.  Bloom lets the listener create their own musical experience through the use of “moods” to provide an ambient musical bed, while provide different scalar modes that let them play patterns by touching the ipod or iphone’s screen.  The program then generates music based on these patterns–in much the same way that Spore generates music in the game.  In this sense, the user “perceives” they are controlling what they here–which is true–the patterns a user plays is incorporated by Bloom which continues to generates music with similar patterns.  However, one important facet here is that Bloom is providing a musical framework–a structure in which this happens.  When you “play” Bloom, it’s nearly impossible to create dissonance that doesn’t resolve, since everything is related harmonically.  In a sense, user choice drives what they hear–and what they hear makes “sense”.

So, how can generative music be designed in a way that “makes sense” or even provides a “meaningful” experience for the learner?