Audio & Educational Technology

…So with all this attention to technologies that ‘revolve’ around or have otherwise ‘attempted’ to incorporate some form of style-based automated composition into a technology product, what have I learned?  And, how does this relate to educational technology?  These are two questions that I started with–hoping that these questions would lead to deeper questions of how to pursue a path that could lead to a project–and ultimately help uncover research questions that could guide me through a dissertation.

Again, where does this leave me?  What have I learned about style-based auto-comp. as has been produced in industry?  The products produced are promising in that:

• There are several products that have recently come to market that incorporate generative music–that is music that changes dynamically over time.  Examples of these include:  iphone apps such as Bloom!, composition tools such as Noatikl, and generative audio in game environments such as Spore.
• Although these generative music applications are interesting, to what extent do they relate to style?  This is a more complicated answer.  I’ve read a good chunk of David Cope’s first book, ‘Computers and Musical Style’, and I’ve also gotten through the first part of another book of his, ‘Virtual Music…Computer Synthesis of Musical Style’.  According to Cope, musical style has a structure that be related to linguistics.  Cope’s SPEAC system (a hierarchical analysis consisting of a: statement, preparation,extension, antecedent, & consequent) gives the ability to analyze (and with his system) generate music in a way that can adapt to musical “styles”.  Along with Cope’s notion of augmented transition networks (ATN’s), which are related to–and to some degree dependent upon Chomsky’s notion of Transformational Grammar, Cope has been successful in designing a system (EMI : Experiments in Musical Intelligence) that can reproduce musical styles to the likes of the greats–Bach, Chopin, Rachmaninoff, and the like…

Musical Style…

“Styles” in Cope’s view are also described through attention to musical ’signatures’, ‘tension-resolution’ logic, ’semantic meshing’ (a series of nested musical contexts that range from local notes & measures to phrases, periods & large-scale sections), and final musical ‘recombination’ (which theoretically allows for recombination of discrete passages of music that can be recombined in ways that are still stylistically “viable”, according to Cope).

But these issues of style uncover other questions:  Cope’s work focuses on reproducing ’styles’ of different composers.  How can these systems support generation of music to match the style of someone interacting with a system–can someone listening to music participate in the construction of what they hear?  Well, Pandora makes an attempt at doing this.  Pandora doesn’t generate music, but it does try to give the listener what they want to hear.  Here’s my theory on this–which is based on connectionist modeling concepts described in Eric Clarke’s book, “Ways of Listening”, and described in the first chapter of the book–Perception, Ecology, and Music.

• Here, Clarke provides a background for connectionist modeling from McLelland’s: Parallel Distributed Processing (Rumelhart and McLelland 1986) and describes a theoretical experiment that describes the ability of a ‘network’ to learn (based on programming & guided input) the musical preferences of listeners (who are listening to discrete melodies) and be theoretically be able to predict and classify new melodies using the preferences on which the model is based.  (I’m oversimplifying Clarke’s example, sorry Clarke).  However, he describes this process of predictability as hinging on assignment of “connection weights between ‘input units’ (basic definition rules/conditions–in this case intervals between notes in each melody) and ‘output units’ (’like or dislike’ of the listener)
• My theory is that Pandora is based on exactly this system of analysis of ‘connection weights’ that use listener ‘voting’ to determine how well Pandora ‘understands’ listener preference–e.g. “style” and is therefore able to predict with a degree of probability whether the listener will like what comes up next.  If this is true–then can we say this way of matching listener ‘preference’ is related to the idea of musical ’style’?

Next question:

How do these ideas of generative music as it relates to style tie back to design of educational technology?

I believe that a discussion about musical “style” would be incomplete without at least a mention of Pandora.  Pandora comes to us as a result of the Music Genome Project, by Tim Westergren.   Pandora comes to us as a result of musical analysis of “tens of thousands” of artists.  As Tim describes it, they analyzed,

“the musical qualities of each song one attribute at a time. This work continues each and every day as we endeavor to include all the great new stuff coming out of studios, clubs and garages around the world.”

Tim goes to describe how their team attempted to describe music according to unique “attributes” that they call “genes”.

“We ended up assembling literally hundreds of musical attributes or “genes” into a very large Music Genome. Taken together these genes capture the unique and magical musical identity of a song – everything from melody, harmony and rhythm, to instrumentation, orchestration, arrangement, lyrics, and of course the rich world of singing and vocal harmony.”

Professor Pennycook & I were discussing the mechanism they likely used to do this, which almost certainly involves meta-data rather than musical pattern-matching.   Pandora lets users create stations based on their favorite songs or artists, and will deliver songs/artists that match the attributes of the artist/song used to create the station.  You can also vote in favor of the selection played or against it.  In this way, you can “train” Pandora to match the songs that you think the station should play.  Pandora will actually send you an email reminding you to do this periodically.  One quesiton we had is whether our votes influenced the way the database tags the music.  One thing that we found curious was why in some cases when we created a station, the first song that plays is often not by the artist for which the station was named.  This happened with Dr. Pennycook with Mahler and with me when creating a station for Beethoven.

In any case, Pandora has clearly been designed around a taxonomy of musical attributes that are, at least, populated by its creators.  My question is–does this taxonomy also imply a “folksonomy” (re: Thomas Vanderwal)?

Does this represent “generative music”?  No–but perhaps it’s another step in a direction to consider what it means to adapt to listener “style”.  I use Pandora regularly, and think it’s tres cool.  Frankly, I’ve heard some people describe it as a ‘killer app’‘.  This is debatable, but I do know that Pandora & their “musical genome project” is significant in that it’s the closest app that I’ve found that tries to give me my tunes, my way.

After digging into Cope’s book a bit, I find the musical examples that describe how patterns of musical style & structure relate to their grammatical ‘counterparts’–and how these elements could be used to generate music.  (It also reinforced for me the usefulness in becoming more proficient in my understanding of theoretical & compositional grammar).

The process that Cope presents with phrasal structure & Automated Transition Networks is quite fascinating.  I’m curious if any companies have operationalized the idea.  I also wonder how far an approach like Cope’s can be taken to adapt to composition of complex works with several instrumental parts. I can imagine how automated generation would apply nicely to environments needing, several shorter segments (e.g. a computer game)…

My research survey in style-based automated composition is happening as a result of an independent study supervised by Professor Bruce Pennycook at the University of Texas @ Austin.  Professor Pennycook directed my attention to the work of David cope, who has done much of the grounding for this field.  I’m starting my review of David Cope’s work by reading “Computers and Musical Style”, published in 1991.

In reading about musical style representations, I suppose I’m not surprised to learn that there are overlaps between linguistic patterns in the way that we construct language to the way in which music can thought of as language.  To help illustrate this connection, Cope compares the diagrammatic parsing of sentences in a way that’s analogous to a diagrammatic parsing of music.

Cope also formulates a definition of ‘musical style’ for his book in a way that relates style with characteristics of music & musical structures.  He talks about style as,

“the identifiable characteristics of a composer’s music which are recognizably similar from one work to another.  These include, but are not limited to, pitch and duration…timbre, dynamics and nuance. “

Cope goes on to discuss other elements that have a “grammar” which constitute style when appear repeatedly across multiple works.  These include: melody, harmony, & counterpoint, as well as their connections to introductions, motives, transitions, modulations, & cadences.

I look forward to digging deeper in hopes that I can catch a glimpse of how these elements fit together & can be structurally represented as style, which as I believe Cope has gone on to demonstrate, can then be represented through the language of programming.