Mitja Hmeljak

Audition and Vision

A. What are the fundamental psychological differences and similarities between audition and vision?
Are there any unbridgeable differences?
Are those differences basically due to the information itself or to our neurocognitive constraints?

B. What sort of goals should a music visualization system try to achieve?
Is there a reason to expect that music, an essentially temporal and auditory phenomenon should be amenable to visualization?
What advantages could be expected if it could be done?
Might visualization of music have any consequences for listener/lookers once they are accustomed to such a transformation?

 A: Features of auditory vs. visual perception

B: music visualization:

Techniques in Visualizing Music

Summarize techniques found in your readings for visualizing music in
(a) audio (i.e., derived from a performance) and
(b) symbolic (i.e., derived from the score) form.
What are the relative benefits of each?
What are their weak points?

Symbolic visualizations:


Visual Display of Rhythm

You are designing a system that will display visually the RHYTHM of an unfamiliar piece of music in real time.
The system should represent how a listener's sense of the rhythmic structure emerges and evolves in time.
You can base it on any theory of rhythm, for example, that of Lerdahl and Jackendoff (though they won't help with how rhythm emerges in real time).

Discuss both representation and implementation issues for the system.

(associational structure := "web" of motivic (timbral, etc) associations between potentially disjoint groups, not part of the rhytmic structure of a musical piece)

Beat spectrum visualization (from Foote)

Procedural Animation and Sonification

(1) give a detailed example of a non-trivial procedural animation, and then
(2) indicate what methods, algorithms, and heuristics you would use to produce an appropriate sonification of the animation.

Procedural Animation

define facet attribute flat integer; lfac:=.85; tfac:=.9;
flat_tri := {
   set facet ff flat max(ff.edge,dihedral)<.18;
   delete edges ee where length<.1/tfac and min(ee.face,flat);
   refine edges where length>.7*lfac and dihedral>.5;
trian := { n lfac; u; l lfac; u; g; w .001/tfac; t .05/tfac; flat_tri; u; g; V; u; }; 


mapping data to specific dimensions of sound:

Last updated: July 2, 2004
mitja (at) indiana . edu