Is the waveform the optimal visual cue symbolizing music?
That is the question to which Rob Walker, a friend and a contributing writer to The New York Times Sunday Magazine, replied at length in his designobserver.com blog earlier this month. (Disquiet.com doesn’t have a “belatedly” tag, and this post is a good reason to consider adding such a thing.)
In the course of pondering the rise of the waveform as a visual icon for music, he generously quoted me and referenced this site multiple times: on comparing the waveforms of winners and losers of a Steve Reich remix contest (one example shown up top), on “looking at” the sound of fireworks, in discussion (on Twitter) with Ken Ueno on his album cover, and other related subjects.
The DesignObserver piece, titled “Stealth Iconography: The Waveform,” is a fascinating treatment of the subject, tying together the logos of sites like Freesound.org and Soundcloud.com, both of which feature a waveform, to the waveform’s appearance in jewelry and contemporary art. As he says toward the opening, the rise of the waveform “can be gauaged by the fact that it has inspired some to de-digitize it into the physical world.” Here, for example, is a proposed “waveform labeling system” suggested by designer Joshua Distler. Each track on a Björk album is identified by its waveform, title, and numerical sequence:
Just to continue the discussion, a few thoughts:
¶ Music v. Sound: It’s great that Freesound.org is recognized in the article for having employed the waveform prior to Soundcloud.com’s launch. It’s important not only as a means to track the image’s trajectory, but also because it broadens the waveform’s meaning. I entirely agree with Rob that the waveform is a useful visual icon for music — and I think its application is fundamentally broader, applying to sound. Music being organized sound — thank you, Varèse — the waveform’s utility in serving as a marker for both “music” and “sound” emphasizes what they have in common, and that any perceived distinctions occur along a continuum.
¶ Data Visualization: The waveform-image propagation has something to do with the exploding interest in data visualization — what exactly, I am not sure; I’m still wrapping my head around it. To talk about the sound waveform right now is to back up and talk about the rise of data visualization. No doubt to Edward Tufte’s dismay, the enginneer-mediated society in which we now live is flooded with nifty vibrant charts of all sorts of data — from cloud formations of frequently used terms in political debates, to blinky-blinky grids of product consumption, etc. It’s like we’re all part of some grand quantitative sociological research project. The problem being: these charts don’t necessarily have any specific meaning intended. We’re at a “look what we can do” stage that reduces much data visualization to the practical utility of a laptop screensaver or T-shirt design: colorful visuals, signifying nothing, with the added deficit of being so impossibly beautiful that it takes you awhile to realize there isn’t much to take away from them. They provide the illusion of meaning. We know our culture is technologically mediated. We sense that we can know more about our world by crunching the data and looking at it in new ways, that data visualization will answer old questions in new ways. The waveform seems like a succinct, utilitarian realization of this concept.
¶ The Equalizer: The waveform’s strongest pop-cultural precedent goes back at least to the days of the high-fidelity home stereo system, specifically to the standalone graphic equalizer, which let you control aspects of your sound at a much more nuanced level than just dual treble and bass knobs, and which in some systems showed you what the music “looked like” in terms of where sound levels peaked at various points along the spectrum.
¶ Sketching Sound: This idea of “what sound looks like,” moving beyond the waveform, is part of the synaesthesia inherent in so-called “sound art.” The idea is the reason I started the “Sketches of Sound” series here back in April 2010, and why I was especially glad that, amid drawings of kazoos and synthesizers, artist Gustavo Alberto Garcia Vaca opted to present a sine wave.
Read the full piece by Rob Walker on waveforms at designobserver.com. (There are some interesting comments on the post, too, about the relation between the waveforms and the EKG readout, and about Johann Wolfgang von Goethe’s praise for Ernst Chladni.)
A waveform doesn’t really tell very much about what’s going on. It shows energy distributed over time. Higher peaks mean more volume, but can otherwise be anything from white noise to a nice and harmonic sound. So as an icon it’s nice, but for me, reading the sound (of sound art, for instance) while mastering music I use a spectrogram. This also shows me the pitch distribution distributed over time. So a spectrogram would be a better substitute for a traditional score.
As for the soundcloud wave forms: these are all ‘normalized’: a quiet and a loud track will show the same height of peaks.
As for the bested Steve Reich remix waveforms : there is again very little to conclude there, except perhaps in a remote observation that music nowadays needs a constant energy level (the bested really also could be Metallica or Michael Jackson songs). A temporary lowering of energy level is not appreciated (dynamic variations in general, aren’t).
In a DJ’s point of view, a waveform helps him to see the structure of a track. Due modern music’s (sometimes) extreme loudness/mastering, often it’s not enough, so we color the waveform with the average volume in our application (DJ PLAYER app in the App Store). That gives more information.
I agree with Jos Smolders’ comment – a spectrogram provides a more intuitively informative visualization of sound than does an amplitude waveform. I regularly use the spectrogram in my music player (Foobar2000) to understand what’s going on in a sound.
Having said that, the waveform is fundamental to recorded and electronic sound, since it appears in various guises: Graph the movement of a tuning fork, and you will get a sinusoid waveform (displacement over time). Measure the air pressure near the tuning fork, and you get the same waveform (pressure over time). Use a microphone to record the sound, and the electric current shows the waveform again (voltage over time). Similarly, start with a square waveform generated by a synthesizer (voltage/time), connect it to a speaker (displacement/time) and you hear the same waveform (pressure/time).
So, one waveform describes different aspects of the technological transformation of sound, revealing a fundamental correspondence between its various manifestations. But one of the drawbacks is that some basic elements of sound, such as pitch, are not so easy to see in a waveform: Whereas a waveform combines different frequencies, a spectrogram separates them (see Fourier theory). In short, the waveform has only two dimensions (time and amplitude), but a spectrogram has three (time, amplitude and frequency). In our perception of music, of course, we experience many dimensions, and any visualization will fail to capture the richness of our experience.
I agree with you, Guy, but your story of the pitchfork works only one way: from source to representation. In the early days of digital representation of sound there were these ‘mountains’ of sound. Actually that is 3D representation where the x-axis represents time, the y-axis is amplitude and the z-axis is frequency. The modern spectrograms use the x and z-axes, whereas the value or height of the y-axis is represented by a color (dark or blue = low and bright red = high volume).
Respectfully, Jos, I still hold with what I wrote above. My point about the waveform is that whatever shape it is, it remains largely the same as it is transformed from one medium to another. This applies whether recording or synthesizing. I gave two examples with different waveforms (recording a sine wave and generating a square wave). I argue that it does work both ways, and that this is why the waveform is a significant representation of sound.
Maybe I wasn’t clear about the spectrogram – I meant that it represents three ‘dimensions’ of sound, as opposed to the waveform’s two dimensions, not that is a ‘3D’ model. The spectrograms that I use work in the way you described, using colour to represent amplitude (I am also familiar with those ‘mountains of sound’ that show the same information as a 3D graph plot). My point is that the spectrogram is more useful because it shows more information than the waveform.