Is There Such a Thing as a Sonic QR Code?

One needn't watch the new Spider-Man movie for a possible answer.

20140430-shazampdf

There are at least two things that Sony Pictures marketing executives did not consider when preparing a cross-promotion between its new Spider-Man film and the song-identification app Shazam. I first read about this promotion this morning on io9.com, because pretty much the first thing I read every morning is Morning Spoilers on io9.com. The film in question, The Amazing Spider-Man 2, opens this Friday, May 2, in the United States. Expecting extended discussion about Peter Parker’s doomed romance with Gwen Stacy or the rise of his frenemy Harry Osbourne to lead the high-tech firm founded by his father, instead there was news of an intriguing little digital-audio phenomenon.

The Sony-Shazam promotion involves viewers of the Spider-Man movie waiting until the end credits, during which the Alicia Keys song “It’s On Again” is heard. Viewers can then use the Shazam app to identify the song. Doing so brings up a special opportunity to add, for free, photos that hint at members of the Sinister Six — villain characters from Sony’s rapidly expanding Spider-Man franchise — to their personal photo galleries. (It should be noted that the Keys song is itself a sort of cross-promotion. It’s full credit is: Alicia Keys feat. Kendrick Lamar – “It’s On Again.”)

The first of these things that Sony Pictures may not have considered is that Shazam shares a name with a superhero from a rival comics publisher, DC. Would it have been too difficult to sign up, instead, with Soundhound, or MusixMatch, or the elegantly named Sound Search for Google Play, among other song-identification services? Perhaps none of this matters. Sony is already engaged in a cold war with other studios among whom the Marvel universe of characters is subdivided. A second-tier, if beloved, character from another universe entirely means nothing when there are already two Quicksilvers running around in your own. For reference, below is an uncharacteristically stern Shazam, drawn by Jeff Smith (best known for his work on Bone):

20140430-shazam

In any case, the second and more pressing matter is that one needn’t stay until the end credits of the new Spider-Man film to activate the Shazam code with the Alicia Keys song. One needn’t even see the Spider-Man film, let alone wait for it to open in a theater near you. Right now, two full days before the film’s release in the United States, you can pull up the Alicia Keys video on YouTube, and the Shazam app on your phone will recognize that as the correct song, and your phone will, indeed, then provide you with the prized photos. In fact, at this point you don’t even need to do that, since the photos have already proliferated around the Internet. (See them at comingsoon.net and at the above io9.com link.)

But an interesting question arises, which is: How different would the Alicia Keys song played during the end credits have to be from the original version of the song for only the credits rendition to be recognized by Shazam as the correct one to cough up the Sinister Six photos? More to the point, can a specific version of a song function as the sonic equivalent of a QR code. QR codes are those square descendents of zebra codes, such as the one shown below. The “QR” stands for “quick response.” They can contain information such as a URL, which when activated by a phone’s camera can direct the phone’s browser to a particular web page. This QR code links, only semi-helpfully, to the web page on which this article originally appeared:

20140430-qrsonic

Of course, from a procedural standpoint, Sony could have gotten around this alternate-version approach by having the song only be available in the credits, but that would have cut into sales of the soundtrack album — which would either have to lack the song entirely, or have its release delayed until several weeks after the film’s debut.

The recipes of these different song-identification apps, such as Shazam and its arch enemy Soundhound, are closely guarded secrets. Enough information is provided to allow for developer-level discussion, but ultimately the apps’ success (both in terms of successful-identification statistics and user adoption) depend on the how-to being at least semi-obscured. But there is quite a bit of information out there, including a 2003 academic paper by Shazam co-founder Avery Li-Chun Wang outlining the company’s approach at the time (PDF), which I found thanks to a October 2009 article by Farhad Manjoo on Slate.com. The summary at the opening of the paper reads as follows:

We have developed and commercially deployed a flexible audio search engine. The algorithm is noise and distortion resistant, computationally efficient, and massively scalable, capable of quickly identifying a short segment of music captured through a cellphone microphone in the presence of foreground voices and other dominant noise, and through voice codec compression, out of a database of over a million tracks. The algorithm uses a combinatorially hashed time-frequency constellation analysis of the audio, yielding unusual properties such as transparency, in which multiple tracks mixed together may each be identified. Furthermore, for applications such as radio monitoring, search times on the order of a few milliseconds per query are attained, even on a massive music database.

The gist of it, as summarized in handy charts like the one up top, appears to be that an entire song is not necessary for identification purposes, that only key segments — “higher energy content,” he calls it — are required. At least in part, this allows for songs to be recognizable above the din of everyday life: “The peaks in each time-frequency locality are also chosen according amplitude, with the justification that the highest amplitude peaks are most likely to survive the distortions listed above.” It may also explain why much of my listening, which being ambient in nature can easily be described as “low energy content,” is often not recognized by Shazam or any other such software. As a side note, this gets at how the human ear listens differently than a microphone. The human ear can listen through a complex noise and locate a a particular subset, such as a conversation, or a phone ringing, or a song for that matter.

Now, of course, there’s a difference between the unique attributes of emerging technologies and the desired results of marketing initiatives. Arguably all that Sony wanted to come out of its Shazam cross-promotion was to get word out about Spider-Man, and to buy some affinity for the Sinister Six with a particular breed of fan, and to that end it has certainly succeeded. Perhaps it also hoped to gain a little tech cred in the process, even if that cred is more window dressing than truly innovative at a technological level.

Still, the idea of a song as a true QR code lingers. Perhaps Harry Osbourne and Peter Parker could team up and develop a functional spec.

Tape Delay in Progress

An experiment by Dance Robot Dance

Often as not, some of the best listening on SoundCloud is simply of people trying their hand at new tools. One person’s test run of a fresh plug-in is someone else’s arm-chair, near-first-hand peek at the creative process. Take Brian Biggs, aka Dance Robot Dance, who is documenting his efforts at old-school tape delay. The details, as he describes in this sample post, are fairly technical —

Figured out how to run my signal through Ableton to my Tascam reel-to-reel deck and create a real tape delay with controllable feedback. “tape delay” is edited down from the hour or so that I recorded. You can hear both guitars as well as an electric piano that is the “Electric” synth in Ableton, routed to the tape deck via send/return and back again to an Ableton Live where the send of its return channel is then fed back to the tape deck again. “oustide over there” is made up of pieces of “tape delay,” where these parts were slowed down and reversed with an recording from my back yard and soaking of Eos reverb. All the delays you hear on these tracks are from the Tascam 34 deck.

— yet the resulting music is anything but, layers of melting guitar forming slow motion clouds of melodic gestures and fledgling drones on pause, throughout which are the gentle, centering chirps of neighboring birds. The track heard here is “Outside Over There.” The other piece mentioned is also on SoundCloud.

Track originally posted for free download at soundcloud.com/dance-robot-dance. More from Biggs, who is based in Philadelphia, Pennsylvania, at dancerobotdance.com, where he will almost certainly soon be posing a more detailed document of his progress.

What’s More Frightening Than Breaking Glass?

The drone that subsumes it

The sheer brittle cascade of “KEYS” by Radio Free Ul-quoma is enough to cement it in memory. It’s like glass breaking in a constant swirl of activity, over two minutes of crackling noise. A deep swell of a drone seems to arise out of the constant activity, eventually subsuming the crackle into its own lulling haze. When the crackle returns, it can barely manage the briefest of recurrences, before fading again. The stasis is even more frightening that the destruction.

Track originally posted for free download at soundcloud.com/radiofreeul-quoma. Found at the recommendation of Jmmy Kpple.

Aliens + Interrogative Music @ SETI

Video of my 20-minute talk on the Disquiet Junto (plus Ed Frenkel's and a Q&A) from April 22, 2014

When you reference Ezra Pound’s statement that “The artist is the antennae of the race”at SETI, the “antennae” part takes on a whole richer meaning. SETI hosts weekly colloquium at its Mountain View, California, offices, and a few times a year those talks put aside interstellar science and are, instead, organized by SETI’s artist-in-residence. Right now that artist-in-residence is Charles Lindsay, and he invited me and mathematician Edward Frenkel, professor at UC Berkeley and author of the well-received book Love and Math: The Heart of Hidden Reality, to talk last week. This is a lightly edited video of the talk. The video is lo-fi, but the sound is good. The format of the video is that, after a short introduction by Lindsay, I talk for 20 minutes, then Frenkel talks for about 20 minutes, and then there’s an extended conversation, between the three of us, and then involving questions from the audience. I’m quite proud to have had my humanity thrown back at me during the Q&A by Lawrence Doyle. Also in the audience was SETI co-founder Jill Tarter.

My talk is about what I sometimes refer to as “Networked Creativity,” which I do here. Other times I call it, simply, “Doing Stuff Together Separately,” or “Ambient Participation.” In the talk I walk through the activity and development of the Disquiet Junto, the weekly music projects I’ve moderated since the first week of 2012. In the course of my talk I play five examples of results of these weekly music projects. The one by Mark Ward was particularly resonant at SETI, because it involved sounds recorded from Voyager 1 as it left the solar system. These are the five musicians whose tracks were included in my SETI talk:

Project 0036 / Grzegorz Bojanek / Poraj, Poland

Project 0002 / J Butler / Pittsburgh, Penn.

Project 0089 / Mark Ward / Sheffield, England

Project 0107 / Naoyuki Sasanami / Tokyo, Japan

Project 0066 / Jess Lemont / Milwaukee, Wis.

The talk was somewhat tailored for SETI, so with that in mind, here is a transcript of my opening statement, just for context:

“I just want to say thanks, first, to SETI for inviting me, to Ed for sharing the stage with me, and to Charles for setting the whole thing up. It is very much appreciated. I recently had a book published, as Charles mentioned, and so my publisher would like to thank you, as well. [Jill Tarter asks from the audience, “Do you have copies here?”] Just this one, that Charles brought, because he’s much smarter about these things than I am. I learn a step at a time.

So, when Charles asked me to speak at SETI, he asked what I wanted to talk about. I do a lot of different things, all circulating around the intersection of sound, art, and technology, and I recently published this book. I write for places like Nature, I have my own website, Disquiet.com, since 1996, I teach a course about the role of sound in the media landscape. I gave a lot of thought to SETI, to what would be appropriate. I thought about the central focus of communication to what you do here, I thought about indirect and chance communication, especially communication that isn’t inherently verbal. I thought about the interconnected arrays of radio telescopes, and about the network effect of [email protected], that pioneering achievement.

So, in turn, I welcome this, ultimately, as an opportunity to speak about a specific thing I’ve been doing for a while now, an ongoing and expansive networked community of hundreds of musicians around the world, and sound artists, that I initiated at the start of 2012. I should say that the project is now a I little over two years old, but I’m still learning to speak about it because of all the investiagations I’m involved in, this is the one I probe the least in terms of trying to figure out how it works. So this talk is me walking around it, trying to figure it out, because I don’t want to totally demistify it, but I do want to share what I’ve learned these past two years about working with hundreds of musicians, upwards of 450 at this point, around the world each week.”

The structure of the talk is as follows: I explain how the Junto works. I walk through three different projects (0036, in which we made music that explore how classical music connects with abstract expressionism; 0002, in which sounds of fog horns and trains are combined; and 0089, the Voyager 1 piece). I give an overview of the range of projects, 120 weekly ones as of when the talk was given. I talk about how this work arose from my enjoyment of interviewing musicians and artists — how an interview involves asking 50 questions of one person, and the Junto in turn is like asking one question of 50 people. I discuss how many Junto projects involve forging partnerships, and then how each project probes ideas. By way of example, I play music from a project that involves exploring ideas from my book on Aphex Twin’s album Selected Ambient Works Volume II (project 0107 above). I share a bad joke about experimental music concerts — that everyone in the audience is also an experimental musician — and try to turn it on its head and look at the positive aspects of that notion of community. I then express misgivings about the term “experimental music” and discuss how I’m slowly exploring an alternate phrase, “interrogative music,” to get away from the broad generalization of an experiment and to get closer to the purpose, the intent, the pursuit. I talk through examples of online music communities that came before and after the Disquiet Junto. I talk about the notion of “parallel play” in childhood development, and how it relates to doing something with the knowledge that someone else is doing it nearby (even if “nearby” means across the world, but also in the same network of creative individuals). I note the term “acoustemology” and talk about what the “sonic potential energy” of the Internet might be. I play a fifth and final Junto piece, in which members commune with a Junto regular who passed away a year ago this month. I talk about the Ezra Pound quote regarding how “Artists are the antennae of the race.” And in closing I talk about how communities of creative individuals — whether musicians, or artists, or scientists — set the stage for their participants to achieve greater things than they might have individually, even if they don’t directly collaborate with each other.

And at the end of the talk I mention that the next project, the 121st, would explore ideas from Frenkel’s book.

Like the Junto, this talk is a work in progress, but it’s a pretty good snapshot of where my head is at right now.

By the way, if the “antennae” of Ezra Pound’s statement “The artist is the antennae of the race”takes on new meaning at SETI, this is all the more the case when you’re sharing the stage with a mathematician who, as an ethnic Jew raised in Russia, suffered from intense state-sanctioned anti-semitism that clearly took Orwell and Kafka as playbooks. Frenkel, during his SETI lecture, doesn’t dwell on the anti-semitism he experienced as a teenage math prodigy raised in Russia, though it is at the core of his compelling and educative book Love and Math. This particular connection to Pound is one I hadn’t made until it came up in conversation on Tuesday. Such additional connections and layers of meaning are the natural result of a discussion by individuals who have quite different pursuits, and Tuesday, for me at least, was no disappointment in that regard. It was a highly enjoyable conversation.

Video posted at youtube.com. It’s also online at SETI’s plus.google.com page. More on Frenkel’s book at loveandmathbook.com. More from Lindsay at charleslindsay.com. Visit SETI at seti.org.

Disquiet Junto Project 0121: Math Rock

Two projects of varying complexity inspired by Edward Frenkel's book Love and Math

20140425-frenkel2

Each Thursday at the Disquiet Junto group on SoundCloud.com a new compositional challenge is set before the group’s members, who then have just over four days to upload a track in response to the assignment. Membership in the Junto is open: just join and participate.

Tracks by participants will be added to this playlist as the project proceeds:

Planning the week’s Disquiet Junto made my head hurt, in a good way, like that time I played board games with drunk NSA contractors in New Orleans. This week is somewhat unusual: in place of the standard one project we have two interrelated projects (“0121a: Jigsaw Composition”; “0121b: Braid Abstractions”). You’re encouraged to do just one of them, though certainly feel free to try both — there is one that will take less time, and one for those who have more time to dedicate. We may again employ such a multi-project approach in the future.

Both this week’s projects draw from ideas explored in the book Love and Math: The Heart of Hidden Reality by UC Berkeley professor Edward Frenkel, with whom I spoke at the SETI Institute in Mountain View, California, on April 22, at the invitation of the SETI artist-in-residence, Charles Lindsay. In his book Love and Math, Frenkel quotes Henry David Thoreau, “We have heard about the poetry of mathematics, but very little of it has yet been sung.” This week we’ll help some small bit of Frenkel’s math to sing. Frenkel weighed in on the development of both this week’s Junto projects. I think we’ll see more projects come out of that SETI event in the future, likely drawing upon SETI data.

These projects were published in the late afternoon, California time, on Thursday, April 24, with 11:59pm on the following Monday, April 28, 2014, as the deadline.

These are the instructions that went out to the group’s email list (at tinyletter.com/disquiet-junto).

Note that there are two project options this week: 0121a is a little more simple than 0121b.

. . .

Disquiet Junto Project 0121a: Jigsaw Composition

Several times in his book Love and Math, the author Edward Frenkel uses the metaphor of a jigsaw puzzle to describe the process of solving a mathematical problem. As he writes, “It’s useful to think about mathematics as a whole as a giant jigsaw puzzle, in which no one knows what the final image is going to look like.” The following exercise explores that idea in sonic form. There are 4 steps:

Step 1: Choose a recent piece of your own music, and select a 20-second segment, preferably one with a fairly clear melodic component.

Step 2: Make four sets of subsections of that track. Set A is the track broken into 60 sections of equal length. Set B is the track broken into 40 sections of equal length. Set C is the track broken into 20 sections of equal length. Set D is the track broken into 10 sections of equal length.

Step 3: Randomize each of those sets and for each set create a track of those segments playing in random order (with no repetition of a segment within a given set).

Step 4: Create a track constructed as followed: Set A followed by Set B followed by Set C followed by Set D followed by the original source segment. You shouldn’t add any sounds to the sequence, though you can fade in at the beginning of the complete track and fade out at the end. The finished work should be 1:40 in length. In this final resulting track, the listener should experience the finished piece slowly coming into sonic view.

. . .

Disquiet Junto Project 0121b: Braid Abstractions

This project has 5 steps. The project explores the subject of “braid groups” that Edward Frenkel writes about in his book Love and Math. Read through all these steps completely prior to undertaking them in sequence. Your choices in steps 1 and 2, for example, may be influenced by your understanding of the intended outcomes of steps 3, 4, and 5.

Step 1: Choose four distinct sounds. They should be soft and tonal in nature.

Step 2: Choose four different note values within a single octave. (In this project, working with closely proximate note values — such as microtones — is not recommended, but certainly try it out if you would like to.)

Step 3: Draw six different diagrams, each a variation on the following sample diagram provided at the end of this step. Each diagram consists of two plates and four threads. Each thread starts out at one point on a plate and ends at one point on the opposing plate. No single point on a plate ever has more than one thread connected to it. (A thread can, certainly, move from a point on one plate to the same point on the opposing plate.) The four threads represent the distinct sounds you have chosen (color-coding them may be helpful). The four points at the start and end plates represent the four different note values you have chosen. Here is the sample diagram showing three threads. Note that you’ll, of course, be employing four threads: http://goo.gl/NQYysE.

20140425-frenkel1

Step 4: Record short segments representing each of the six diagrams. Each segment should be the same length, advisedly between four and six seconds. In each note, the four tones will move from the initial note value of the relative position on the first plate to the note value represented by the relative position on the second plate.

Step 5: Create a single track consisting of a randomly generated sequence of these segments (preferably with no consecutive repetition of a segment). You shouldn’t add any sounds to the sequence, though you can fade in at the beginning of the complete track and fade out at the end. The length of the track should be roughly between 45 seconds and a minute and a half.

Note: The image is sourced from Frenkel’s book, with his permission.

. . .

Deadline: Monday, April 28, 2014, at 11:59pm wherever you are.

Length: The length of your recording should be 1:40 if you do project 0121a and roughly between 45 seconds and a minute and a half if you do project 0121b.

Information: Please when posting your track on SoundCloud, include a description of your process in planning, composing, and recording it. This description is an essential element of the communicative process inherent in the Disquiet Junto.

Title/Tag: When adding your track to the Disquiet Junto group on Soundcloud.com, please include the term “disquiet0121-frenkeljunto” in the title of your track, and as a tag for your track.

Download: It is preferable that your track is set as downloadable, and that it allows for attributed remixing (i.e., a Creative Commons license permitting non-commercial sharing with attribution).

Linking: When posting the track, please be sure to include this information:

More on this 121st Disquiet Junto endeavor — “Two projects of varying complexity inspired by Edward Frenkel’s book Love and Math”— at:

Disquiet Junto Project 0121: Math Rock

More on the Disquiet Junto at:

The Disquiet Junto Project List (0001 – 0574 …)

Join the Disquiet Junto at:

http://soundcloud.com/groups/disquiet-junto/

The “braid” image associated with this project is sourced from Edward Frenkel’s book Love and Math, with his permission. More on the book at:

http://www.loveandmathbook.com/