New Disquietude podcast episode: music by Lesley Flanigan, Dave Seidel, KMRU, Celia Hollander, and John Hooper; interview with Flanigan; commentary; short essay on reading waveforms. • Disquiet.com F.A.Q.Key Tags: #saw2for33third, #field-recording, #classical, #juntoElsewhere: Twitter, SoundCloud, Instagram

Listening to art. Playing with audio. Sounding out technology. Composing in code. Rewinding the soundscape.

Is There Such a Thing as a Sonic QR Code?

One needn't watch the new Spider-Man movie for a possible answer.

20140430-shazampdf

There are at least two things that Sony Pictures marketing executives did not consider when preparing a cross-promotion between its new Spider-Man film and the song-identification app Shazam. I first read about this promotion this morning on io9.com, because pretty much the first thing I read every morning is Morning Spoilers on io9.com. The film in question, The Amazing Spider-Man 2, opens this Friday, May 2, in the United States. Expecting extended discussion about Peter Parker’s doomed romance with Gwen Stacy or the rise of his frenemy Harry Osbourne to lead the high-tech firm founded by his father, instead there was news of an intriguing little digital-audio phenomenon.

The Sony-Shazam promotion involves viewers of the Spider-Man movie waiting until the end credits, during which the Alicia Keys song “It’s On Again” is heard. Viewers can then use the Shazam app to identify the song. Doing so brings up a special opportunity to add, for free, photos that hint at members of the Sinister Six — villain characters from Sony’s rapidly expanding Spider-Man franchise — to their personal photo galleries. (It should be noted that the Keys song is itself a sort of cross-promotion. It’s full credit is: Alicia Keys feat. Kendrick Lamar – “It’s On Again.”)

The first of these things that Sony Pictures may not have considered is that Shazam shares a name with a superhero from a rival comics publisher, DC. Would it have been too difficult to sign up, instead, with Soundhound, or MusixMatch, or the elegantly named Sound Search for Google Play, among other song-identification services? Perhaps none of this matters. Sony is already engaged in a cold war with other studios among whom the Marvel universe of characters is subdivided. A second-tier, if beloved, character from another universe entirely means nothing when there are already two Quicksilvers running around in your own. For reference, below is an uncharacteristically stern Shazam, drawn by Jeff Smith (best known for his work on Bone):

20140430-shazam

In any case, the second and more pressing matter is that one needn’t stay until the end credits of the new Spider-Man film to activate the Shazam code with the Alicia Keys song. One needn’t even see the Spider-Man film, let alone wait for it to open in a theater near you. Right now, two full days before the film’s release in the United States, you can pull up the Alicia Keys video on YouTube, and the Shazam app on your phone will recognize that as the correct song, and your phone will, indeed, then provide you with the prized photos. In fact, at this point you don’t even need to do that, since the photos have already proliferated around the Internet. (See them at comingsoon.net and at the above io9.com link.)

But an interesting question arises, which is: How different would the Alicia Keys song played during the end credits have to be from the original version of the song for only the credits rendition to be recognized by Shazam as the correct one to cough up the Sinister Six photos? More to the point, can a specific version of a song function as the sonic equivalent of a QR code. QR codes are those square descendents of zebra codes, such as the one shown below. The “QR” stands for “quick response.” They can contain information such as a URL, which when activated by a phone’s camera can direct the phone’s browser to a particular web page. This QR code links, only semi-helpfully, to the web page on which this article originally appeared:

20140430-qrsonic

Of course, from a procedural standpoint, Sony could have gotten around this alternate-version approach by having the song only be available in the credits, but that would have cut into sales of the soundtrack album — which would either have to lack the song entirely, or have its release delayed until several weeks after the film’s debut.

The recipes of these different song-identification apps, such as Shazam and its arch enemy Soundhound, are closely guarded secrets. Enough information is provided to allow for developer-level discussion, but ultimately the apps’ success (both in terms of successful-identification statistics and user adoption) depend on the how-to being at least semi-obscured. But there is quite a bit of information out there, including a 2003 academic paper by Shazam co-founder Avery Li-Chun Wang outlining the company’s approach at the time (PDF), which I found thanks to a October 2009 article by Farhad Manjoo on Slate.com. The summary at the opening of the paper reads as follows:

We have developed and commercially deployed a flexible audio search engine. The algorithm is noise and distortion resistant, computationally efficient, and massively scalable, capable of quickly identifying a short segment of music captured through a cellphone microphone in the presence of foreground voices and other dominant noise, and through voice codec compression, out of a database of over a million tracks. The algorithm uses a combinatorially hashed time-frequency constellation analysis of the audio, yielding unusual properties such as transparency, in which multiple tracks mixed together may each be identified. Furthermore, for applications such as radio monitoring, search times on the order of a few milliseconds per query are attained, even on a massive music database.

The gist of it, as summarized in handy charts like the one up top, appears to be that an entire song is not necessary for identification purposes, that only key segments — “higher energy content,” he calls it — are required. At least in part, this allows for songs to be recognizable above the din of everyday life: “The peaks in each time-frequency locality are also chosen according amplitude, with the justification that the highest amplitude peaks are most likely to survive the distortions listed above.” It may also explain why much of my listening, which being ambient in nature can easily be described as “low energy content,” is often not recognized by Shazam or any other such software. As a side note, this gets at how the human ear listens differently than a microphone. The human ear can listen through a complex noise and locate a a particular subset, such as a conversation, or a phone ringing, or a song for that matter.

Now, of course, there’s a difference between the unique attributes of emerging technologies and the desired results of marketing initiatives. Arguably all that Sony wanted to come out of its Shazam cross-promotion was to get word out about Spider-Man, and to buy some affinity for the Sinister Six with a particular breed of fan, and to that end it has certainly succeeded. Perhaps it also hoped to gain a little tech cred in the process, even if that cred is more window dressing than truly innovative at a technological level.

Still, the idea of a song as a true QR code lingers. Perhaps Harry Osbourne and Peter Parker could team up and develop a functional spec.

By Marc Weidenbaum

Tags: , , / Leave a comment ]

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*
*

Subscribe without commenting

  • about

  • Marc Weidenbaum founded the website Disquiet.com in 1996 at the intersection of sound, art, and technology, and since 2012 has moderated the Disquiet Junto, an active online community of weekly music/sonic projects. He has written for Nature, Boing Boing, The Wire, Pitchfork, and NewMusicBox, among other periodicals. He is the author of the 33 1⁄3 book on Aphex Twin’s classic album Selected Ambient Works Volume II. Read more about his sonic consultancy, teaching, sound art, and work in film, comics, and other media

  • Field Notes

    News, essays, surveillance

  • Interviews

    Conversations with musicians/artists/coders

  • Studio Journal

    Video, audio, patch notes

  • Projects

    Select collaborations and commissions

  • Subscribe



  • Current Activities

  • Upcoming
    December 13, 2021: This day marks the 25th anniversary of the founding of Disquiet.com.
    December 28, 2021: This day marks the 10th anniversary of the Instagr/am/bient compilation.
    January 6, 2021: This day marks the 10th anniversary of the start of the Disquiet Junto music community.

  • Recent
    July 28, 2021: This day marked the 500th consecutive weekly project in the Disquiet Junto music community.
    There are entries on the Disquiet Junto in the book The Music Production Cookbook: Ready-made Recipes for the Classroom (Oxford University Press), edited by Adam Patrick Bell. Ethan Hein wrote one, and I did, too.
    A chapter on the Disquiet Junto ("The Disquiet Junto as an Online Community of Practice," by Ethan Hein) appears in the book The Oxford Handbook of Social Media and Music Learning (Oxford University Press), edited by Stephanie Horsley, Janice Waldron, and Kari Veblen. (Details at oup.com.)

  • Ongoing
    The Disquiet Junto series of weekly communal music projects explore constraints as a springboard for creativity and productivity. There is a new project each Thursday afternoon (California time), and it is due the following Monday at 11:59pm: disquiet.com/junto.

  • My book on Aphex Twin's landmark 1994 album, Selected Ambient Works Vol. II, was published as part of the 33 1/3 series, an imprint of Bloomsbury. It has been translated into Japanese (2019) and Spanish (2018).

  • disquiet junto

  • Background
    Since January 2012, the Disquiet Junto has been an ongoing weekly collaborative music-making community that employs creative constraints as a springboard for creativity. Subscribe to the announcement list (each Thursday), listen to tracks by participants from around the world, read the FAQ, and join in.

    Recent Projects

  • 0511 / Freeze Tag / The Assignment: Consider freezing (and thawing) as a metaphor for music production.
    0510 / Cold Turkey / The Assignment: Record one last track with a piece of music equipment before passing it on.
    0509 / The Long Detail / The Assignment: Create a piece of music with moments from a preexisting track.
    0508 / Germane Shepard / The Assignment: Use the Shepard tone to create a piece of music.
    0507 / In DD's Key of C / The Assignment: Make music with 10 acoustic instrument samples all in a shared key.

    Full Index
    And there is a complete list of past projects, 511 consecutive weeks to date.

  • Archives

    By month and by topic

  • [email protected]

    [email protected]

  • Downstream

    Recommended listening each weekday

  • Recent Posts