“‘Seeing’ Music from Manga”

Synesthesia thesis

Many years ago, I spent a solid half decade publishing manga, eventually becoming the editor-in-chief of the U.S. edition of the magazine Shonen Jump. It was an amazing experience, one I treasure to this day. However, my work on manga rarely overlapped with my interest in sound.

Traveling to Japan for work over the years did give me the opportunity to attend concerts at Tokyo venues I’d otherwise never have even heard about. Also, there were times while preparing an issue of the magazine when we got to debate how to translate sound effects, or how to effectively help a young reader (this was back from 2004 to 2009, before manga was as mainstream as it became outside Japan) understand things like the vertical ellipses that signify an extended silent pause. Most of the shonen comics I worked on were about fighting (e.g., Naruto, One Piece), the main exception being, as one reader put it, “the one about moving stones around a board.” (The latter was a joke at the expense of Hikaru no Go, which was one of my favorite things we published.)

And so it was with great interest that I read a new academic study from two authors based at National Taiwan University, Taipei: “‘Seeing’ music from manga: visualizing music with embodied mechanisms of musical experience.” The article looks specifically at how music is represented in manga (for the uninitiated, the word is both singular and plural, as is the norm in the Japanese language) that take music as their subject. Iju Hsu and Wen-Yu Chiang, the article’s authors, study such highly recommended series as Nodame Cantabile (about a classical music student ) and Detroit Metal City (the title is self-explanatory), among others.

Are You Experienced?: This diagram from the article maps “how music is transformed into visualized music”

The article, which came out in the recent volume 21 of the journal Visual Communication, explores visual metaphors for sound in various manners, notably the Visual Metaphor Identification Procedure, or VISMIP. Another approach explores “six embodied mechanisms that induce emotion.” In the words of the authors: “this study sheds light on our overall understanding of audio-visual cross-modality, musical experience, metaphor and embodied experience.” It’s dense stuff, and I’m still making my way through it for a second time — and beginning to explore the trove of articles and books cited as references. (Thanks, Gene Kannenberg Jr. and Bart Beaty!)

Leave a Reply

Your email address will not be published. Required fields are marked *