Intellistener

A non-linear audio tool that let listeners cross-link segments of spoken-word tracks into a semantic web of "hyperaudio", showcased with a physical installation where navigation and playback were controlled by a single rotary knob.

As my final-exam project for the MA at the Piet Zwart Institute, I developed a non-linear audio editing and playback tool. The edit-suite of Intellistener let the listener create cross-links between different segments of different spoken-word audio tracks: e.g. dialogue 1 in radio interview A relates to segment 2 in conversation B. By linking audio content and describing and categorizing those relations semantically, a web of spoken hyperaudio was built, with a structure much like the web of hypertext.

Often radio is just consumed passively, but some radio asks for a much more active and engaged listening: interviews that last over 20 minutes and become conversations, pauses left in to let the listener think and recap. As with books, sometimes the text invites you to read back, browse back to an earlier chapter or pause and sidestep to another book. Similarly that can be done with audio. We just don’t have the right tools yet to do so.

One of my favourite radio programmes from the time of my master studies was De Avonden (VPRO). It featured writers, artists and philosophers, and many of its topics came back in different conversations with different people over the years. I often felt the need to jump back to a previous recording to relisten to the exact statement, and to share it: “Hey, listen to this. It might be related to our previous discussion.” That seeded the plan for a system of annotated audio.

The playback installation presented the visitor with a chair, a big screen and a single rotary button. The screen showed a tree structure of all possible navigation routes through the mesh of audio segments, with a round red indicator travelling along it: the non-linear playback head. With the rotary knob the listener could speed up or slow down playback and choose which branch of the tree of possibilities to follow.

To test the beta version I organized a series of workshops. In the Waag in Amsterdam, a small group of beta testers got their first hands-on contact with Intellistener. They annotated a shared selection of De Avonden episodes, searched for connections between them and discussed the process of annotating audio.

The project and its hyperaudio experiments are archived at intellistener.com.