This post is about auditory display and related topics. This subject is not taught in most recording programs so it may be new to readers of this blog. But it is a field with new opportunities.
By its broadest definition “auditory display” can refer to any information that is communicated in sound. Thus, at this broad level, an audiobook could be considered “auditory display.” But most researchers and practitioners in what is called “auditory display” among the research community tend to be more focused on non-speech presentation of auditory information or, perhaps, presentation forms that involve both speech and non-speech sounds. This certainly could be said of the major research organization for auditory display, The International Community for Auditory Display (ICAD) which puts on an annual conference: The International Conference on Auditory Display. Full disclosure requires me to note that I was the president of ICAD from 2000 through 2003.
ICAD was founded by Gregory Kramer in 1992. The conferences are held every year and the next one, as I write this, is scheduled for late June, 2017, at Pennsylvania State University. I encourage anyone to visit the ICAD website, www.icad.org, which has a wealth of information about the field, sound examples, lists of publications, etc.
To set the stage for what auditory display is let me quote from the ICAD website:
ICAD is a forum for presenting research on the use of sound to display data, monitor systems, and provide enhanced user interfaces for computers and virtual reality systems. It is unique in its singular focus on auditory displays and the array of perception, technology, and application areas that this encompasses. Research areas covered by ICAD include:
- Auditory exploration of data via sonification (data controlled sound) and audification (audible playback of data samples)
- Real Time monitoring of multivariate data
- Sound in Immersive Interfaces (Virtual Environments) and Teleoperation
- Perceptual issues in Auditory Display
- Sound in generalized computer interfaces
- Technologies supporting Auditory Display creation
- Data handling for Auditory Display systems
- Applications of Auditory Display
Included within each of these areas of inquiry are many issues concerning application, theory, hardware/software, and human factors. Integration with speech-audio implementations, with graphical display techniques and their concomitant perception issues also pose significant challenges in each area.
Some of the terms may sound quite technical but all are based on the underlying idea that sometimes our ears are the best tool of perception. At a basic level this can include design of audible signals for everyday devices from computer or phone warning tones to the signals used in the subway when one swipes a pass. Companies hire sound designers, or sound design firms, to make this kind of everyday signaling effective and researchers work on ways to make this form of communication more effective.
Possibly areas of auditory display that are less intuitive to the outsider are sonification and audification. This involves converting a data set into sound of some kind since our ears may become aware of data anomalies that may not be immediately apparent in visual display of the data, or someone may need to keep track of a real time data stream while doing something else that occupies one’s eyes (such a performing delicate surgery). Sonification can involve using the data to drive a sound producing mechanism. I saw a demonstration of a program which helped a computer programmer scan code for errors in computer code. Each subroutine produced a kind of melody so a faulty one would produce a sound that was quickly recognizable as faulty. There are many other forms of sonification ranging from the simple to the complex. Audification is similar except the data is turned into sound directly rather than used to drive another sound generation system.
Of course another common use of auditory display is assistive technology for the sight handicapped. When ICAD was founded one of the first demos I saw was of a sonic Finder for the Macintosh computer (pre OS-X) so a blind or partially sighted use could use the mouse to perform basic operations of the operating system, such as opening programs, moving files, etc.
As can be seen from the ICAD list there are both technical and psychological aspects to auditory display: creating hardware and software and understanding how sound is perceived. Auditory Display is a new field (when taken as a separate field from other disciplines) so there are opportunities for young people to get involved and chart new territory.
With the great interest currently in virtual reality in which someone is visually immersed in an environment, there is a concomitant need for designers and engineers who can create environments that also immerse one sonically. This is not as trivial as it may seem because sound VR has to change perspectives as the viewer/listener moves their head. In a visual immersive environment (where the viewer wears special goggles) an image might be presented of a farmhouse. When the viewer turns their head to the right they might see a field of cows. In sonic terms the cow sounds would have to seem to be coming from the right when the user faces the farmhouse, but come from the center when the head is turned to face the cows. Not all sound objects would change perspective in the same way – the sound of a plane flying overhead would usually not be panned that much when the viewer moved their head from the farmhouse to the cows. Thus sound for immersive VR is a challenging field with many opportunities related to sound design, technical display of the sound, and psychological understanding of how a person perceives various sounds in various scenarios.
I am only touching on the field in this post. I again urge the reader to visit the ICAD website (www.icad.org) and to participate in the conferences. I expect to be in Pennsylvania for the June event. If someone has a question about the field or about ICAD, I also encourage you to submit it via the comments section of this blog.
Leave a Reply