Skip Navigation

A Developer's Guide to Creating Talking Menus
for Set-top Boxes and DVDs

BACK TO CONTENTS

COMBINED VERSION TO PRINT

URL: http://ncam.wgbh.org/resources/talkingmenus


Defining the Problem, Exploring Solutions
    Access to a Set-top Box — An Ideal Scenario
    Access to a DVD — A Current Implementation
The Art of Design
Prompts and Responses: System Concerns for Talking Menus
Speaking of Graphics
Best Practices for Talking Menus
    System Operation
    On or Off
    Voice Choice
    Prompts and Feedback
    Timing and Global Variables
    Translating the Visual to the Verbal
    Visual to Spoken, Step by Step
For the Set-top Box Developer
    STB Memory Constraints
    Synthesized or Recorded Speech Constraints
    Possible Approaches
For the DVD Developer
    Workflow
    Interface
    Bit Budget
    Menus and the Development Environment
    Asset Management and Quality Assurance
    Keep in Mind
About NCAM
Acknowledgements

WGBHNational Institute on Disability and Rehabilitation ResearchU.S. Department of Education


Written by Chris Schmidt and Tom Wlodkowski
The Access to Convergent Media Project, July 2003

Funding provided by the U.S. Department of Education, award number #H133G990105-01 through the National Institute of Disability and Rehabilitation Research (NIDRR). Principal Investigator: Larry Goldberg.

The CPB/WGBH National Center for Accessible Media
WGBH Educational Foundation
125 Western Avenue
Boston, MA 02134
(617) 300-3400 voice
(617) 300-2489 TTY
access@wgbh.org
http://ncam.wgbh.org

Copyright 2003, WGBH Educational Foundation

Introduction

The television used to be a simple appliance. Its interface was mechanical — a few rotary knobs that clicked from position to position, providing tactile feedback. Users who are blind or whose vision is restricted could rely on that sensory feedback to help them to choose channels and make other adjustments. Similarly, users who are blind or have low vision could load a videotape into a VCR and play a movie by simply memorizing the functions of a few buttons on the remote control.

Today, however, selecting a television program or playing a DVD is a far more interactive experience that places visual demands on the viewer as never before. A DVD interface presents users with a graphics-rich menu that offers many choices between chapter selections of movies and documentaries as well as supplementary information such as interviews, biographies, outtakes and timelines. Digital set-top boxes offer access to a wealth of information, entertainment and services via electronic program guides (EPGs), which require users to scroll through long lists of on-screen text and graphics to view choices and select a program or service. The latest generation of digital set-top boxes offer EPGs that provide detailed information about programs, the ability to set parental controls, and the ability to program channel selections for future viewing. And in the coming years, an even wider range of activities and choices are likely to appear.

Defining the Problem, Exploring Solutions

The highly visual nature of the interface used in these new digital media formats has created serious and growing barriers for blind and low-vision consumers. The more graphical the interface, the harder it is for a user who is blind to use it.

The user interface within set-top boxes and DVDs may close many doors for individuals who are blind or visually impaired if steps are not taken to ensure some alternate means of navigation in this graphic-rich environment. Viewers who are blind or visually impaired may be shut out from independent access to new programs and services, just as family, friends and colleagues begin to widely utilize them.

Similar barriers were created by graphical interfaces within computer operating systems and on the Web. Over time, major hardware and software developers have enabled solutions that provide alternate navigation and presentation options for desktop computers. And for Web content, the guidelines developed by the Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C) give recommendations on how to build accessible web pages, authoring tools, and browsers. Screen readers, text-to-speech synthesizers and digitized audio descriptions are increasingly used to voice text and describe graphics. Unfortunately, these solutions have not yet been applied to convergent-media set-top boxes or DVDs, each of which use different technologies with different kinds of receiver and display devices, and different user interface designs. Yet the technologies that now provide access to computers can be adapted to create a powerful audio interface for set-top boxes and DVDs. Readers who are just beginning to explore media access technologies may want to read more about navigation challenges and audio description of images for blind and low vision users (and captions of audio content for deaf and hard-of-hearing users) at http://main.wgbh.org/wgbh/pages/mag/.

The following guidelines, describing access solutions for set-top-box interfaces, grew out of a three-year effort undertaken by the CPB/WGBH National Center for Accessible Media (NCAM), with funding from the National Institute of Disability and Rehabilitation Research of the U.S. Department of Education. The models and suggestions presented here reflect lessons learned during development of talking menu prototypes for electronic program guides offered by set-top boxes. Guidelines also reflect NCAM's experience in actual product development of four accessible DVDs in conjunction with PBS' American Experience: Chicago: City of the Century; Partners of the Heart; Marcus Garvey: Look for Me in the Whirlwind; and Abraham and Mary Lincoln — A House Divided.

We hope these guidelines will encourage software and hardware developers to begin thinking now about how best to provide audio menus, audio encoding and even audio-synthesis capability with their titles and platforms before digital-media standards reach full maturity.

Some of the technologies discussed here are mature, and others are still taking shape. Each is designed around a graphical interface which by nature creates significant barriers for those who cannot see. Regardless of the technology, however, NCAM's goal was the same: to provide a useable audio interface similar to those found in other graphical user interfaces. In the end, for users who cannot see, the difference between having access to a world of information and being shut out altogether may ultimately depend on the availability of accessible talking menus.

Access to a Set-top Box — An Ideal Scenario

It's 10 p.m. and Joe, who is blind, feels like watching television. Joe's wife and son, both of whom are sighted, are not available to read the program listings. But even so, Joe thrives on living independently and prefers not to rely on his family. So he picks up the remote and switches on the TV and set-top box. Recently, his cable provider upgraded their technology and installed in his home a brand-new digital set-top box (STB) that boasts a suite of enhanced services, including the ability to deliver the information displayed on the screen via spoken audio. For the first time, the graphical electronic program guide (EPG) is accessible to users like Joe.

So when Joe turns on his television, he realizes that the talking-menu features on the STB have been switched off. Using the remote, he presses the menu key and a voice prompts him to select a configuration. The talking menu also reminds him how to select a setting from his list of stored STB configurations. Joe chooses audio navigation. Because Joe has become an experienced user of audio navigation, his previously stored personal configuration includes a setting to drive the speech synthesizer in fast mode, cutting down on the time it takes to read text aloud.

Joe switches back to the main menu screen and the STB reads aloud a list of available choices. Using the arrow keys on the remote control, Joe steps through the menu and selects the electronic program guide. The EPG is a grid that displays program times across the top, a list of channels along the left side, and program titles and information in a series of columns. But because Joe is blind, he needs the information to be presented in a linear fashion. Once inside the EPG, Joe listens to a list of options that control the order in which the information will be spoken. Those options include "read from current time forward," "read by date," and others. Joe selects the current-time option. The talking menu begins reading, channel by channel, and Joe hears the following:

"10 p.m." (brief pause)
"Channel 2. CNN: Larry King Live."
"Channel 3. FOX Baseball: Red Sox vs. Yankees."
"Channel 4...," etc.

The STB talking menu will continue reading until the list of offerings is complete. It will then return to the top of list and begin reading the channel offerings under the next time heading. But Joe can interrupt the sequence at any time simply by pressing one of the arrow keys on the remote. A single down-arrow press will interrupt the audio output and put the STB talking menu into a suspended state. Another single down arrow will read the next channel entry. Two down arrows in quick succession will skip to the next time slot. Three down arrows in quick succession will skip to the next day.

When Joe hears that the Red Sox game is on channel 3, he presses the Select key on the STB remote and joins the game in progress. But after three Red Sox batters strike out in quick succession, he decides to end the punishment and hunt for something a little more uplifting. He switches back to the EPG but this time he decides to search the listings for a specific program type. Joe is in the mood for a movie so he selects "Search" from the opening screen. The talking menu announces a list of program genres. Using the remote, he steps through the list and selects Movies. Once again, he listens as the talking menu reads down the list of current programs. One choice sounds intriguing and he pauses to learn more. Using the remote he highlights the title of the movie and presses the Select key on the remote. The system responds by giving him a detailed description of the film.

Thanks to the support of America Online, a division of AOL Time Warner, this project created a prototype that reflects a possible approach for accessible EPGs. The demonstration, based on the actual EPG designed for the AOLTV service, can be downloaded at http://ncam.wgbh.org/convergence/epg.html. It runs on Windows and Macintosh OS9/Classic and demonstrates a flexible audio interface for a full-featured EPG.

Access to a DVD — A Current Implementation

Suppose that Joe decides instead to watch a DVD, Chicago: City of the Century. This exercise reflects reality since four DVDs have been released to date that are fully accessible via talking menus. This documentary film is one of the first to offer audio navigation and Joe is curious to find out how user-friendly the system is. From the size and feel of the package he can't tell if he has selected the right film, but as soon as he inserts the disc into the player he's greeted by a professional narrator who announces the name of the film, followed by, "This PBS DVD is equipped with audio navigation. To activate, press 1 now, followed by the Select key." The same information is also displayed on the screen, as shown below.

Chicago opening screen as described above.

Joe follows the instructions. Now the screen changes and displays the text next spoken by the guide:

Audio navigation is now on. Menu items and content will be spoken to you. To turn audio navigation off, press 1 followed by the Select key from any menu.

Next, Joe hears the on-screen FBI warning. A minute later, he has entered the main menu screen. With audio navigation enabled, the talking menu announces, "This is the main menu for Chicago: City of the Century. Part I: From Mudhole to Metropolis. Disc 1 of 4."

Chicago main menu as described in the text.

Visually, the screen is quite elaborate, with stylish graphics and menu items embedded among still images. But the talking menu goes on to explain how to use the remote control to make on-screen choices: "Use the up and down arrow keys to scroll through the five menu items. Choose an item by pressing the Select key." The instructions are straightforward, simple and are only provided in audio format. They are not printed on screen.

Joe scrolls through the menu. Every time he selects a new item, the talking menu announces that item and any additional text that appears on screen. For example, when Joe hits the down arrow and highlights "Play Program," an additional text box appears that reads, "From Marquette and Joliet's 1673 Exploration to the Great Fire of 1871." The talking menu reads this text as well.

Chicago main menu with 'play program' highlighted

Every time Joe makes a selection, the talking menu announces any updated information. It also repeats the basic navigation instructions for using the up, down and select keys on each menu, though Joe can interrupt it at any time by scrolling to the first menu item. After exploring the disc and its features, Joe finally returns to the main menu. He scrolls through choices and when he hears "Play Program" he presses the Select key. Joe settles back to enjoy the film, impressed by the simple but elegant design of the audio navigation feature.

The Art of Design

The above scenarios — idealized and actual — suggest design solutions that promise equal and independent access to television and DVDs for users who are blind or visually impaired. But there is still much work to be done and many questions to be answered before these technologies will be widely available and commonly implemented. And if previous experience holds, now is the perfect time to establish best practices for content providers and developers alike, before they begin to design accessibility into their products en masse.

Interface design is an art as well as a science. An intelligent designer can create an interface that wrings all the performance it can out of the hardware. But it must also accomplish much more. It must anticipate user needs, abilities, assumptions and preferences while striking just the right balance between ease of use and intrusive handholding. An interface that is confusing, illogical or inconsistent can drive a user away. But an interface that goes too far in the other direction, offering so much assistance and guidance that it actually prevents the user from making any forward progress, can be equally maddening. Finding the right balance isn't easy. User preferences change over time, both individually and collectively. Styles change and technology continually opens new doors. The quest to create the perfect interface is never-ending. However, the questions and the guidelines that shape that quest are well-established and can serve as a useful framework for defining best practices when applied to audio navigation design.

To create an elegant audio interface for users who cannot use a graphical interface, developers and content providers need to resolve a number of questions. For the sake of clarity, we'll group these questions into two broad categories:

Answers to questions in the first group have much to do with the actual manipulation of the hardware and can be answered definitively. Answers to questions in the second group are harder to pin down, since they are rooted in human psychology and depend very much on a developer's ability to understand and anticipate the needs of a user who relies on different sensory inputs.

Prompts and Responses: System Concerns for Talking Menus

Joe's positive experience using the talking menus provided by both the electronic program guide and by the DVD is due partly to the way in which the audio interface both prompts for and responds to Joe's inputs. Designers of a similar talking-menu interface should first consider the following questions:

Speaking of Graphics

The most difficult challenge facing designers of audio-navigation systems is finding the optimal strategy for translating graphics into their spoken equivalents. A well-designed audio-navigation system may not always adhere to the design concept underlying the graphic interface. The audio-navigation interface must take into account the realities of human auditory processing abilities, which are distinct from the means by which we comprehend visual or even tactile information.

That said, no one expects developers to create two distinctly different interfaces, one for sighted users and one for users who are blind. To do so would not be practical or cost-effective and may not even serve the interests of users. Remember that many users will listen to the audio navigation while looking at the visual menus. However, developers of audio-navigation systems should be willing to take a few judicious liberties with the visual interface when creating an audio equivalent. For example, a button labeled "Main Menu" on the visual interface might be spoken as "Return to Main Menu."

In the idealized scenario discussed earlier, Joe used talking menus to make sense of the electronic program guide. One of the first things he did was to instruct the EPG to present information in a linear fashion, reading forward from the current time, down the list of channel choices. The fact that the graphic representation of the material is a two-dimensional grid is unimportant to Joe. While the grid allows a sighted viewer to quickly locate the intersection of a particular channel and a particular time of day, that visual shortcut is not available to Joe. In fact, if the talking menu tried to explain that structure to Joe or to force him to visualize that layout in his mind before making a selection, he would probably have a much harder time navigating. The fact is, information presented in a linear list is easier to hear and to remember than information spoken aloud in two-dimensional grid — if such a thing is even possible.

Let's take another example. Imagine a calendar, December, 2002. On which days of the week do Christmas and New Year's Day fall? A quick glance tells us Wednesday for both. To learn the answer so readily, we rely on the graphic design of the calendar. We know that the weeks are arranged in rows. We know that numbers mark each day. We don't need to begin scanning at the top to find the 25th and the 31st — our eyes can jump right to them. As a piece of graphic design, the two-dimensional calendar combines simplicity with elegance, and is highly usable.

For someone like Joe, however, the power of the graphic design is lost. To him, a month is a string of days, grouped in weeks. Within that simple description, he is free to organize the concept in his own mind any way he chooses. It is unlikely that he merely reconstructs a two-dimensional grid and then "reads" from it using his mind's eye. That strategy presents too many possibilities for error. Rather, he's more likely to want to know on what day of the week the first of the month falls. In this case, December 1st is a Sunday. Quickly adding three weeks, or 21 days, he learns that the 22nd is also a Sunday, putting Christmas three days later, on Wednesday.

If Joe wanted to scan the entire month for appointments, he might choose to do so by the week — asking the calendar to read first through all the days in the first week, then the second and so on. He could also check by day — stepping through the Mondays, the Tuesdays, and so on. These methods are analogous to a sighted person looking across the rows or down the columns of the calendar. But Joe never needed to know, or cared, that the calendar is graphically presented as a grid.

One of the greatest services the designer of talking menus can provide can provide for users who are blind is to temporarily ignore the logic underlying the graphic representation of data, and think how best to help the audio-dependent user absorb the menu items. This is especially true for developers working on complex applications, such as the STB. It is less true for developers working on applications with very simple and easy-to-use menu trees. In other words, thinking outside the box of the visual interface becomes more important as the graphical interface itself grows in complexity.

In order to create an interface for navigating complex menu structures, designers should first ask themselves the following questions:

Answering these questions, as well as the usability questions presented above, can provide a framework for a discussion of best practices when designing talking menus for users who are blind or have vision impairments.

Best Practices for Talking Menus

System Operation

The first set of questions in the previous section mainly addresses operational concerns that directly affect the user's ability to easily control the audio navigation system. The second set of questions deals with the user's ability to understand the content of the menus.

It may be helpful to think of the two groups in another way. In as much as the first list determines the mechanical aspects of the audio-navigation feature, they can also be thought of as the kind of questions that could be used to shape the behavior of an entire product line or library. It would make sense, for example, if all DVDs released by a particular distributor shared common features regardless of the content or the style of a given title. Likewise, as set-top-box technology evolves and one generation of audio-navigation interface gives way to the next, users would benefit if the founding principles of interface behavior were passed continuously from one software iteration to the next. Answering these questions with a global approach in mind will also help establish a universal grammar for the design of audio interfaces.

On or Off

There is perhaps no more important design decision than the one that determines how audio navigation will be enabled or disabled. Obviously, if a user who is blind cannot enable the system, it is useless. But by the same token, if a sighted user cannot disable the system, it might be seen as a liability, despite the fact that many sighted users can derive benefit from an audio-navigation feature. The real issue here is choice: can the system be switched on and off, and how easily? Our solution is to have the insertion of the DVD trigger an audio prompt that instructs the user how to enable the audio-navigation feature, as illustrated in the first screen of the Partners of the Heart DVD.

This DVD is equipped with audio navigation. To activate, press 1 now, followed by the Select Key.

This ensures that even a first-time user can easily access the audio-navigation feature without help from a sighted friend. Also, the user always has the option of either enabling or disabling audio navigation at any time during operation of the DVD or STB.

Developers should follow these recommendations when designing the enable/disable feature:

Voice Choice

There are two ways to give voice to an audio-navigation system — a recorded human voice, or a computer-synthesized voice. Each has its distinct strengths and weaknesses.

Recorded human voices are in common use today. Many menu-navigation systems use human voices to provide prompts, most notably in telephone-voice-navigation systems. To build a voice-navigation system using a human voice, a person must be cast in the role of the speaker and that person must be recorded pronouncing all of the words or phrases contained in the menu tree. Those recordings are then played in response to user choices.

The advantages of using human voices include:

The disadvantages of using human voices include:

Computers that can synthesize a human voice have been in existence for two decades. Initially, the speech had a very mechanical and unnatural sound to it. But today, the best speech synthesizers are quite natural sounding and offer a range of choices of race, gender, age and language characteristics.

The advantages of using synthesized voices include:

The disadvantages of using synthesized voices include:

Because so much of the relevant technology is currently in development or transition, there is no simple answer to the question of whether or not to use synthesized voices.

Today, digitized human voices are the norm. However, in an ideal world, playback devices would contain the hardware and software required to turn text into convincing human speech. That is not yet the case. However, the technology to create synthesized voices outside of a given DVD player or STB does exist, offering the possibility of a hybrid approach.

For example, DVD designers seeking to reduce costs could create synthesized voices at their own workstations, and then embed them, as they would a human voice, in the audio-navigation feature included on the disk. This would eliminate the cost of the speaker and of the recording studio. It would also allow the developer to change the content of the audio menus as easily as he or she could alter visual menus and other graphical content.

But in so far as most audio-navigation systems will continue to rely for the foreseeable future on recorded human voices, the following recommendations can help smooth the transition when it comes to the exclusive use of synthesized voices.

Developers should follow these recommendations when creating audio-navigation prompts and responses:

Prompts and Feedback

Interactive quality is the heart of the audio-navigation system. Aside from the basic ability to turn speech on and off, it is the consistency, predictability and ease of use that will make or break a talking menu. Simply put, the better the talking menu can anticipate the user's needs without interfering with the user's ability to choose, the more successful it will be. To that end, the talking menu should behave as follows:

Timing and Global Variables

Some choices faced by developers seem insignificant, but a mistake can easily confuse users. Attention to detail in interface design always pays off. That's why it's important to enumerate several low-level aspects of the audio interface that are easy to overlook but have a real impact on the user's experience.

As the user interacts with the navigation function, he or she expects the system to respond to input from the remote control in very specific ways. If these expectations are not met, the user can become anxious and begin to question whether the system is set up properly, whether the remote is working, and so on.

To avoid this distress, developers should take care to make their audio-navigation systems as responsive as possible through the following behaviors:

Navigating between and within talking menus should be straightforward. Sighted users expect a graphic interface to be simple, elegant, intuitive and responsive. Users who are blind deserve no less.

Translating the Visual to the Verbal

When the operational foundation of the audio interface — the behaviors outlined above — are well-established and predictable for the user, developers will find that much of the sting of translating visual information to spoken text has been removed from the process. As long as the user knows that certain controls will always elicit predictable responses — and thus enjoys a high degree of confidence that he or she will not become lost within a menu structure — the developer is free to imagine creative ways to organize information in order to capture the intent and feel of a graphic menu system, without feeling confined by its look.

As already discussed, people process audio information very differently than they process visual information. The graphic field is two-dimensional — imagine a plane with aspects of height and width. The eye naturally skims freely across the entire visual field, processing both dimensions simultaneously and looking instinctively for intersections between those two dimensions. This is why the eye can take in a grid, such as a program guide, and quickly find the intersection between the time of day and a given channel.

A simple visual list can contain rich contextual information about the relative importance or relationship between items. Text items can be colored, indented or enlarged to suggest those relationships, and all of that information is available to the eye at a glance.

For example, the first chapter-selection screen on the Abraham and Mary Lincoln — A House Divided DVD is a list of eleven items. To the eye, they readily group into two categories of action: the first, seven selections to explore various chapters; the second, four selections at the bottom of the screen that allow the user to switch to different menus, including a return to the main menu. This contextual information is readily available to a sighted user at a glance and it adds, in effect, a second dimension to the list of eleven items.

Lincolns — chapter selection screen as described above.

Herein lies a crucial difference between the visual and auditory fields. Specifically, the auditory field is nearly one-dimensional. Merely reading this list aloud will not help the listener to appreciate the transition from one category to the next. To build a conceptual understanding of the difference between the first seven items and the last four items, the user who is blind will have to pay very careful attention. And it may be unreasonable for a developer to assume that the user will do so.

The problem is easy to identify. Listening is a time-based activity, in which one piece of information follows another at a fixed speed. It is difficult to skim that information or to process more than one auditory stream at a time. There is no auditory equivalent of a list where the items are grouped, either in clumps or arranged in a two-dimensional grid.

In order to mimic visually embedded contextual information in an auditory list, aspects of speech would have to change from one item to the next — timing, volume, emphasis, speed, language and so on. The listener would have to concentrate with great determination in order to connect items based on their spoken attributes, and if the list was long and complex, the listener could easily be overwhelmed.

It is for these reasons that developers seeking to translate graphics to spoken text must learn to separate the information they seek to present in words from the information they seek to communicate through context or other visual cues. Once developers achieve that separation, they can better choose which information must be communicated and which information may well be discarded. Beyond speaking the name of a menu item, developers may choose to add other spoken information to communicate the graphical content that the blind user cannot perceive. But beware: adding too much audio information may only confuse the user. For the user who is blind, the primary goal is to navigate, not to learn to appreciate the visually pleasing graphic.

Visual to Spoken, Step by Step

Menus are lists. The first step in translating a graphic menu into a spoken menu is to determine how it is structured. Is it a linear list? Is it, in fact, a group of sub-lists? Or is it a grid, which is a group of lists that share certain attributes?

The second step is to decide whether or not any of the contextual visual information is absolutely necessary to communicate in order for the user to successfully navigate. Do the colors of the words really matter? Does the font size matter? Does the location on the screen matter? The answers to these questions may help determine how many different sub-lists are present on the same screen.

The third step is to create the spoken-menu structure based on the obvious or not-so-obvious list structure embedded in the graphic. For example, imagine a graphic screen with six menu items. What if three of the items are rendered in one color and relate to choice of programming, while the other three are in another color and relate to system preferences? The implication is that this list of six items is actually two lists of three items each. If the audio-navigation system simply allows the user to move from item to item, confusion may set in unless some additional audio cue lets the user know that the six items are actually two groups of three.

The following are guidelines to help developers relay visual contextual information verbally:

The guidelines for translating visual information to spoken information can be summarized as follows:

For Set-top Box Developers

A set-top box, or STB, refers to any device inserted between the cable or satellite feed and the user's television set. These devices have the capability to select and display individual channels. Satellite television today follows the MPEG-2 digital-compression standard, which is the basis of both the Digital Video Broadcast standard (DVB) and the Advanced Television Systems Committee (ATSC) standard. Most U.S. cable providers have either adopted the MPEG-2 standard or are in the process of upgrading their services from analog to digital in order to do so. MPEG-2 allows the delivery of any information that can be digitized, including audio, video and text. That material can then be easily displayed on digital High Definition television sets, or in analog NTSC, PAL or SECAM formats.

STB Operating Constraints

Given the flexibility of the DVB standard, one would expect that adding an audio-navigation capability to an STB would be simple matter of writing the proper software to include in the STB's operating system. However, despite the power and flexibility of the DVB standard, it is currently next to impossible for American cable or satellite services to offer audio-navigation services. The computers inside American STBs are simply too primitive to support this additional capability.

Most STBs in the United States are loaned to consumers by their cable or satellite providers. The cost of the device is essentially included in the subscription fee. In some countries, third-party manufacturers of STBs market their products directly to consumers. However, this trend has not carried over into the U.S. market. This single fact is significant to any discussion of the future of audio-navigation services available to cable and satellite customers. As long as it is the service provider who pays for the development and manufacture of STBs, they will likely choose to manufacture boxes at the lowest price possible. Because they face no competition from third-party manufacturers, there is little incentive to make the computers inside those boxes more powerful than necessary to satisfy the majority of their customers.

Today, most STBs contain a single CPU similar or identical to the Motorola 68000, a chip introduced in the early 1980s. The American STB also contains between 2 and 4 megabytes of RAM but no mechanical storage device such as a hard drive. Most of the RAM is devoted to the operating-system code required to carry out basic interactive functions. As a consequence of the limited power of the typical STB computer, it is unlikely that a software-only solution to audio navigation could be added to that computer's functionality.

Synthesized or Recorded Speech Constraints

As discussed earlier, there are two approaches to creating the audio-navigation interface: speech synthesis and recorded voice samples. In the idealized scenario discussed earlier, the program guide spoke to the user as he moved through the program selections. To create this spoken voice the cable provider would have to either record an actor's voice or somehow provide enough computer power to synthesize that voice from text entries appearing in the program guide.

Because the contents of the program guide are in constant flux, and listings can change at any moment, the job of recording and assembling sufficient audio samples to speak every possible program title would be unmanageable. Unlike pre-recorded telephone navigation systems that piece together words or parts of words to speak times, dates and other regularized information, an STB system based on recorded audio would never be able to stay abreast of the tremendous variety of titles and other information one would likely find contained in a typical program guide (to say nothing of the problem of parsing non-English words or the ever-expanding number of advertisements and product offerings showing up in program guides). It soon becomes obvious that the only possible means of delivery for audio navigation is to employ speech synthesis to convert written text to spoken text on the fly.

Possible Approaches

From the service provider's point of view, there are two ways to approach voice synthesis and delivery. In one approach, the provider could conceivably install speech-synthesis software on the central server, and then feed that digitized speech to the user along with the program guide information. Even assuming that sufficient bandwidth exists to carry this extra traffic, this approach would likely not work. The fact is, the STB has only limited control over the program guide information that flows out of the central server. The interactivity enjoyed by the user is largely an illusion. The server feeds its data in a "carousel" formation. The central server feeds program information in a one-way stream that cycles from beginning to end. The STB "drinks" from that stream and downloads blocks of data, ready to be displayed one listing at a time to the viewer. When users move from one program guide entry to another, using the remote control, they do not interact with the central server, per se. Rather, they interact with the STB, scrolling through information already downloaded into the STB's limited memory. In order to add audio navigation to this system, the synthesized audio would have to flow into the STB in the same "carousel" fashion. That would mean that the STB would have to store large blocks of synthesized audio, just as it does large blocks of text data. Because digitized audio samples are orders of magnitude larger than their text equivalents, the standard STB's RAM would quickly become overloaded and the system would cease to operate properly. Unless the operating system changes to allow users greater interactivity, server-centered speech synthesis is not a viable option.

Alternatively, it should be possible for the STB itself to synthesize speech from text. Speech synthesis is a trivial task for today's desktop computers, and users who are blind rely on text-to-speech synthesizers when using the screen readers that make computers accessible. However, as noted above, the American STB's internal computer doesn't yet have the processing muscle or the memory to deliver this functionality. The only possible conclusion is that until the STB catches up with the kind of computer power found in even low-end desktop computers, audio navigation will not soon appear in cable or satellite systems.

There is another alternative, however. In the U.S. the STBs provided by cable and satellite companies are not, in fact, the only STBs in use. Personal digital recorders, such as those marketed under the TiVo brand, are in fact STBs in disguise, and they contain relatively more-powerful computers.

Personal recorders are designed to automatically choose channels, display program information and record television programs for later playback. To do so, they must download and interact with program guide information. Today, users of TiVo and other personal recorders download those program guides via modem from servers maintained by the vendors of the recorders. Users pay a subscription fee for this privilege. Theoretically, this program guide information could be fed to a text-to-speech synthesizer as part of an audio-navigation feature. Under the hood, these boxes do seem to have enough computing horsepower to be able to add a voice-navigation feature.

Another alternative is to dispense altogether with the STB and feed the cable signal directly into a desktop computer. Today, PC users can purchase a Digital Video Broadcast card that allows them to watch basic cable programs on their computers. These cards cannot decode encrypted premium channels, such as HBO, but because the program guide information delivered by the service provider is not encrypted, the PC should be able to display it. An enterprising developer should be able to write software that is compatible with the screen-reading software used by people who are blind, and thus deliver an audio-navigation system for cable or satellite TV to the desktop.

Program guide developers who seek to make their products accessible to screen-reading software should follow these rules:

Obviously, because some of these recommendations involve embedding "hidden" text for the benefit of the screen reader — and because these considerations are likely new to the developers who create program guides — some reverse engineering of the basic program guide architecture may be necessary to accommodate them.

For DVD Developers

The rate of DVD adoption and penetration into American homes has accelerated in a manner unprecedented in the history of consumer electronic sales. It's easy to understand why the format has become so popular. Picture quality, audio quality and packaging have all played a significant role, but many industry analysts point to another feature unique to DVDs: the ability to include extra features, such as outtakes, commentaries, interviews and even games.

All of these additional features are made possible by the fact that DVDs are not simply recordings of media. They are actually specialized computer programs run by the computers embedded in DVD players. The actual media content is organized and presented by the software. As the discs grow in popularity, that software will include a growing variety of features and functionalities. This is great news for accessibility advocates, since it means that developers can create DVD software that supports built-in interactive audio navigation to assist those who are blind or have limited vision. The bad news is that the current DVD technical specification presents some obstacles that inevitably complicate the task of building in an audio-navigation feature. These difficulties can be overcome, and once the workarounds have been mastered, developers will be on their way to making their products fully accessible.

For developers, authoring a disc that includes audio navigation means thinking in new ways about the following:

Workflow

Adding audio navigation to a DVD does alter the typical developer workflow by adding several extra steps to the development process. Typically, DVD authoring begins with the creation of an overall design that includes the scripting of a menu tree — a kind of road map for the user interface. That script determines the range of options to be presented to users as well as the interconnections between those options, their organization into discrete menu screens and the manner of access to the actual content. When adding audio navigation, an additional script must be created. Each text block in that script must be recorded by a narrator and prepared for inclusion on the disc along with other media files.

There are several questions that title designers must answer before completing these steps. They are:

Once the script is complete and the navigating voice recorded, designers must consider two more problems:

Because the talking-menu media take up space on the final disc, developers need to compress the audio files. This reduces both size and fidelity. Determining exactly the extent of compression will depend on the capacity of the disc and the total amount of content it must carry. In some cases, audio clips will not require significant compression. In other cases, storage concerns may drive the developer to significantly compress the clips. Either way, the determination needs to be made early in the process.

Interface

One of the most important interface decisions to be made in the early stages of development is how to toggle the audio navigation feature on and off. It's not a trivial matter, because if the mechanism for doing so is awkward or counter-intuitive, audio navigation may become an irritant rather than an aid.

Generally speaking, there are two ways of enabling or disabling the feature. Developers should always include an enable/disable feature within the menu tree, as shown below in the accessibility options screen from the Marcus Garvey: Look for Me in the Whirlwind DVD.

The Garvey Help menu with the Accessibility Options submenu selected. Choices are Turn Audio Descriptions On/Off, Turn Audio Navigation On/Off, Turn English Subtitles On/Off, and What is the Descriptive Video Service?.

Users should also be able to enable or disable the feature using the remote. One problem persists, however, in using the remote control: manufacturers have not equipped remote controls with a button dedicated to the audio-navigation feature. Some pioneering developers have worked around this limitation by co-opting existing button combinations.

Within the four DVDs that incorporated NCAM's access suggestions, we created a de facto standard for turning audio navigation on and off: pressing the 1 and Select keys on the remote control. This choice is generally seen as prudent, since it does not normally affect any other function. But it is not ideal. There is no guarantee that DVD developers won't, in the future, find some widely accepted use for this particular key combination, thereby causing a conflict. For that reason, it is imperative that developers also include an enable/disable selection within the disc's menu tree.

Furthermore, the opening screen of the disc, which announces the availability of audio navigation, should provide clear instructions for locating the enable/disable menu option. Which menu should contain the selection depends on the overall design of the disc and interface. Obvious choices are to include it within the same menu that controls subtitling or foreign-language choices, or even to build an entirely separate menu dedicated to accessibility options, as shown below in the Abraham and Mary Lincoln — A House Divided DVD. Wherever it ends up, it must be easy to find and easy to use for both sighted and blind users.

Lincolns main menu with accessibility option highlighted in the list of 5 menu items.

Discs that offer audio navigation ought to announce that fact upon insertion into the DVD player. As previously shown, an opening screen should include visible text as well as an audio clip that informs the user that the disc is equipped with an audio-navigation feature. The method for toggling the feature on and off should be clearly enunciated at the beginning of the DVD. Small-scale usability testing suggests that sighted users will benefit from having talking menus on by default. However, until audio navigation is a more widely accepted feature, the default setting will in most cases leave the talking menu disabled.

Bit Budget

Managing the bit budget is a straightforward affair for DVD developers, made only slightly more complicated by the inclusion of an audio-navigation feature. Currently, DVDs come in two sizes, 5 gigabytes and 9 gigabytes. In general, commercial media, such as film and television programs, are released on 9-gigabyte discs while 5-gigabyte discs are more popular among developers of educational and institutional titles. One reason for the difference is that 5-gigabyte discs can be burned at the desktop while 9-gigabyte discs can only be created using more expensive mastering and duplication equipment. They are therefore less practical for small developers. However, either disc size is certainly adequate for accommodating audio navigation, both for the digitized audio clips and for the additional menus and code required to implement the system.

For planning purposes, developers typically assume that each menu screen will be about 100 kilobytes in size. While the number of menu screens varies widely from title to title, developers who include an audio-navigation feature will soon find that they must manage many times the normal number of menus, thanks to a sharp limitation on the number of audio clips that can be associated with a single menu.

Imagine a screen with five menu items. The user who depends on audio navigation expects to hear each menu item spoken as the cursor moves from one to the next. For the five items, the developer would need to somehow link five separate audio clips to the screen. Unfortunately, this process is somewhat more cumbersome than one would like.

For historical reasons, the technical specification for DVDs allows only one audio clip to be linked to any given menu screen. In order to create the illusion of a five-item audio menu, the developer will actually need to create six visually identical menus, each with its own audio clip. One of the six would necessarily be the menu screen that appears before the user has made a selection. This screen would have an audio clip attached that announces the name of the menu screen and gives navigation information. Then, each of the five menu items would require its own screen with an attached audio clip. Each of these screens would be identical except for the visual highlighting added to the appropriate menu item. (Visual highlights remain important even when using talking menus. Not all users who will depend on these menus will be totally blind — some sighted users may find the talking menus helpful.)

Suddenly, a single menu has turned into six menus. The amount of disc space required increases proportionally, from 100 kilobytes to 600 kilobytes, plus the amount of space required for the six audio clips. This last number is harder to pin down since it depends very much on the length of the clip and the severity of compression imposed upon it during digitizing. What we do know is that the dynamic range of the human voice is narrow and can be compressed significantly without losing intelligibility. Developers can probably get away with a compression of 96 kilobits per second (kbs) or 12 kbs.

A rough calculation can determine the amount of space required for menus and media. First, we'll make the following assumptions:

number of menus = 15
number of audio clips / menu = 6
average length of spoken clip = 5 seconds

Each menu actually requires seven menus: the silent menu, when audio navigation is not in use, the opening menu for audio navigation, plus one menu for each of the items. The total space required for the graphic menus is calculated as follows:

7 x 15 x 100 kilobytes =10.5 megabytes

Second, we can calculate the total length of the audio clips:

6 items x 15 menus x 5 seconds x 12 kilobytes per second = 5.4 megabytes.

Therefore, the total storage cost of the graphic and audio menus can be calculated as follows:

10.5 megabytes (graphic menus) + 5.4 megabytes (audio clips) = 15.9 megabytes.

Though that may sound like a considerable amount of media, the good news is that it's really just a drop in the bit bucket. On a 5-gigabyte disc, 15.9 megabytes consumes only 0.3% of available space. On a 9-gigabyte disc, that amounts to little more 0.15% of available space.

The conclusion is clear: adding a talking menu does increase the total size of the menu system dramatically, but it shouldn't come close to squeezing out the intended content of the disc. It therefore presents no real barrier to implementation.

Menus and the Development Environment

As mentioned, audio navigation presents a novel challenge to developers because of the fact that the DVD specification allows only a single file — for our purposes, an audio clip — to be attached to a given menu. The workaround for this limitation is straightforward, though somewhat cumbersome: developers must create duplicate menus to provide links to the audio clips. For example, the Partners of the Heart DVD includes several menus. The main menu has a title plus five options, as shown below.

Partners of the Heart Main Menu with five options, 'Play Film,' 'Special Features,' Chapters,' 'Accessibility,' and 'Credits'

With audio navigation enabled, the developer intends for the menu to behave in a very straightforward manner. When the screen first appears, an audio clip announces the name of the screen. Each time the user moves the cursor to a new selection, the appropriate audio clip plays to announce the name of the language selected. To create this behavior, the developer will need to create six menus — one for each audio clip. Each time the cursor moves from one selection to the next, the entire menu will be replaced by a new menu. Graphically, however, the behavior of the menu system will be as sighted viewers expect.

For the sake of simplicity in this narrative, we'll assign menu labels to each screen (one for each selection plus the menu title), from M0 to M5, respectively. Similarly, let's assign the labels A0 to A5 to the individual audio clips that will be played.

When the user first enters the menu, M0 is displayed and A0 is played. In this case, M0 shows the entire graphic screen with no highlighting. A0 is the clip that announces the name of the screen and gives navigation instructions. When the user presses the down key on the remote, M0 is replaced by M1 while A1 is played. (M1 is the same screen but with the first selection, "Play Film," highlighted. A1 is an audio clip that says "Play Film." As the user scrolls down the list, successive screens, with the appropriate selection highlighted, appear as the appropriate clip plays. It is important to note that when the screen is currently showing M6 as the selection and the user presses the down button, the most logical behavior is to return to M1 and to play A1. Of course, at any point the user can select the highlighted choice and the DVD player will respond appropriately.

While this workaround is straightforward, it does require careful preparation. One significant requirement is the addition of silent buffers at the head and tail of each audio clip. Apparently many DVD players experience a delay in playback of audio clips due to buffering speed. This delay can cause the head and even the tail of the clip to become truncated. Developers have found that the best solution to this problem is to add a small bit of silence to the head and tail of each clip. The pads ensure that during playback, all meaningful content will be audible.

The length of the silent clip should be determined empirically, as the optimum length depends upon size of the clip and the horsepower of the DVD player, which varies among different models. To begin, a developer might try adding 15 to 20 frames (.5 to .75 seconds) to the head and tail of the clip. Of course a longer silence will also work, but if it becomes too long, the silence will create the perception of a performance lag on players that handle buffering more effectively, so the shorter the better.

Asset Management and Quality Assurance

Adding audio navigation to a DVD title will probably mean adding hundreds of menus and audio clips to the disc. Those additional files should be managed carefully to ensure a smooth development process.

Before beginning work, developers should create a means of referencing and organizing the large of number of files. One solution is to create a naming system for each graphic and audio file that encodes the content and function of each file. For example, a file name could be assembled from a series of hierarchical references, beginning with the disc reference, followed by a screen number, menu selection number and finally audio navigation clip reference. For example, the file name 011304A10 would resolve as:

Disc 01
Screen 13
Selection 04
Audio File A10

With a bit of practice, it should quickly become second nature to read the name of a file and to understand its contents and its purpose, helping to avoid costly mistakes in which an audio file is linked to the wrong action.

As implied, creating these efficiencies can have a profound impact on the amount of time it takes to test and debug a DVD title that includes audio navigation. Initial accounts from participating developers suggest that adding audio navigation extends by a factor of five the amount of time required to conduct quality assurance over the amount of time required for discs without this feature. To prevent quality assurance from taking even longer, developers must be extremely careful to keep their file trees organized.

Keep in Mind

Only a few pioneering DVD developers are creating titles that include audio navigation. It is still somewhat uncharted territory. These guidelines offer support, but each developer who begins the work is, in some way, an innovator. In that spirit, here are three broad recommendations that developers can take to heart when setting out.

Learn the behavior of users
Simply put, developers must understand how their intended end-users would like to interact with the audio interface. Developers should internalize these preferences as much as possible, with the goal of developing an intuitive sense of what is likely to help or confuse a blind or visually impaired user.

Accept the limitations of the spec
The DVD specification was not developed with audio navigation in mind. If it had been, then no doubt it would have allowed developers to link more than one audio file to each menu. But that's not the case, so rather than curse the fates, move forward.

Hack and work around
In the interest of satisfying the user despite the limitations imposed by the spec, developers should tackle the challenge of adding audio navigation to their discs with imagination and determination. Any insistence on quality is sure to pay off.

About NCAM

The CPB/WGBH National Center for Accessible Media (NCAM) is a division of The Media Access Group at Boston's public broadcaster WGBH. NCAM projects and initiatives expand the reach and refine the technologies of captioning and description, while breaking new ground in the fields of technology, media, disability, and education. NCAM's fellow access departments at WGBH include The Caption Center, the world's first captioning agency, founded in 1972, and the Descriptive Video Service which has made television, film and video more enjoyable to viewers who are blind or visually impaired since 1990.

NCAM pioneered captioning and audio description on the Web and currently leads industry standards-setting activities related to built-in television closed-caption decoders, captions and audio descriptions in digital broadcasting, and accessible online learning. NCAM creates and disseminates tools, guidelines and resources related to digital delivery of captions and descriptions for the Web, DVDs, CD-ROMs, and first-run and IMAX films in theaters as well as media device navigation solutions for users with disabilities.

For more information about NCAM projects and other media access guidelines, visit our website at http://ncam.wgbh.org.

Developers are encouraged to visit http://ncam.wgbh.org/richmedia — a growing collection of resources, tools, showcase solutions, and articles that can help designers, multimedia developers, consumers, and access technology researchers stay abreast of current and emerging solutions.

We welcome your comments and suggestions about this guide and about other NCAM resources. You can reach us at:

(617) 300-3400 (v)
(617) 300-2489 (tty)

access@wgbh.org

This document is available online at:
http://ncam.wgbh.org/resources/talkingmenus/all_print.html

Acknowledgements

This work was funded by the "Access to Convergent Media Project", under grant #H133G990105-01 from the National Institute of Disability and Rehabilitation Research of the U.S. Department of Education. More information on this initiative including downloadable prototypes and papers from early research can be found at the project Web site at www.ncam.wgbh.org/convergence.

WGBHNational Institute on Disability and Rehabilitation ResearchU.S. Department of Education

This work was greatly enhanced by the dedication and encouragement of diverse supporters who donated valuable hours and resources to this initiative. The list below represents individuals, organizations, and producers who shared our vision of making sure that ALL viewers are included in the audience for their programs and services.

Chicago: City of the Century
A co-production of WGBH Boston and WTTW Chicago in association with the Chicago Historical Society for American Experience. Funding for the enhanced DVD provided by the Rauner Family Foundation.
DVD production: WGBH Interactive.

Partners of the Heart
A Duke Media and Spark Media production for American Experience
DVD production: Spark Media/Pushy Broad

Marcus Garvey: Look for Me in the Whirlwind
A Firelight/Half Nelson Productions film for American Experience
DVD production: Missing Pixel

Abraham and Mary Lincoln — A House Divided
A David Grubin Productions, Inc. film for American Experience in association with PBS
DVD production: WGBH Interactive/Planet Studio

We are particularly grateful to the National Endowment for the Humanities which enabled free distribution of the first-ever title with audio navigation, the Lincolns DVD, to educators and librarians nationwide in the summer of 2001.

We are also deeply indebted to America Online, Inc., a division of AOL Time Warner, for their participation in this project and for their ongoing support of explorations into accessible navigation.

BACK TO CONTENTS