Making Educational Software and Web Sites Accessible
Design Guidelines Including Math and Science Solutions

WGBH | NCAM | ACCESS TO PIVOT PROJECT

NEXT | CONTENTS

Guideline 2
Provide access to multimedia presentations for users with sensory disabilities.

Audio descriptions provide access to multimedia for people who are blind or visually impaired by adding narration that describes the visuals, including action, scene changes, graphics and on-screen text. Captions added to multimedia presentations ensure that the audio components of the presentation are accessible to individuals who are deaf or hard-of-hearing. Both audio descriptions and captions are useful learning tools for a wide array of users in addition to their originally intended audiences. Captions can provide a powerful search capability, allowing users to search the caption text to locate a specific video, or an exact point in a video. They are also useful for people learning to read or learning English as a second language. Audio descriptions can assist students with learning disabilities by reinforcing through audio what the user is watching on the screen.

Captions and audio descriptions may be integrated into multimedia as a user-selectable option (closed) or permanently recorded along with the main audio or video (open). Closed captions and descriptions may be toggled on and off by the user via a preferences setting, a menu option or, in some cases, a button on the player interface. Open captions and descriptions may not be turned off-everyone sees or hears them, whether they want to or not.

Samples of accessible multimedia delivered in the formats mentioned below, including source code and tutorials, are available from NCAM's Rich Media Access Project.

Creating Accessible Multimedia

There are two formats that support the inclusion of audio descriptions and closed captions in digital multimedia presentations-Synchronized Multimedia Integration Language (SMIL) and Synchronized Accessible Media Interchange (SAMI). SMIL is played by the QuickTime Player (versions 4.1.2 and later); RealPlayer (versions G2 and later); and the Oratrix GRiNS Player. SAMI is played by Windows Media Player only.

SMIL

SMIL was developed by the World Wide Web Consortium (W3C), an international industry consortium that publishes protocols for the Web. The first SMIL recommendation (SMIL 1.0) was published in 1998. SMIL 2.0 was published in 2001 and contains accessibility features not available in SMIL 1.0. These new features are discussed in relevant sections of this document. Take note that not all of SMIL's accessibility features are supported by all SMIL players, however. In these cases, we offer workaround solutions that will work with existing players.

SMIL multimedia presentations are made up of elements-sound, video, pictures and text-that are stored separately and then synchronized at the time of playback. SMIL-formatted multimedia can be delivered via the Internet or locally via a CD- or DVD-ROM. SMIL players include the RealPlayer (G2 or later versions) from RealNetworks, the GRiNS Player from Oratrix and the QuickTime Player from Apple (versions 4.1.2 and later). Visit the W3C's Synchronized Multimedia Web page for complete information about SMIL.

When authored correctly, SMIL allows users to turn captions and descriptions on and off via a player interface. The QuickTime Player, GRiNS Player and RealPlayer each provide a menu selection or dialog box for this feature, but for better accessibility authors should consider adding accessible buttons to the player interface for easier toggling of tracks. In fact, this is crucial when embedding a player into a Web page. Below is a picture of caption and description buttons integrated into the QuickTime Player interface.

D

Alternatively, user preferences relating to captions and descriptions may be stored in a profile on a Web site. On the PIVoT Web site, caption and audio description tracks are controlled in this manner; when a student signs on from any computer, multimedia is delivered in the manner specified in the student's profile. Here is a picture of a user's video preferences, showing caption and description choices.

D

SAMI

SAMI is a Microsoft public specification that allows closed captions to be played in the Windows Media Player. When the Windows Media Player is used as a stand-alone player, viewers can turn the captions on and off using a menu selection. However, when the player is embedded in another application, such as a Web page, the developer must provide the toggling feature through a button in the application's interface. For additional flexibility, this interface can also provide options to change the font size or text color.

At the time of this writing, SAMI does not support closed audio descriptions. Instead, descriptions must be recorded permanently as open descriptions directly into a video's regular soundtrack. If this approach is used, authors should also provide a separate version of the video with the original program audio (without audio descriptions).

FLASH

Flash is an animation technology from Macromedia, Inc. Multimedia presentations authored in Flash can contain large amounts of information yet still remain reasonable in data size, making them easy to download and play. Since Flash uses vector-based graphics, presentations can also be resized without loss of clarity-a boon to users with visual impairments.

A recent development in Flash technology is the ability to include native Flash captions within the presentation itself. A sample captioned Flash movie and complete instructions for current authoring methods can be found at NCAM's Rich Media Resource Web site.

A free utility, the Media Access Generator (MAGpie), can be used to create captions and audio descriptions for SMIL presentations, and captions only for SAMI and Macromedia Flash presentations. MAGpie was created by the CPB/WGBH National Center for Accessible Media (NCAM).

Checkpoint 2.1
Add audio descriptions to multimedia presentations.

Multimedia is often used to convey educational content. Without audio descriptions, students who are blind are at a distinct disadvantage as they have to depend on teachers or classmates to obtain the visual information. Students may be reluctant to ask others to describe the visuals to them, and untrained classmates or even teachers may not provide descriptions that meet the student's specific educational requirements. The techniques explained below show how to add audio descriptions to three popular multimedia formats.

Creating meaningful audio descriptions requires specialized training in how best to convey visual images verbally. The narration should be carefully written to fit precisely into the natural pauses in the program dialog. In cases where a longer description is necessary but there is not a sufficient pause in the program audio to accommodate it, consider authoring extended audio descriptions. In this situation, the program dialog and video are paused while the audio description plays. When the description has finished playing, the video and dialog resume playback. Extended and "regular" descriptions may be mixed in a single multimedia presentation.

When creating described multimedia presentations, it may be best to contact one of the resources listed in Appendix 2, Closed Captioning and Audio Description Resources, since these organizations have expertise in providing audio descriptions for an array of media (broadcast television, movies on video and multimedia). Check with service providers to be sure they can deliver the final product in your preferred file format.

Where budgetary constraints exist, or where it is simply more practical and convenient, students, teachers or others knowledgeable in the specific subject matter may be trained to write audio descriptions. As with image descriptions, audio-description writers must have good writing skills and an excellent command of the vocabulary associated with the subject, as well as adequate access to reference and support materials, such as textbooks and dictionaries, in order to ensure that the descriptions are as clear and accurate as possible. Finally, the descriptions should be reviewed by someone other than the original writer. Training materials used in the Access to PIVoT project may be found in Appendix 5.

Technique 2.1.1
Add audio descriptions to movies using MAGpie.

Both regular and extended audio descriptions can be digitally recorded and integrated into a SMIL presentation using NCAM's free utility, MAGpie 2.01. You may also use any sound-editing program, such as SoundForge, to record the audio files, and then use MAGpie to integrate them into an accessible presentation. No matter how it is done, ensure that the descriptions convey as much information as necessary without being overly long. Describe all important details thoroughly, such as scientific or mathematical expressions, graphs and charts, and use a vocabulary that is appropriate to the subject matter or grade level.

For the best results, record the descriptions in a quiet environment using a high-quality microphone. To minimize errors or ambiguity, speak clearly and use a written script. Finally, when integrating the descriptions into the multimedia, time them precisely to play at the appropriate intervals when they will be the most useful. For example, if the video shows a professor writing an equation on a blackboard, wait until he has finished writing the entire equation before inserting the description describing what has been written.

See the MAGpie Web site for complete information on using the software to create and synchronize audio descriptions.

Technique 2.1.2
Integrate audio descriptions into multimedia presentations using SMIL.

SMIL

SMIL provides information about layout, synchronization and display of different media types for the player to interpret. In addition, SMIL provides test attributes that are used to determine, among other things, the viewer's player preferences for displaying captions and descriptions. The RealPlayer (G2 or later versions) from RealNetworks, GRiNS Player from Oratrix and the QuickTime Player from Apple (versions 4.1.2 and later) all use SMIL to integrate audio descriptions into a multimedia presentation.

You can use MAGpie to author accessible SMIL presentations, as described in technique 2.1.1, but if you wish to write SMIL code yourself, details are provided here. The RealPlayer and GRiNS Player provide reliable support for SMIL 1.0; as of this writing support for SMIL 1.0 in the QuickTime Player is available but less reliable, so for QuickTime presentations it may be safest to embed audio-description tracks directly into the movie as described in technique 2.1.3. See below for an important note regarding SMIL 2.0 and audio descriptions.

SMIL 2.0 and audio descriptions

When creating a described presentation for a SMIL player, authors must write a SMIL file that contains pointers or references to the video file and all audio-description sound files. MAGpie will do this automatically for you. Here is some sample code showing instructions on how and when the player will play the video and audio descriptions.

In this example, the presentation has a duration (<dur>) of 0:01:46.27 (timecodes are represented as hours:minutes:seconds.frames, although other units may also be used). The SMIL file is instructing the player to play the video track (mymovie.rm) in parallel (<par>) with the two audio tracks (ad1.rm and ad2.rm). The first audio description will play at the timecode 0:00:14.30, and the second at 0:00:28.16. The test attribute systemAudioDesc is used to determine whether or not to actually play the audio descriptions, based on the multimedia player's preferences settings. If the user has set the player's preferences to play audio descriptions, those descriptions will be played at the intervals stated in the SMIL file. If not, they will be ignored.

Here are pictures of the RealPlayer and GRiNS accessibility preference settings, showing choices for both audio descriptions and captions.

D

D

Some video clips have insufficient pauses in the dialog or program narration, thus restricting the amount of time available for the insertion of useful audio descriptions. In these cases, SMIL 2.0's "exclusive" element (<excl>) may be used to sequence an "extended" audio description by informing the player that the main media (and other tracks, such as captions) should be paused until the interrupting element-the audio description-has finished playing. These extended audio descriptions will lengthen the overall duration of the presentation, but they provide an effective method for adding description when it is most needed.

The following example shows the code that specifies an extended audio description:

Here, mymovie.rm is paused at 85.5 seconds while the first description (ad1.rm) plays. When ad1.rm is finished playing, the video will resume playing. At 164.5 seconds into the original timeline, the video pauses again while the second description (ad2.rm) plays, then resumes when the description is finished.

SMIL 1.0 and extended audio descriptions

At the time these guidelines were written, support by the RealPlayer, GRiNS Player and the QuickTime Player for SMIL 2.0 extended audio descriptions was not complete. Instead, authors who wish to incorporate extended audio descriptions may do so using a SMIL 1.0 workaround for the RealPlayer or embedded tracks for the QuickTime Player (see later in this section). The SMIL 1.0 solution is illustrated in the code below:

In this sample, mymovie.rm pauses at 85.5 seconds, allowing the first extended description (ad1.rm) to play for its full length of 11.2 seconds (from 85.5 to 96.7 seconds). Once the description has finished playing, the video resumes playing at 85.5 seconds until 164.5 seconds, at which time it pauses while ad2.rm plays for its full length of 82.3 seconds. This process is repeated as many times as there are extended audio descriptions. Note that this workaround only works with the RealPlayer. It is not an ideal solution-among its drawbacks is a brief black flash between the time the audio description ends and the next video segment begins-but it can be used as a temporary solution until multimedia players fully support SMIL 2.0's extended audio description capabilities.

Technique 2.1.3
Embed audio-description tracks in QuickTime movies.

While audio descriptions may theoretically be incorporated into QuickTime presentations using SMIL, Apple's support for SMIL was incomplete at the time these guidelines were written. For complete reliability, authors should embed the description track into the movie itself.

In a QuickTime movie, sound, video and text are contained in separate tracks. Sound tracks to accommodate different languages or audio descriptions can be added easily. The following instructions show how to add audio descriptions that fit into the pauses in the movie's dialog or program narration. The method applies to QuickTime Player for both Macintosh and Windows, with slight variations in keyboard commands:

1. Open the QuickTime Player and the clip to be described. Choose Get Movie Properties from the Movie menu, or press Command-J on Macintosh, or Ctrl-J in Windows.
2. Using any sound-recording software, record and edit the first description to be incorporated into the movie clip. Open this audio clip in a new window in QuickTime Player, then select, or highlight, the entire sound clip (or the portion you want to insert into the movie clip) and choose Copy from the Edit menu.
3. In the QuickTime Player with the video, move the slider to the end of the pause where the description will go. Hold down the Shift key and move the slider backwards until you reach the beginning of this pause. This selects the segment of the movie clip that will receive the audio description.
4. If you're using a Macintosh, hold down the option key, open the Edit menu, and choose Add. If you're using a PC, hold down the Ctrl and Alt keys, open the Edit menu, and choose Add. This adds the description to the movie as a new sound track. In the Get Movie Properties window, click on the left drop-down menu. You will see that "Sound Track 2" is now one of the menu choices.
5. Play the clip to hear the narration you've just inserted.

Repeat steps 2-5 to insert subsequent audio descriptions. Always remember to select the segment that will contain the audio description by first going to the end of that segment and then moving the slider backwards to the beginning. If you do not do this, the QuickTime Player will add the description to the end of the segment, rather than the beginning.

QuickTime can also incorporate extended audio descriptions, using a similar method:

1. Open the QuickTime movie.
2. Find the first spot in the movie where the video will pause and the extended audio description will play.
3. Select Export... from the File menu.
4. At the bottom of the dialog box, change the Export: combo box to Movie to Picture, and the Use: combo box to Uncompressed. This will export the frame that is currently in view. Click on the Save button after noting (or changing) the filename.
5. Open a new QuickTime Player window and then open the file you just created in step 4.
6. Choose Select All from the Edit menu, and then choose Copy from the Edit menu.
7. Using the sound-recording software of your choice, record and edit the first description you want to incorporate into the movie clip.
8. Open a new QuickTime Player window, and open the audio-description you recorded in step 7.
9. Choose Select All from the Edit menu.
10. Hold down the Option and Shift keys on a Macintosh (or the Shift, Control and Alt keys in Windows) and select Add Scaled from the Edit menu.
11. Save the combined sound/image file
12. Choose Select All from the Edit menu and then choose Copy from the Edit menu.
13. Click once on the window with the movie into which the extended audio descriptions will go.
14. Assuming that the slider has not been moved since the image was exported, choose Paste from the Edit menu. If the slider has moved, locate the exact point in the movie where the movie matches the exported image before pasting.
15. Repeat steps 2-14 as needed.
16. Choose Save As... from the File menu. Save your work frequently, always saving the movie as a self-contained file.

Technique 2.1.4
Add audio descriptions to Windows Media.

Windows Media currently has no support for closed audio descriptions, but audio descriptions (regular or extended) can be integrated into a Windows Media presentation by recording them along with the regular program audio. Since these descriptions are open (that is, they can't be turned off), authors should also provide a separate version of the presentation with the original program audio (without audio descriptions).

Checkpoint 2.2
Add closed captions to multimedia presentations.

The inclusion of closed captions in educational multimedia presentations dictates whether the student who is deaf or hard-of-hearing is an active learner or a passive learner when the media is used on-line or in class. While a good percentage of the presentation may be visual, any audio that complements the visual information must be captioned.

Providing closed captions is significantly different from providing a text transcript of the audio portion of the presentation (although it is recommended always to provide a text transcript in addition to captions. Transcripts provide an easy way to scan a movie's script, to search for particular terms, and can also be converted to braille.) For example, captions use special techniques to identify speakers and sound effects. And since the caption text is synchronized with the video, it is easier for viewers to associate the captions with the whole presentation. Because writing captions can be a complex process, it may be best to contact one of the resources listed in Appendix 2, Captioning and Audio Description Resources, to have the work done for you.

Where budgetary constraints exist, or where it is simply more practical and convenient, students, teachers or others knowledgeable in the specific subject matter may also write captions. Anybody with excellent writing, spelling and listening skills can write captions but, if possible, use people who demonstrate sufficient expertise in the subject at hand. Captioners must have adequate access to reference and support materials, such as textbooks and dictionaries, in order to ensure that the captions are as accurate as possible. Finally, the captions should be reviewed for accuracy by someone other than the original writer. Training materials used in the Access to PIVoT project may be found in Appendix 4.

Technique 2.2.1
Write captions for multimedia presentations using MAGpie.

The easiest method for adding captions to multimedia presentations is to use NCAM's free captioning utility, MAGpie. MAGpie allows authors to write captions once and output them in formats for RealNetworks' RealPlayer, Oratrix's GRiNS Player, Apple's QuickTime and Microsoft's Windows Media Player. These players each use proprietary text-display formats, so captions that play in one player will not play in another (except for the GRiNS Player, which plays RealNetworks' RealText format). See the MAGpie Web page for complete information on using the software to create and synchronize captions.

As of this writing, there are serious shortcomings with the display of scientific and mathematical expressions in captions. No standard method exists for displaying complex math or science notation within multimedia players. (Existing mathematical markup languages, such as MathML, are not supported in multimedia players.) Therefore, representing anything beyond simple mathematical expressions in captions can only be accomplished via text.

Research and development efforts will eventually make the proper display of scientific and mathematical expressions in captions possible. The World Wide Web Consortium (W3C) is currently considering defining a standard timed-text format that could eventually be adopted by all multimedia player manufacturers. This new format will, it is hoped, provide support for the display of scientific and mathematical notation. Not only would this improve the capability of captions to convey important mathematical or scientific information; it would also eliminate the need for multimedia players to use proprietary text-display formats.

SVG and Flash captions

One interim solution to the problem of displaying math symbols in captions is to use Scalable Vector Graphics (SVG) in place of text captions. Using SVG, fonts can be embedded in the caption file and rendered in an SVG viewer which itself is incorporated into a multimedia player via a plug-in. SVG captions can be created by transforming a MAGpie 2.01 XML file to SVG-currently a process that requires the use of additional software and a good deal of manual coding. Automated authoring tools are not available at the time of this writing. Currently the only major multimedia player with SVG support is the RealPlayer for Windows, using the Adobe SVG Viewer plug-in.

Below is a example of a complex expression rendered in SVG.

D

Complete instructions for creating SVG captions, as well as a sample SVG-captioned movie, may be found at NCAM's Rich Media Resource Web site.

Macromedia Flash is an animation technology from Macromedia, Inc., that displays vector-based text and graphics. Presentations authored in Flash can contain large amounts of information yet still remain reasonable in data size, making them easy to download and play. Flash presentations can also be resized without loss of clarity-a boon to users with visual impairments.

A recent development from Macromedia is the ability to include native Flash captions playable within the Flash player itself. Multimedia authors can write captions using MAGpie, then use MAGpie's XML project file that works in conjunction with a Flash SWF file for caption display. However, as with SVG captions, this requires a good deal of manual coding and the use of third-party software. Automated support for this process is expected soon. In the meantime, a sample captioned Flash movie and complete instructions for current authoring methods can be found at NCAM's Rich Media Resource Web site. Note that as of this writing it is not feasible to use Flash captions to display mathematical symbols.

Technique 2.2.2
Integrate captions into multimedia presentations using SMIL.

RealPlayer, the QuickTime Player and Oratrix's GRiNS Player use the W3C's Synchronized Multimedia Integration Language (SMIL) to sequence captions in a multimedia presentation. SMIL provides information about layout, timing and display of different media types (such as video and text) for the player to interpret. In addition, SMIL provides test attributes that are used to determine, among other things, the viewer's preferences for display of captions and descriptions.

When creating a captioned presentation for a SMIL player, authors must write a SMIL file that contains pointers or references to the video file and the text file containing the captions. MAGpie will do this automatically for you (see technique 2.2.1). If you need to author a captioned SMIL presentation in some other way, create a file using the code samples below as a guide. The first example shows the <layout> section of a SMIL file.

In this example, information between the <layout> tags instructs the player to draw two regions, one for video (videoregion) and one for text (textregion). Height and width sizes are given in pixels.

Below is some sample code showing the body of the SMIL file, where the player is given instructions on how and when to play the video and captions.

The information between the parallel (<par>) tags states that the two source files (one for video and one for captions) should be started at exactly the same time and be played in parallel, thus synchronizing the captions with the program audio. Since no start time is specified, SMIL assumes all elements will begin playing at time 0:00:00.0 (timecodes are represented as hours:minutes:seconds.frames, although other units may also be used). The systemCaptions test attribute is used to determine whether or not to actually play the captions, based on the multimedia player's preferences settings. If the user has set the player's preferences to play captions, those captions will be played at the intervals stated in the text file. If not, they will be ignored.

Here are pictures of the RealPlayer and GRiNS accessibility preference settings, showing choices for both audio descriptions and captions.

D

D

Here are pictures of a movie with and without closed captions.

D

D

The caption file contains the actual captions and timecodes for the multimedia presentation, as shown below.

Note: This example shows a RealText file for use with the RealPlayer and the GRiNS Player. The QuickTime text format, QText, playable only with the QuickTime Player, looks slightly different:

Technique 2.2.3
Integrate captions into multimedia presentations using SAMI.

SAMI (Synchronized Accessible Multimedia Interchange) is an HTML-based format from Microsoft used to display captions within the Windows Media Player. Because it is HTML-based, it can take advantage of flexible display properties of that format. It is only playable on Windows Media Player for the PC.

As with SMIL, SAMI caption files can be created with MAGpie. See the MAGpie Web site for complete information.

Below is an excerpt from a SAMI file.

A major difference between SAMI and the SMIL approach to caption display is that SAMI keeps caption text, formatting and timing instructions together in one file, whereas SMIL does not. In the example above, formatting information is held in the <STYLE> section of the file, and the captions plus timing information are held in the <BODY>. Storing caption and style information in one file has advantages over separate storage, among them the fact that administrators have to keep track of only one file on the server.

NEXT | CONTENTS