Adding Extended Audio Descriptions with SMIL 1.0

SMIL 1.0, unlike SMIL 2.0, does not have any formal mechanism for adding audio descriptions which pause the timeline of the main media. However, at the time this document is being created, the major players do not yet support SMIL 2.0, so this technique provides a way to include extended audio descriptions with SMIL 1.0. This technique is fine in theory, but often the players that support smil 1.0 have a difficult time, so we suggest thorough testing.

View a sample SMIL 1.0 movie with extended descriptions. The example works acceptably, but you may notice a black flash when successive video clips start.

Sample SMIL and explanation

<body>

  <par>
   <seq>

     <par>
       <video src="video.rm" region="videoregion" clip-begin="0s" clip-end="5.4" 
       dur="8.7" fill="freeze"/>   
       <audio src="no1.wav" begin="5.4"/>
     </par>

     <par>
       <video src="video.rm" region="videoregion" clip-begin="5.4" clip-end="24.1" 
       dur="20.3" fill="freeze"/>
       <audio src="no2.wav" begin="18.7"/>
     </par>

     <par>
       <video src="video.rm" region="videoregion" clip-begin="24.1" clip-end="29.6" 
       dur="7.7" fill="freeze"/>
       <audio src="no3.wav" begin="5.5"/>
     </par>

     <par>
       <video src="video.rm" region="videoregion" clip-begin="29.6" clip-end="34.5" 
       dur="5.7" fill="freeze"/>
       <audio src="no4.wav" begin="4.9"/>
     </par>

     <par>
       <video src="video.rm" region="videoregion" clip-begin="77.4"/>
     </par>

   </seq>
  </par>
      
</body>

The markup above is broken into five <par> segments. In each, there is a <video> and an <audio> tag (the last <par> has no <audio> tag intentionally). The convention with extended audio desciptions is that the main media pauses during the description. The way to make this happen in SMIL is to set a "clip-begin" and "clip-end" which dictate the start and end of the video clip, and to set a duration for the clip that is longer than what is defined by the "clip-begin" and "clip-end". The fill="freeze" holds the last frame of the video during the extended description. The <audio> tag has a "begin" attribute with a value that is equal to the "clip-end" value of the preceeding <video> tag.

The way to determine the values for "clip-begin", "clip-end", and "dur" is to find out the time the portion of the video before the audio description starts and ends, and to find out the total length of the extended audio description. The "clip-begin" and "clip-end" define their own values, but the "dur" value is the sum of the length of the extended description and the clip defined by the "clip-begin" and "clip-end". In the first <par>, the video clip starts at 0 seconds, ends and 5.4 seconds, and the description length is 3.3 seconds, so the "dur" value is 5.4s + 3.3s = 8.7s.

The math is simple and repetitive. If you change an audio description you will most likely need to modify all of the 'downstream' timecodes. We created a spreadsheet that does the math for you, to help maintain your sanity (Excel 4.0). If you need a different spreadsheet format, please contact ncam_tech@wgbh.org.

Back to Audio Descriptions in Rich Media