Showing posts with label Subtitles. Show all posts
Showing posts with label Subtitles. Show all posts

A Scalable System for Ingestion and Delivery of Timed Text

Offering the same great Netflix experience to diverse audiences and cultures around the world is a core aspect of the global Netflix video delivery service. With high quality subtitle localization being a key component of the experience, we have developed (and are continuously refining) a Unicode standard based i18n-grade timed text processing pipeline.  This pipeline allows us to meet the challenges of scale brought by the global Netflix platform as well as features unique to each script and language.

In this article, we provide a description of this timed text processing pipeline at Netflix including factors and insights that shaped its architecture.

Subtitles in an IP World

An end-to-end demonstration of EBU-TT-D subtitles being delivered via MPEG DASH and displayed by a client.

DPP Launches Access Services Standards

The UK's Digital Production Partnership (DPP) has agreed an industry document format standard for the exchange of subtitles for the hard of hearing and audio description.

ligned with the EBU’s new subtitle document format (EBU-TT), produced in July 2012, the new standard has been created to support UK broadcasters’ requirements for subtitle and audio description script transfer.

The DPP’s subtitle document format extends EBU-TT with metadata required to support the workflow for delivering prepared subtitles and captured live subtitles in addition to the scripts used during the production of audio description.

All DPP subtitle documents are valid EBU-TT, itself based on the W3C TTML recommendation. The format separates the text and it’s associated display timing from information about that text, such as where it should be placed on the screen, the font style, size and colour, and separate metadata such as whether a given subtitle is describing dialogue, music, sounds effects etc., and what language the subtitles are in.

The format is a flavour of XML whose format can be validated using off the shelf tools and extended to meet specific requirements while still being interchangeable. In addition it can support all Unicode characters and arbitrary fonts, should downstream platforms support them. EBU is in the process of finalising the draft guidance for converting from the legacy STL format to EBU-TT.

DPP’s subtitle document format will allow UK broadcasters and access service providers to move away from legacy formats, proprietary or otherwise, and towards an open future-facing format that can be used to provide subtitles on broadcast television and online. By agreeing this format before making the transition, DPP is able to lay the groundwork for a common UK interchange format that will benefit all businesses that need to exchange these documents.

Companies that manufacture access services authoring and processing tools have a clear target format, which is vital in a historically fragmented marketplace. Broadcasters and distributors similarly will have a lower cost of adoption of this richer format that is not encumbered by the constraints of legacy formats.

Kevin Burrows, DPP Technical Standards Lead and CTO Broadcast & Distribution, Channel 4, said, “This new subtitle standard, encompassing the existing EBU TT specification, will allow for the display of subtitles on current and future consumer platforms by the UK broadcasters. This will benefit viewers by enabling a consistent viewer experience across their services.”

Source: Digital Production Partnership

EBU Puts Subtitles On Line

The EBU has published a new specification for the distribution of subtitles: EBU-TT-D (Tech 3380). The XML based EBU-TT-D format is a low-complexity way to combine subtitle text, styling, timing information, and positioning details to allow implementers to provide users with a subtitle experience at least as good as that on current TVs, regardless of the platform on which they are watching the content.

EBU-TT-D was developed in less than a year, by taking into account expertise from users, distribution parties, hybrid TV organizations and CE manufacturers. The work built on the EBU XML Subtitles group’s knowledge gained when creating the EBU-TT (EBU Tech 3350) subtitle format for production interchange and archiving.

The specification is derived from the base W3C TTML specification. It strongly constrains the feature set of TTML to make it easier for decoder/renderer implementers to add subtitle overlays to video without the complexity that is present in TTML to support other scenarios.

Work is in progress in HbbTV and DVB to reference EBU-TT-D within the upcoming HbbTV 2.0 and DVB DASH standards. The EBU has also published the first carriage specification document for EBU-TT-D, EBU Tech 3381 v0.9, which defines how to carry EBU-TT-D in ISO BMFF, itself a necessary step for distributing EBU-TT-D via DASH. This builds on work done by MPEG, not yet published in international standard form.

Source: EBU

Captioning for Streaming Video still a "Wild West"

Captioning video on demand “is a wild west,” at least in the U.S, where the service is mandated by law and therefore a key topic for content owners and their partners to get their heads around.

Delivering a measured analysis of the issues, Telestream product manager Kevin Louden captioned his own session, "Practicalities of Putting Captions on IP-Delivered Video Content" with the question: "Can anyone see your subtitles?"

Louden began by pointint out that there are legal, moral, and business reasons to make sure your content is captioned.

“The 21st Century Communications Act in the U.S. mandates that content previously broadcast or intended for broadcast have captions to it,” he explained. “This comes into effect in stages between now and 2016.

“Even if you don't do it by law [no other region of the world has quite the same legislation] some people say it's simply the right thing to do, and from a business perspective you can broaden audiences for your content by reaching out to multiple language groups.”

So how is it done? Just as there are lots of different video and audio formats for streaming and progressive download protocols there are lots of caption file formats for video on demand, the main ones being W3C TT/DFXP and WebVTT/SRT.

The former is an open standard which contains lot of information about position, font size, color, and so on for a rich presentation of the information and is “potentially very complicated,” he said.

WebVTT/SRT, on the other hand, is a text-based format native to HTML5 video tags, “very simple in its current iteration” but with little or no control of presentation features in the file.

“This is what people cobbled together before there were any standards in place, and because of that there are a lot of entrenched workflows,” Louden said.

To smooth the multiplicity of formats, two leading standards bodies are attempting to create a universal file interchange format, as a sort of mezzanine or master level.

SMPTE 2052, being proposed in the U.S., is an XML-based time text file which emerged from the Act so that content owners or their partner organisations could create deliverable formats from broadcast content for IP distribution in all its streaming media end user forms.

In Europe, EBU-TT is a similar proposition and a subset of the TTML format, for use as a universal handoff.

For organisations wanting to generate captioning information for linear video on their own websites there are several options. JW Player, for example, has in built support for WebTT, SRT and DFXP while Flow Player supports W3C TT, SRT

Numerous video encoding tools, perhaps already in situ at a facility, contain subtitling and captioning capabilities for translating between formats.

Alternatively one can employ graphical overlays or physically burn the subtitles onto the picture, a pracitce which is still remarkably common, reported Louden. “You don't need special players or sidecar files, but obviously there's not much flexibility.”

Charging a third-party service provider is a useful way of delegating the problem but, says Louden, “in theory you hand over your master SMPTE TT or EBU TT safe harbour file as the interchange format, but the reality is that people are used to their own existing profiles and will request an SRT, WebVTT format since this is the way it's always been done.”

Turning to adaptive bitrate provision, Louden noted that the main ABR formats cater for different captioning files.

The HLS specification for iOS devices contains a means of embedding 608 captions in a video's MPEG headers, while Smooth Streaming and HTTP Dynamic Streaming both support the sidecar formats DXFP and TTML (useful for repurposing linear and non-linear VoD). Where MPEG-DASH fits into this equation is up in the air.

Louden pointed out a couple of bumps in the road for anyone looking to caption their content, which included taking care of rights, especially when repurposing legacy broadcast content.

“If you sent the work out to a caption house then beware that many of them work on individual negotiations, so while you may have a licence to broadcast that information you may not have the web rights for it,” he advised.

“Also be careful editing content," he said. "Any retiming of the content will have a knock-on to the timecode-synced caption information. You have to be sure when you do your format translation that the captions are retimed too, perhaps manually.”

The demand for a universal captioning standard was agreed on by delegates in the room, but no one really believed that a standard could be agreed or made to work in practice because of commercial pressures among competing vendors.

By way of addendum: Louden noted the little differences in definition between the two continents.

“In the U.S. 'captions' display text and other sound information for the hearing impaired," he said. "Subtitles are translations to different languages, whereas in Europe both of these things are commonly referred to as the same thing—a subtitle.”

By Adrian Pennington, Streaming Media

Live Streaming of Video and Subtitles with MPEG-DASH

This presentation was made at the MPEG meeting in Shanghai, China, in October 2012, related to the input contribution M26906. It gives the details about the demonstration made during the meeting.

This demonstration showed the use of the Google Chrome browser to display synchronized video and subtitles, using the Media Source Extension draft specification and the WebVTT subtitle format. The video and DASH content was prepared using GPAC MP4Box tool.


A Guide to Closed Captioning for Web, Mobile, and Connected TV

Captioning is coming to Internet video. Legislation goes into effect in the US during 2012 and 2013 that mandates closed captioning on certain categories of online content – see Zencoder's post for details on the legislation. But even apart from this legislation, closed captioning is a good thing for accessibility and usability, and is yet another milestone as Internet video marches towards maturity.

Unfortunately, closed captioning is not a single technology or “feature” of video that can be “turned on”. There are a number of formats, standards, and approaches, ranging from good to bad to ugly. Closed captioning is kind of a mess, just like the rest of digital video, and is especially challenging for multiscreen publishers.

How Closed Captions Work
The first thing to understand is how closed captions are delivered, stored, and read. There are two main approaches today:

  • Embedded within a video: CEA-608, CEA-708, DVB-T, DVB-S, WST. These caption formats are written directly in a video file, either as a data track or embedded into the video stream itself. Broadcast television uses this approach, as does iOS.

  • Stored as a separate file: DFXP, SAMI, SMPTE-TT, TTML, EBU-TT (XML), WebVTT, SRT (text), SCC, EBU-STL (binary). These formats pass caption information to a player alongside of a video, rather than being embedded in the video itself. This approach is usually used by browser-based video playback (Flash, HTML5).

What about subtitles? Are they the same thing as closed captions? It turns out that there are three main differences:
  • Goals: Closed captions are an accessibility feature, making video available to the hard of hearing, and may include cues about who is speaking or about what sounds are happening: e.g. “There is a knock at the door”. Subtitles are an internationalization feature, making video available to people who don’t understand the spoken language. In other words, you would use captions to watch a video on mute, and you would use subtitles to watch a video in a language that you don’t understand. (Note that this terminological distinction holds in North America, but much of the world does not distinguish between closed captions and subtitles.)

  • Storage: Historically, captions have been embedded within video, and subtitles have been stored externally. This makes sense conceptually, because captions should always be provided along with a video; 100% accessibility for hard-of-hearing is mandated by legislation. Whereas subtitles are only sometimes needed; a German-language video broadcast in Germany doesn’t need to include German subtitles, but that same video broadcast in France would.

  • Playback: Since captions are passed along with the video and interpreted/displayed by a TV or other consumer device, viewers can turn them on and off themselves at any time using the TV itself, but rarely have options for selecting a language. In these situations when subtitles are added for translation purposes, they are generally hard subtitles and thus cannot be disabled. However, when viewing DVD/Blu-Ray/VOD video, the playback device controls whether subtitles are displayed, and in which language.

Formats and Standards
There are dozens of formats and standards for closed captioning and subtitles. Here is a rundown of the most important ones for Internet video:
  • CEA-608 (also called Line 21) captions are the NTSC standard, used by analog television in the United States and Canada. Line 21 captions are encoded directly into a hidden area of the video stream by broadcast playout devices. If you’ve ever seen white bars and dots at the top of a program, that’s Line 21 captioning (more information.)

  • An SCC file contains captions in Scenarist Closed Caption format. The file contains SMTPE timecodes with the corresponding encoded caption data as a representation of CEA-608 data.

  • CEA-708 is the standard for closed captioning for ATSC digital television (DTV) streams in the United States and Canada. There is currently no standard file format for storing CEA-708 captions apart from a video stream.

  • TTML stands for Timed Text Markup Language. TTML describes the synchronization of text and other media such as audio or video. See the W3C TTML Recommendation for more.

    Example:
    <tt xml:lang="" xmlns="http://www.w3.org/ns/ttml">
    <head>
    <styling xmlns:tts="http://www.w3.org/ns/ttml#styling">
    <style xml:id="s1" tts:color="white" />
    </styling>
    </head>
    <body>
    <div>
    <p xml:id="subtitle1" begin="0.76s" end="3.45s">
    Trololololo
    </p>
    <p xml:id="subtitle2" begin="5.0s" end="10.0s">
    lalala
    </p>
    <p xml:id="subtitle3" begin="10.0s" end="16.0s">
    Oh-hahaha-ho
    </p>
    </div>
    </body>
    </tt>

  • DFXP is a profile of TTML defined by W3C. DFXP files contain TTML that defines when and how to display caption data. DFXP stands for Distribution Format Exchange Profile. DFXP and TTML are often used synonymously.

  • SMPTE-TT (Society of Motion Picture and Television Engineers – Timed Text) is an extension of the DFXP profile that adds support for three extensions found in other captioning formats and informational items but not found in DFXP: #data, #image, and #information. See the SMPTE-TT standard for more.

    SMPTE-TT is also the FCC Safe Harbor format – if a video content producer provides captions in this format to a distributor, they have satisfied their obligation to provide captions in an accessible format. However, video content producers and distributors are free to agree upon a different format.

  • SAMI (Synchronized Accessible Media Interchange) is based on HTML and was developed by Microsoft for products such as Microsoft Encarta Encyclopedia and Windows Media Player. SAMI is supported by a number of desktop video players.

  • EBU-STL is a binary format used by the EBU standard, stored in separate .STL files. See the EBU-STL specification for more.

  • EBU-TT is a newer format supported by the EBU, based on TTML. EBU-TT is a strict subset of TTML, which means that EBU-TT documents are valid TTML documents, but some TTML documents are not valid EBU-TT documents because they include features not supported by EBU-TT. See the EBU-TT specification for more.

  • SRT is a format created by SubRip, a Windows-based open source tool for extracting captions or subtitles from a video. SRT is widely supported by desktop video players.

  • WebVTT is a text format that is similar to SRT. The Web Hypertext Application Technology Working Group (WHATWG) has proposed WebVTT as the standard for HTML5 video closed captioning.

    Example:

    WEBVTT

    00:00.76 --> 00:03.45
    <v Eduard Khil>Trololololo

    00:5.000 --> 00:10.000
    lalala

    00:10.000 --> 00:16.000
    Oh-hahaha-ho


  • Hard subtitles (hardsubs) are, by definition, not closed captioning. Hard subtitles are overlaid text that is encoded into the video itself, so that they cannot be turned on or off, unlike closed captions or soft subtitles. Whenever possible, soft subtitles or closed captions are generally be preferred, but hard subtitles can be useful when targeting a device or player that does not support closed captioning.

Captioning for Every Device
What formats get used by what devices and players?:
  • Flash video players can be written to parse external caption files. For example, JW Player supports captions in SRT and DFXP format.

  • HTML5 captions are not yet widely supported by browsers, but that will change over time. There are two competing standards: TTML, proposed by W3C, and WebVTT, proposed by WHATWG. At the moment, Chrome has limited support for WebVTT; Safari, Firefox, and Opera are all working on WebVTT support; and Internet Explorer 10 supports both WebVTT and TTML.

    Example:
    <video width="1280" height="720" controls>
    <source src="video.mp4" type="video/mp4" />
    <source src="video.webm" type="video/webm" />
    <track src="captions.vtt" kind="captions" srclang="en" label="English" />
    </video>

    Until browsers support a format natively, an HTML5 player framework like Video.js can support captions through Javascript, by parsing an external file. (Video.js currently supports WebVTT captions.)

  • iOS takes a different approach, and uses CEA-608 captions using a modified version of CEA-708/ATSC legacy encoding. This means that, unlike Flash and HTML5, captions must be added at the time of transcoding. Zencoder can add captions to HTTP Live Streaming videos for iOS.

  • Android video player support is still fragmented and problematic. Caption support will obviously depend on the OS version and the player used. Flash playback on Android should support TTML, though very little information is available.

  • Some other mobile devices have no support for closed captions at all, and hard subtitles may be the only option.

  • Roku supports captions through external SRT files.

  • Some other connected TV platforms do not support closed captioning yet. But they will soon enough. Every TV, console, cable box, and Blu-Ray player on the market today wants to stream Internet content, and over the next year and a half, closed captioning will become a requirement. So Sony, Samsung, Vizio, Google TV, et al will eventually make caption support a part of their application development frameworks. Unfortunately, it isn’t yet clear what formats will be used. Most likely, different platforms will continue to support a variety of incompatible formats for many years to come.

Closed Captioning for Internet Video: 2012 Edition
The landscape for closed captioning will change and mature over time, but as of 2012, here are the most common requirements for supporting closed captioning on common devices:
  • A web player (Flash, HTML5, or both) with player-side controls for enabling and disabling closed captioning.

  • An external file with caption data, probably using a format like WebVTT, TTML, or SRT. More than one file may be required – e.g. SRT for Roku and WebVTT for HTML5.

  • A transcoder that supports embedded closed captions for HTTP Live Streaming for iPad/iPhone delivery, like Zencoder. Zencoder can accept caption information in a variety of formats, including TTML, so publishers could use a single TTML file for both web playback and as input to Zencoder for iOS video.

Beyond there, things get difficult. Other input formats may be required for other devices, and hard subtitles are probably necessary for 100% compatibility across legacy devices.

Source: Zencoder

New EBU Subtitling Specification Published

The EBU has published EBU-TT part 1, with TT standing for Timed Text. It's a follow-up to the widely used EBU STL specification, which was originally published in 1991. The new format is XML-based, which makes it “human readable” and more suited to modern integrated file-based production methods.

EBU-TT is a simplified version of the W3C Timed Text specification, which means it fits well into the broad family that includes W3C TTML and SMPTE TT, which is more focused on the US environment and on distribution. EBU-TT was developed by the EBU's XML Subtitles group, chaired by Andreas Tai of IRT.

Part 1 (Tech 3350) has been published, defining a structure for the interchange and archiving of subtitles; part 2, now being drafted, will provide mapping guidance for users who want to migrate from EBU STL to EBU-TT. Work has also started towards a specification for live subtitling, with a workshop scheduled for 9 August.

Source: EBU

What is Timed Text?

A nice video introduction to Timed Text by Bruce Devlin.

EBU-TT Subtitling Format Published for Industry Comments

The EBU has published a new Subtitling Format specification (EBU Tech 3350). The new format is called EBU Timed Text (EBU-TT) and provides an easy-to-use method to interchange and archive subtitles in XML.

EBU-TT is based on the W3C Timed Text Markup Language (TTML) specification. The EBU format can be seen as a constrained version of the W3C spec, aimed at providing a solution more tailored to broadcast operation. This is especially relevant as broadcasters are increasingly moving to file-based HDTV facilities, where subtitles are created, edited, exchanged and archived together with the content.



The previous EBU subtitling format was EBU STL (Tech 3264), developed at a time when information was still exchanged on floppy discs. However, as many broadcasters still use STL or have archived STL files, great care was taken in the development of EBU-TT to make sure that it provides backwards compatibility with its predecessor.

The EBU is also providing an XML Schema for EBU-TT.

Source: EBU

Lack of 3-D Captioning Standard Stymies Development

As more content is being produced in 3-D, the need for captioning, now mandated by the U.S. government, has been brought to the forefront. While all of the vendors in this category are aware of the need to do it, very few customers have asked for it, which holds back development.

“We certainly have the capability to produce captions in 3-D space, but we’re not investing a lot in R&D until there is customer demand and a standard specification for how to do it,” said José M. Salgado, president and CEO of Los Angeles-based SoftNI, a veteran captioning and subtitling software provider.

To be clear, the issue has to do with closed-captioning, not necessarily “subtitling.” 3-D subtitling is typically predetermined by the content producer and is inserted into a plane (below, on the side or on top of the screen) that’s most aesthetically pleasing to the eye. Because subtitles are simply a part of the picture, there is no need for new technology to transmit or display them.

Closed-captioning, on the other hand, serves a greater need and must be done uniformly. This data is sent as text with timing information by a broadcaster or program provider and turned on or off at the TV set by the consumer. There is a method for doing this in 2-D (called CEA-708) that’s standardized by the Consumer Electronics Association. Every TV set sold in the United States must be able to recognize this code and display it when required. In that code, you can still control the positioning of the captioning but not the 3-D depth. The result is that the 3-D experience is often not the best it could be.

“Captions are still transmitted in 2-D, even for 3-D content, but there is no way yet to make use of 3-D depth in the captions,” said Jason Livingston, product manager at Computer Prompting & Captioning (CPC). “All of the 3-D TV sets sold today can only decode 2-D caption information. It’s a problem that people are starting to be aware of, but we’re a long ways from having an industrywide agreement.”

Currently, captioning material is sent to 3-D TV sets in the same manner as 2-D HDTV. Captioners can control the 2-D placement of the captions, just like they do now. As long as the captioner does a good job, it will not obscure anything important, and the portion that is obscured will be the same regardless of whether the video is 2-D or 3-D.

This has frustrated some viewers because all of the work that goes into framing a 3-D scene and the depth perception is lost when a caption box covers it. With no standard way of accommodating multiple layers within a scene, there’s no control over space and depth.

“You’ll still see closed-captions appearing where people are used to seeing them, but there’s no code for ensuring a pleasing 3-D viewer experience,” Livingston said. “As soon as the CEA and FCC establish a technical standard for how to transmit 3-D closed-captions, then it will be in our software. From a manufacturer’s perspective, it does not make sense to do it until the technical standards have been decided and published.”

For the time being, Livingston recommends that his customers produce closed-captioning in the same way they currently do for 2-D content.

“Captioning vendors don’t have any say in how the industry will ultimately decide how to handle 3-D closed-captions, and we can’t tell the TV set manufacturers, ‘This is the code we want you to use,’ because they won’t implement it until the CEA establishes an industry standard.” Livingston said. “So, we’re all waiting and doing a few tests until then.”

Unlike closed-captioning, which is transmitted separately from the picture and can be turned on and off, subtitles are burned into the picture and cannot be turned off by the viewer. But, captioning companies such as CPC and SoftNI are ready to offer 3-D subtitles in their software today.

“We’re just waiting for demand,” SoftNI’s Salgado said.

SoftNI offers its Subtitler Suite and Digital Suite software products for subtitle burn-in and metadata insertion into HD and SD digital files. CPC’s MacCaption (for Apple computers) or Caption Maker (for PCs) has the ability to encode closed-captioning and burn-in subtitles as well. Other vendors include Cheetah International and service providers National Captioning Institute and Boston public TV station WGBN.

“We’ve had some viewers ask about 3-D subtitling, but we have not much interest from content creators yet,” CPC’s Livingston said. “The basic structure is there in our products for doing it, but we’re still working on the user interface and trying to understand what tools content creators want.”

The fact that captioning is done in software bodes well for these captioning companies, because as soon as a standard is announced, the software can be easily upgraded via a free download to accommodate 3-D captions — and it can be done in a matter of weeks.

Adding to the issue, the government recently signed the Twenty-first Century Communications and Video Accessibility Act (S. 3304), which mandates captioning for Web-delivered content that has also appeared on traditional broadcast TV. The legislation also states that all CE receiving devices large enough for video must be equipped to support captioning functionality. So, a new set of concerns will become apparent in 2011 because there are a number of Web display formats that don’t support closed-captioning at all, and those that do use a number of incompatible standards

However, a number of vendors, including the ones mentioned here, support the Described and Captioned Media Program (DCMP) and can help add captions to Web-based video. For a list, visit this link.

By Michael Grotticelli, Broadcast Engineering

Safe Graphic Insertion for Stereographic Material

This white paper describes a proposed format and methodology that assures that a consistently matched quality of overlay is applied to stereographic material as is currently enjoyed and expected by the viewing public in 2D.

By Jonathan Jenkyn and Simon Hailes, Screen Subtitling Systems

Subtitling for Stereographic Media

This white paper describes and summarises the stereographic delivery mechanisms being considered and their implications for subtitling. It also summarises some proposed recommendations and solutions with regard to these mechanisms and implications.

Source: Screen Subtitling Systems

SDI Media & Sony to Create 3D Subtitling Placement Solution

SDI Media announced that Sony Creative Software has adopted and integrated SDI's XML schema into their Z-Depth 3D subtitle editing application. Z-Depth creates the information needed for proper placement of subtitles or menus in the 3D space of a 3D title presentation.

The purpose of this document is to propose a standardized XML based file format for storing Z-axis data for 3D subtitles applicable to Theatrical (Digital Cinema, Digital Intermediates), Blu-Ray and Broadcast venues. Recording only the changes in Z values over the course of the picture should be sufficient as it eliminates the redundancy of recording individual Z values for each frame of the picture.

Source: SDI Media