Gracenote Readies its Own Second-Screen Platform

Gracenote is about to introduce its very own second-screen content recognition platform at CES. The company, which became a wholly owned subsidiary of Sony three years ago, aims to compete with similar solutions from Yahoo’s IntoNow and social check-in services like Miso and GetGlue.

Gracenote’s advanced content recognition technology makes it possible to identify both on-demand movies as well as live TV content. Gracenote President Stephen White gave me a quick demo of the technology last week in San Francisco.

Click to watch the video

Gracenote’s service is comparable to IntoNow in that it uses a tablet’s microphone to listen to the audio track of what’s playing on TV. It then checks the resulting fingerprint against a growing database of video content to deliver information to the tablet — a process that takes five seconds or less, according to White.

He told me Gracenote’s service will eventually be able to deliver rich scene-level metadata for tens of thousands of movies, going as far as offering links to buy any of the products shown on-screen. Information for hundreds of movies will be available when the service launches in earnest next March.

Contextual information for live TV isn’t quite as deep, but Gracenote uses its partnership with Tribune Media Services to cross-reference TV Guide data with what a user is currently watching. Gracenote wants to offer its advanced content recognition platform to CE manufacturers, broadcasters and developers of second screen apps, White explained.

This is technically not the first time the company has been powering this kind of second-screen experiences: Gracenote’s subsidiary Gravity Mobile built the technology behind the ABC second-screen apps that are powered by Nielsen. However, those apps are based on watermarks incorporated into the shows ABC build its apps for, which obviously requires the cooperation of the broadcaster.

Gracenote’s fingerprinting approach, on the other hand, works with any kind of content. That means third parties could also use this kind of technology to sell ads against content they don’t own or control. “It’s definitely disruptive,” admitted White. However, he said broadcasters for the most part are very interested in second-screen solutions, if only to defend their own ad dollars. Said White: “The broadcasters realize that it’s gonna happen with our without them.”

By Janko Roettgers, GigaOM

Startup Umami Serves Side of iPad Content For TV

New York-based startup Umami will jump into the "second screen" fray with the expected release in the next few weeks of an iPad app, free to consumers, that will serve up contextually relevant content for shows on 40 broadcast and cable networks.

Umami fingerprints the audio in TV content across the 40 networks using a large-scale digital video recorder system. When a user fires up the app, it "listens" for which channel is currently on by comparing it to the Umami fingerprint database, then pulls up news, cast pages, episode guides and social media feeds from various sources in a flipbook-like format. The system works on DVR recordings, too.

The business model for the year-old firm is to land deals with TV networks and producers, to deliver ads and show-related material to avid fans.

"We wanted to make publishing to our platform dirt simple," Umami CEO Scott Rosenberg said. He claimed Umami already has several media partners lined up, though he declined to identify them.

Umami (pronounced "ooh-MA-mee") is a Japanese word that refers to a fifth taste of "savoriness." The idea: the app is like a flavor-enhancer for TV.

The company faces a slew of competitors, ranging from Nielsen -- whose MediaSync product uses audio watermarks for second-screen apps with live TV -- to Shazam Entertainment, Yahoo's IntoNow, Invidi Technologies and Spot411 Technologies. There's even another New York-based startup called SecondScreen Networks.

Rosenberg sees his primary competitor as networks trying to build apps themselves, a proposition he notes is time-consuming and results in a show- or network-specific app that's a narrow slice of the entire TV viewing experience.

On the other hand, an app like MTV's WatchWith, which provides content synchronized with the networks' top primetime shows, could coexist with Umami. "We don't think those activities are mutually exclusive," he said.

The founders tout their experience in TV and new media. Rosenberg most recently served as vice president of advanced advertising at Rovi, and has worked at BlackArrow, Intel and ReplayTV. Umami chief technology officer Bryan Slavin has worked at broadband video ad firm Lightningcast (acquired by AOL), Leap Wireless and BroadSoft.

The startup formed in the summer of 2010 and this spring raised $1.65 million in seed funding from Battery Ventures, New Enterprise Associates and independent investors.

By Todd Spangler, Multichannel News

How Anthony Rose Plans to Revolutionise TV

I have seen the future of TV and it is called Zeebox. The next project from Anthony Rose, the technologist who built KaZaA and BBC iPlayer in to some of the most disruptive digital media plays, is due to go live in October. Topped by Peoplesound founder and ex EMI SVP Ernesto Schmitt as CEO, the pair’s startup raised $5 million in seed funds from unidentified investors in June and has been operating in stealth as “tBone”. But it has been renamed and has just located at offices at London’s Covent Garden, where it has a staff of almost 30 (including former iPlayer engineers) and where the pair showed me an exclusive demo…

What is Zeebox?
Attempting to ride both the multi-screen TV engagement trend and the increasing adoption of internet TVs, Zeebox is a real-time system for social TV viewing and for engaging deeply around those shows that depends on recognising sofa-based second screens as the place for innovation.

The free Zeebox app for iPad (and, later, iPhone and Android) is a TV guide that displays what shows Facebook and Twitter friends are watching. Owners of compatible connected TVs can flip channel straight from the app, as though it were a remote control. Although the command takes place over the internet, the change happens as quick as or quicker than some standard infrared remotes.

Viewing Together
Notifications appear on-screen to indicate friends’ presence in channels. Users can chat in the iPad app and send invites to join one another for simultaneous viewing - accepting an invite results in the channel changing. “Jack, come and watch The Apprentice with me,” Schmitt tells me, by means of example.

As well as these personal connections, Zeebox users’ collective actions can shape the experience. The app displays real-time data for which shows are “trending” up or down. In a scenario Rose presents, a notification appears to say Top Gear is currently “hot” (perhaps Jeremy Clarkson has said something particularly egregious). The opportunity to surface breaking news in this serendipitous way is clear, along with the prospect of improving TV ratings measurement with actual real-time data.

Making TV Hyperlinked
But this “second-by-second” approach is the fabric of more than just Zeebox’s social interaction. Using both commercially licensable broadcast metadata and frame-by-frame analysis of live TV pictures and audio, Zeebox will apparently understand exactly what is on the TV screen at any given moment (“just as Google spiders the web”), in order to serve up all manner of related material on the handheld app.

As example, Schmitt shows how, whilst Tom Cruise is interviewed on Top Gear, the app will auto-display “infotags” for spoken topics (say, “Ferrari 458”, “Abu Dhabi”, “Sebastian Vettel” and “Tom Cruise” himself), as Cruise is speaking. Each topic becomes an in-app link to a corresponding piece of online content, on Wikipedia, IMDB, iTunes Store or whatever.

The method involved is Zeebox’s “secret sauce”, the subject of a pending patent application, but it’s called automated content recognition (ACR), a field with several vendors including Civolution.

“As context emerges on TV, these infotags just keep ticking up,” Schmitt says. “I find this so unbelievably exciting. Anything being discussed on TV is right there for you.” Or, as Rose puts it: “It’s like crack - you just keep wanting stuff, and getting it second-by-second. TV just becomes better.”

One of the intended uses of “infotags” is commerce. Schmitt wants viewers to be able to buy things relating to what they see on screen. As I flip channel to QVC, he assures me Zeebox will know what’s on-screen is a cubic zirconia ring - and offer me more information, as well as ways to buy that ring.

Programme Context
Rose wants in-app TV show pages to display live tweet streams as well as broadcaster-owned HTML “widgets” for custom show engagement. “BBC Red Button’s non-interactive, a bugger to author for and a bugger to use,” he says. “Imagine a next-generation Red Button toolset that allows people to author things for an IP age.”

From these show pages, Zeebox will also offer links to available on-demand episodes. They could be played on the tablet or smartphone, but Rose tells me users may eventually be able to use those handhelds to invoke playback on the TV. “I don’t have a full answer to these things yet, but we’ll experiment with the full infrastructure,” he says.

How it Works
At its most basic, an iPad user can “check-in” to shows manually (though Rose and Schmitt hate the GetGlue- and Foursquare-style gamification concept). To automate that process, the app can listen for shows’ audio fingerprints, Shazam-style. Connected TV owners get the full automatic experience because those TVs already know what shows are on.

“The browser in connected TVs lets you create HTML overlays and widgets,” Rose says. “We’ve created a lightweight, Javascript-based plugin that, on many 2011 and 2010 TVs, can be software-updated and user-installed.”

Zeebox is currently in demo on Samsung Smart TV and Rose, the former CTO of the YouView connected TV consortium, says: “YouView’s got a nice underlying architecture that will allow the Zeebox plugin to run on it, so we look forward to those discussions in the fullness of time.”

A Zeebox open API is also proposed to empower developers to build similar functionality in to their own apps. “There’s a shitload of technical work that needs to be done,” Rose says. “Getting there is non-trivial. We want to go to the moon.”

By By Robert Andrews, PaidContent

How Social is Your Favorite TV Show?

TV viewers are increasingly engaging in social media while watching their favorite shows. But which shows are drawing the most commenters and the largest share of the conversation? Cambridge, Mass.-based startup Bluefin Labs is using social media to not only measure engagement during TV shows, but also to find connections between shows that those viewers enjoy watching.

The company’s Response Level and Response Share metrics are essentially new ways to think about how a TV show stacks up against the competition. The new metrics are designed to measure how audiences talk about shows, both in terms of total number of commenters on a show, and the percentage of that show’s share of the conversation during the time in which it aired. And it’s rolling out its Bluefin Signals dashboard as a way for broadcasters and advertisers to see the connections between these shows and conversations.

Click to enlarge

The platform determines what people are commenting on without the more traditional use of scanning for hashtags or keywords during a certain show, using video fingerprinting technology to determine what’s happening onscreen instead. It then matches TV action with social media response. Deb Roy, co-founder and CEO of Bluefin, Labs refers to this matching up as “mapping the TV Genome.”

The platform measures social media responses to more than 3,000 TV shows and more than 100,000 individual airings of those shows. Each month, it processes about 3 billion social media comments, matching them up against 2 million minutes of live TV during that time. The company hopes to have full coverage of all live TV shows next year.

While knowing how many viewers are commenting about a particular show might be of some value to broadcasters and advertisers who use social media sentiment during a show as a proxy for engagement, the more valuable tool might be in Bluefin Labs’ ability to draw connections between shows. This data on “cross-show engagement” is useful in determining affinity between shows — for example, recognizing that fans of One Tree Hill are also likely to watch Hellcats and 90210.


Click to enlarge

That data can be used by broadcasters to help increase viewership — for example, running small ads on one show to promote another that fans might also like. Or it can be used by advertisers: If ad inventory on one show is in high demand, but its viewers have an affinity toward a show with a lower profile, agencies and brands can reach the same group of people with a less costly ad buy by betting on the second show.

Bluefin Labs isn’t the only company striving to measure social engagement with TV: Wiredset’s Trendrr recently introduced a social media TV chart, and SocialGuide aims to track social media mentions of shows in real-time.

By Ryan Lawler, GigaOM

WiO Connects TV to Mobile, Makes TV Ads Interactive

A new mobile app platform called WiO is set to revolutionize the TV watching experience by allowing customers to immediately get information about the products and services they see advertised on screen, both in TV commercials and within the shows themselves.

Through a mobile app running on customers’ phones, marketers can offer a variety of follow-up actions to the TV viewer, including coupons, reminders, contact info and more. In total, there are 10 follow-up actions offered. And the consumer is in complete control of which ones, if any, they respond to.

The company behind WiO, WiOffer, is the creation of Andrew Pakula and Matthew Greene, both of whom have experience working in and with major media, tech and digital marketing firms, including DoubleClick, Yahoo and Ogilvy. This experience, Pakula explains, has allowed them to learn a lot about what customers respond to and how.

How the WiO-Enabled TV Commercials Work
Before a customer can use the WiO app, the commercial or TV show has to first be WiO-enabled. To do this, the advertiser sends WiOffer their asset – that is, their commercial or the portion of the show where the product placement is visible and/or mentioned by the characters within the program. Using the clip as a digital ID, the mobile WiO app running on customers’ phones can then “hear” when the commercial plays and pop up a screen offering more information.

There are 10 different options a customer can choose from, some of which are subject to what the advertiser is providing. These include access to coupons, PDF brochures, app downloads, website addresses, retail locators, contact information, calendar events, reminders and even one option which will auto-dial the advertiser directly, in the case of TV commercials where a phone number is displayed. The unique thing about connecting TV to the mobile platform in this way is how many of these tasks are automated. For example, choosing the reminders, calendar event or contact info options will instantly save that information to a customer’s phone, with no manual effort required on the customer’s part.

In addition, when coupons are provided, the customer can save these “WiOffers” on their phone, where they will be accessible until the expiration date. To use a coupon, the customer just has to show their phone to the retailer.

Meanwhile, on the advertisers’ side, metrics surrounding customer response can be tracked in real-time, allowing them to adjust their advertising and offers on the fly to boost engagement, as need be.

Competition in the New TV Landscape
WiO is not the only company with this same idea. The music identification app Shazam recently raised funds to push into television. Some TV shows and ads now tell viewers to “Shazam” them in order to receive bonus content and discounts. Another company, IntoNow, uses audio recognition to encourage users to “check in” to what they’re watching on TV. GetGlue offers something similar. Even Microsoft is getting into the action with its NUads advertising platform, which uses the voice and gesture control in its Kinect for Xbox 360 to create interactive TV ads.

But unlike with Xbox, WiO is device agnostic, Greene says. He insists that WiO is different than the so-called “social apps,” too. Even though WiO allows for sharing to Facebook, Twitter, SMS and email, its goal is not to socialize the TV-watching experience. “Check-ins are a bit of distraction,” says Greene, “if not an enormous distraction.” And, referring to Shazam, he claims the idea of connecting a music app to actual transactions is a bit complicated. The Shazam ads point you to a mobile landing page, he notes. WiO aims to connect the brand directly to the customer so they can start talking immediately.

The WiO app will launch in a few weeks, first on iPhone. It will arrive on Android 60 days later. The company says it can’t comment on its advertising partners at this time, but from what we saw, there are some well-known brands in talks with the company now.

By Door Sarah Perez, 365Online

Spate of ACR Initiatives Brings New Efficiencies to Ads & Apps

Suddenly, it seems, anyone moving into the advanced advertising and apps spaces who doesn’t have an Automatic Content Recognition (ACR) strategy in play risks being left behind in the race to facilitate efficient use of connected devices to drive new revenues.

ACR applies to a range of techniques which can employ audio and/or video fingerprinting, digital watermarking or, in at least one case, “video signaturing” to trigger a real-time implementation of an ad or app in conjunction with what any given user is viewing or listening to.

“The beauty of this technology is we can use existing devices without huge infrastructure issues,” says Alex Terpstra, CEO of Civolution, which is one of many companies bringing ACR capabilities to market.

Civolution previewed its forthcoming ACR solution, which can be used with either the firm’s fingerprinting or watermarking technologies, at the recent NAB Show in Las Vegas. “We port our software to the devices like tablets and smartphones where we can help applications developers synchronize the applications automatically to the main [TV] screen,” Terpstra says.

“If you would have a TV show, even a live show, that has an appealing app on a tablet device, a lot of things could happen in that app that are related to the actual program on TV if you were able to accurately synchronize the two,” he says. “That’s what we do with our watermarking and fingerprinting technologies.”

Civolution has its eye on the connected TV space as well, Terpstra notes. “We have our technology run on the chipsets within these Web-connected devices to provide the ability to recognize content within the TV sets,” he says.

“Especially with using watermarks, we can determine the precise location of an advertisement that is being played out, and we can recognize which advertisement it is,” he explains. “If you know the exact location and which advertisement it is, you can also replace it with another advertisement that’s a local ad or targeted ad if the user’s profile is also known to the server. We’re working with advertising companies to close the whole loop.”

ACR has been in play for awhile in a wide range of applications, such as the Nielsen Audio Video Encoding (NAVE) system, the audio watermarking element of which for the past decade or so has been used as a way to identify and track program content for TV ratings purposes. The Nielsen meter equipped with NAVE capabilities registers what the viewer is watching automatically by virtue of identifying the source of the audio watermark in the program.

Another long-standing ACR user is Shazam, the provider of fingerprinting-based music discovery apps for mobile devices, which last year extended its business with introduction of the Shazam Audio Recognition Advertising (SARA) program. First used by clothing brand Dockers in the “Wear the Pants” campaign during the 2010 Super Bowl, SARA employs Shazam’s music recognition system to let brand advertisers tag a commercial for direct interaction with the Shazam user base, now numbering over 100 million people worldwide.

The tag induces viewers to engage with the promotion by pointing their mobile device in the direction of the commercial and hitting the Shazam button on the device. Shazam’s audio technology recognizes the specific advertisement and returns a customized result to the mobile device.

In a sign of how hot ACR has become, Yahoo! last month acquired a three-month-old startup in the field, IntoNow, for a reported $20-$30 million, winning a battle to buy the firm that included Twitter and Facebook, according to the online outlet TechCrunch. IntoNow, led by former Google and Viacom executive Adam Cahan, uses audio fingerprinting to allow users to identify and share television programs with their friends via iPhones.

Yahoo! believes the technology will help socialize the video programming offered through its platform. “Relying on social channels as a means for discovering content – whether it’s on a PC, mobile device, or TV – is rapidly on the rise,” says Bill Shaughnessy, Yahoo!’s senior vice president of product management. With a growing data base currently comprising 150 million minutes of content, IntoNow can fingerprint TV shows in real time with 99 percent accuracy, without requiring the participation of the networks.

But there’s a big advertising play as well, as revealed by IntoNow’s recent deal with Pepsi, which allows users to respond to commercials in a new campaign for Pepsi Max. The first 50,000 viewers who use IntoNow to tag a Major League Baseball-themed Pepsi MAX commercial will receive coupons for a free 20-ounce soda at retailers like Target and CVS. The company says automotive and movie advertisers are likely to be the next entities to sign on to the platform.

While the applications are similar, there’s actually a big difference between how watermarking and fingerprinting technologies get the job done. In the case of audio or video watermarking, the identification of the content is based on a digital code embedded invisibly in the audio or video stream that can be readily matched against a data base containing all such codes. Fingerprinting, on the other hand, relies on matching of marked slices of content with the same slices in a data base, which entails a search across the data base, typically in a two-step approach that first finds a set of likely matches and then drills down to get to a precise match.

Both technologies have their drawbacks and advantages. Watermarking is subject to precise placement of the identifying code where it won’t interfere with the content and with sufficient robustness to survive through all the processing steps between the content source and the end user. Fingerprinting avoids the robustness and precision placement issues but requires a processing-intensive search support infrastructure.

In general, for ACR purposes, audio fingerprinting or watermarking is preferred over video, insofar as the signals are easier to identify with companion devices using their microphones rather than their Web cams, and, in the case of fingerprinting search matches, there’s less information to process with an audio file . But audio can’t deliver a fingerprint when the track is silent, and there can be synchronization issues with use of language dubbing in foreign films. Thus, some suppliers are using video fingerprinting as a backup.

Ultimately, the business model may be the determining factor in choosing between watermarking and fingerprinting-based systems for advertising. Watermarking precisely identifies not only the content but the network it appears on, but it also requires participation of the content provider as well as the advertiser or app supplier. Fingerprinting doesn’t convey where the ad appears but can operate independently of any involvement by content providers.

Still another startup, TV Interactive Systems, has come up with another approach to ACR which it calls video signaturing. While it works somewhat like fingerprinting, VS doesn’t look for a unique file segment as required with fingerprinting. Instead, VS identifies “clues” or characteristics of a given piece of content which individually don’t make for an accurate match, but which taken together in batches of about ten per second ensure an accurate match with the data base without requiring the more costly processing used for identification in fingerprinting. Beyond the mobile device realm one of the key opportunities for ACR on the advertising front is the connected TV. TV Interactive Systems is focused exclusively on this end of the device market.

Among more recent developments in ACR is the new fingerprinting system offered by Vobile, which for several years has been supplying VideoDNA technology along with a data base of authorized video fingerprints, metadata and business rules from movie studios and programming networks to enable fully automated identification, tracking and management of content. Vobile’s integrated ACR solution will drive novel smart TV applications, such as broadcast monitoring and advanced advertising models, says CEO Yangbin Wang.

“Internet-enabled smart television brings TV entertainment to a new level,” Wang says. “With our latest ACR solution, content owners can track content distribution anywhere, anytime to any device. This opens up a world of new revenue opportunities for television programmers.”

Startup Zeitera, along with pursuing ACR TV apps for companion devices, is offering its fingerprinting cloud-based service to TV set manufacturers, providing them a means of working directly with advertisers to enable placement of targeted ads in linear programming without requiring any participation by the content suppliers.

In its latest move, Zeitera has introduced a software development kit and API development toolkit for programmers, CE manufacturers and app developers who want to implement the firm’s Vivid audio and video fingerprinting ACR solution in conjunction companion apps on tablets and smartphones. Zeitera offers its solution as a cloud-based service which is designed to eliminate a lot of the heavy lifting associated with individual users’ implementation of ACR.

“The Vvid Mobile API and SDK Development Platform for iOS and Android fills the needs of those broadcasters and content owners that would like to have smartphone and tablet applications that are able to synchronize with their TV programs, commercials and movies,” says Zeitera CEO Dan Eakins. “Zeitera is pioneering the market for white label ACR services and has several partners signed on developing applications with these APIs.”

Source: Screen Plays