University of Bath Unveils Vector-Based Video Codec
The development of a vector-based video codec by the University of Bath points unerringly at the death of the pixel within the next five years, but the year-old founding group behind this project -- Bath, Root6 Technology, Smoke & Mirrors and Ovation Data Services – know that they cannot achieve that killing target alone.
“We need to get many more companies involved to help us accelerate,” says Philip Willis, professor of Computing and director of the Centre for Digital Entertainment. “When this first phase is rolled out we will have two prongs – one is to get further funding for the additional research needed, but also right now to get the current stuff out there and to get people understanding the benefits of it.
“We need to get the core technology working. But there are application areas out there – for example on the web and on tablets and mobile phones - where we don’t have cover yet,” he adds. “Mainly we are working directly with post production and indeed technology companies like Root6, who support that kind of world. But we need to expand beyond that.
“The world is not standing still. There are different interests pulling in different ways, and unless we involve a good cross section of manufacturers and broadcasters we are not going to make any longer term impact,” he continues.
What precisely did the university merit patents for? “So far it is the prior art before this project started – so that we could draw a line under that and work out what we are doing beyond that,” says Willis. “That core patent covers the whole business of representing images, contours, and methods – for generating those contours and then turning them back into video. It is so that you have got the complete codec as a piece of software.”
The technology that turned the wheel and brought vectors back into fashion is Vectorised Photographic Images (VPI) developed by Professor Willis and his long time collaborator Dr John Patterson (formerly of Glasgow University, and now a visiting senior research fellow at the University of Bath). They first worked together (1992-5) on the ANIMAX project, and presented a joint paper on VPI at CVMP in 2009.
With vectorised images, the problem has always been how to fill in between the contours, but the Bath team has finally solved this with ‘double diffusion’. Another key factor is VSV (Vectorised Streaming Video), the video form of VPI. VSV is at its very beginning and the main problem currently is formidable processing times.
For offline format conversion work this is not much of an issue, but realtime is a way off. However, VSV is readily ‘parallelisable’ and with enough cores and enough threads it could eventually be produced in realtime.
VPI and VSV are called ‘resolution-independent’ formats because images are modeled in continuous and not discrete terms. Users can sample as often as they wish and at as high a colour depth as required, all from the same image format.
“The method that we’ve got is independent of bit depth. We can go to whatever the industry needs. There is nothing that is hard-wired into that,” Willis says.
Asked about a future standardisation effort, he comments: “We do that when we have a critical mass of companies on board, because in this world most of the standards are either de facto ones driven by the companies themselves, or they go through the standards process because there is an industry group behind them.”
Willis has a full-time programmer who is creating the preliminary codec, and the project team has been working with video content supplied by Smoke & Mirrors so test results can be shown at the CVMP (Conference for Visual Media Production) in December (Vue Cinema).
Asked to predict the death date for pixels, Willis says: “That’s tempting fate. First of all, we need to do further work on some of the high level operations that you can do with these images. What we have at the moment is a codec. The industry needs to manipulate images, and we need to develop the technology that we can move on to them for doing that manipulation. And that’s when they can become seriously interested across the whole spectrum of things that the technology can address.”
Root6 Technology is the bridge between the University of Bath academics and end users, its first tasks being to build a processing pipeline, and create a commercial application for the core vectorisation technology (codec). Was more sophistication required when compared to facilitating other codecs?
“Not necessarily. The idea is generally to try and wrap it such that it works in similar ways to other codecs, but we have been looking at improving the bit depth support of our existing pipeline up to 16 at least,” says Head of Software Development Nick Ridley.
“I am excited in terms of the fact that this is a completely different way of thinking about everything. There is a lot of work to do, but it is very much a future technology,” he adds.
“Our involvement in the project is pretty much ‘well there is no point having this thing unless you can actually demonstrate it,’ so we are looking forward to demonstrating the codec,” says Root6 MD Marcus Hume-Humphreys. “There are at least a couple of applications where this technology will really change the way people work. One of those, still an enormous nightmare, is in frame rate conversion.
“Having this technology available, as a mastering format from which to derive things without a process of standards conversion, is something that could potentially save Smoke & Mirrors and hundreds of other post houses an enormous amount of time,” he adds.
“With the prototype codec we think we are ready to produce some media for the golden eyes in the industry,” he continues. “We will have working demonstrations within the next 3-6 months. This is not a gradual migration. This will be kind of flick the switch. If we get it right it should see the way for the future for quite some time. But it is not something that is going to happen overnight.”
Smoke & Mirrors CTO Mark Wilding sees Bath’s codec as, “A wicked pie.” It is spatial image resolution that he wants to exploit. “How much money have post houses spent trying to do image recognition with pixel-based frames? Possibly millions.
“It is not because we are not developing the right software. It is because we’ve got the wrong underlying technology. If we had a camera that recorded vectors onto a hard drive it would then give us all the image recognition stuff. It would all come out in the wash.
“It needs turning on its head. The money they are putting into trying to find patterns in pixels should be spent on developing vectors. The pixel-based products we need in post production have stagnated. We cannot do anything more to the pixel. This could be the real shake up.
“Imagine Autodesk Flame that worked in vectors, and all the amazing stuff we could start doing. It is just revolutionary. So bring it on,” he continues. ”We want to see the death of the pixel.”
By George Jarrett, TVBEurope
