The PERSEUS Plus Story
Creating ‘The next generation Video CODEC that can run on existing devices’
By Rick Clucas (Head of Innovation, V-Nova)
The Industry Challenge
Since the invention of television transmission in the 1930s, the assumption has been that in order to deliver something better or in a more efficient way new decoding hardware was always required.
This assumption has been holding back revolution in the industry ever since. Today this has never been more true, each new standard has required much more complex devices for both encoding and decoding, this has meant that the only time a new more efficient CODEC has been widely adopted has been when there has been another factor that required new decoding devices anyway, e.g. the move from SD to HD and HD to UHD.
Today a large portion of the digital video that is viewed around the world is still in MPEG2, despite the fact that h.264 services have been around for over 13 years and HEVC for 4! The economic case just doesn’t exist for replacing existing decoding devices, even if they are a minority of the total population of receiving devices.
But what if we didn’t have to replace said legacy devices?
With each new generation of CODEC formats, the massive increase in processing power required to encode has meant that it is often difficult to get much of an improvement in bitrate in live situations. This might be acceptable for some offline encodes, but with a significant increase in costs.
In 2012 we were given a challenge by a leading satellite broadcaster: ‘Could you half the bitrate of our existing HD channels on our existing deployed Set Top Boxes?’, ‘If so, that would transform the economics of our business!’
We at V-Nova believe that real-time performance – e.g. live encoding, which is important also to VOD services such as catch up TV, network PVR, etc. – is what is important to the industry, so we added some additional constraints.
- Must be encodable with software updates to existing live and VOD encoders.
- Must not increase the encoder processing load, and should ideally reduce it.
- Must not increase the decoder power consumption, and should ideally reduce it.
The Solution
The current wisdom is that each new CODEC format needs new hardware acceleration to decode it, but what if we looked at it from the other way around, e.g. “what could we get the existing hardware to decode?”.
Our starting point was our PERSEUS Pro CODEC format, which is a hierarchal fully parallel CODEC that offers industry leading performance for lossless and visually lossless Intra Only production and contribution use cases and runs entirely in software on CPUs or GPUs. But how could we translate these benefits to the constraints of what is now a 14 year-old Set Top Box chipset and compete with the power consumption of a ‘hardware’ h.264 decoder?
I had spent most of my career looking at problems from the other way. At Argonaut we had come up with the idea of creating what would now be called the world’s first GPU (the Super FX chip of the Nintendo Super NES) in order to deliver 3D games on a games console that only had a character mapped screen. I had then gone on as founder and CTO of ARC cores to create a processing architecture for solving just this kind of problem when building chips. I also knew that a lot of the MPEG2 chips had been build using the ARC as the ‘master controller’ of ‘hardware’ MPEG2 decoders, and had worked with other companies building h.264 decoders that were all effectively a collection of hardware accelerators connected to an efficient processor.
Had I already invented the answer to the problem two decades earlier?
It turns out that all the ‘hardware’ video decoders that we have looked at are a collection of hardware blocks controlled by some form of ‘master controller’, or to put it another way software with hardware assist.
So if we could create a CODEC format that could use some of the hardware blocks with software, we would be able to run on existing chipsets and massively improve the bitrate while not increasing the power consumption of the device. Many people seem to talk about hardware decoding being some magic function that consumes no power: this is not of course the case, as power is consumed by transistors switching, the number of them and the frequency at which they switch is what consumes the power, by creating the most efficient mix of hardware and software you get the best power efficiency.
For various IP reasons I can’t tell you which hardware blocks we used to create the first version of PERSEUS Plus, but the principle is that we use a PERSEUS format ‘plus’ (i.e., in combination with) a CODEC format, which for the first deployments has been h.264 but could also be MPEG2, HEVC, VP9 or even AV1 in the future. During the encode process we generate extra data that is then combined using a mixture of hardware (where available) and software on the decoder to reconstruct the full image, but at a much lower bitrate than just using the CODEC format after the ‘plus’ (a.k.a., ‘base CODEC format’).
Although PERSEUS Plus was original created for 14 years old chips using a mixture of hardware and software available in the device, we can run purely in software in current generation mobile phone chips and not consume any more power than the ‘hardware’ versions in the devices. We can of course also still use hardware blocks to further reduce power consumption.
On the encoder side we created an ‘Encoder Plug In’ that can be easily integrated by encoder vendors into their existing software encoder platforms and that will generate streams using the ‘base’ CODEC with Perseus added as extra metadata or as a separate TS PID depending on the application. Using the ‘base’ CODEC at a lower resolution translates into strong reductions of required encoding processing power, maintained even after accounting for the (inherently fast and parallel) PERSEUS part. The output stream is encapsulated in compliance with the ‘base’ CODEC standard, and the ‘base’ CODEC encode is decodable by any decoder that can decode that CODEC: if the media player is not aware of PERSEUS, the decoded video will be at a lower resolution and quality, but it will still play. This approach means at the same time that Perseus has no negative impact (reducing bitrate is positive!) on the rest of the delivery infrastructure, e.g. DRM, CDNs, Origin Servers, packaging, metadata management, ad insertion, etc.
This ‘Plus’ approach also means that operators can do risk-free phased rollouts where customers who have PERSEUS enabled decoders see the higher resolution better quality video, while the decoders unaware of PERSEUS still see video as before. This is especially useful in the common scenario where customers are using PERSEUS to upgrade customers from SD to HD while not increasing (often reducing) the total bitrate: PERSEUS enables them to charge a premium and deliver a better Quality of Experience while reducing their delivery costs.
The Future
We recently announced PERSEUS 2, the next generation of PERSEUS Plus and PERSEUS Pro. For what concerns PERSEUS Plus and distribution use cases, PERSEUS 2 further improves encoding/decoding processing power efficiency and at the same time provides another step-change visual quality improvement in the key customer operating points:
- 100 kbps, the minimum necessary to deliver mobile video to all consumers
- 300 kbps, for reliable, enjoyable HD mobile video experiences
- 1 Mbps, for monetisable full HD mobile video
- 2 Mbps, for HD IPTV programming for all xDSL and fiber users
- 6 Mbps, for high quality UHD movie streaming and 4K 360/VR immersive experiences at scale
- 10 Mbps, for scalable DTH/Cable/Fibre UHD sports services.
Whilst improving even further the quality of the above operating points, our fantastic engineering team are also constantly working on reducing the power consumption of decoder solutions even further and expanding the availability of PERSEUS enabled decoding, for example by also using JavaScript decoding for HTML5 use cases.
One of the great things about the PERSEUS CODEC format is that its relative efficiency increases even more as the resolution goes up. We can already deliver stunning results with UHD for live sports at 10 mbps, using standard, low cost and compact encoding hardware: this will enable UHD to become a commercial realty.
Summary
PERSEUS Plus h.264 already delivers on the quality/bitrate promises of next generation CODEC formats but does it on today’s hardware using less encoder and decoder power than h.264.
As HEVC, VP9 or AV1 get traction, PERSEUS Plus can be used in combination with even more efficient ‘base’ codecs, providing even more compression advantages. At the same time, also the efficiency of PERSEUS Plus encoders will keep improving, as we add several additional optimizations in the coming months.
In short, PERSEUS Plus can work on top of any CODEC format and can always stay ahead of current and future CODECS.
To learn more about PERSEUS Plus, email us on info@v-nova.com
