IntelQuickSyncVideo

Revision 4 as of 2017-06-16 10:10:48

Clear message

Hardware-Accelerated Video Playback on Intel CPUs in Ubuntu

Note this document covers Intel CPU/GPUs only.

Have Some Standards

There's a lot of complicated stuff written on this topic. And it is often overcomplicated, incorrect and unreliable. This document aims to get you working efficient video playback on Ubuntu, as simply as possible.

It's important to know the difference between working and not working. If it's working then you should be able to play modern 4K or full HD videos with around 20% CPU usage or less on a laptop and 10% or less on a desktop. If it's much more than that then something has gone wrong. For example, my Haswell desktop uses only 3% CPU to play 4K 60FPS H.264 video.

Quick Start

Here's how to make things work quickly. What "works" well is limited to MP4 container files and one particular video player. If you need web browser video, or MKV files for example, then you should look at the Future Roadmap instead.

  1. Install Ubuntu.
  2. Make sure you're using X, and not Wayland. This means either Unity 7 or GNOME is OK. Just don't use GNOME on Wayland because it's not yet working.

  3. Install the missing packages:
     $ sudo apt install vainfo i965-va-driver gstreamer1.0-vaapi gstreamer1.0-libav gstreamer1.0-plugins-ugly
    (those last two are actually only required to decode the audio in your videos)
  4. Check that acceleration is installed and supported:
     $ vainfo
    You should see a list of video codec profiles that your system is capable of playing with full hardware acceleration.
  5. Reboot. It's not yet clear why, but you might need to reboot before the best performance is achievable.
  6. Play your video using gst-play-1.0 for the best performance:

     $ gst-play-1.0 myvideo.mp4

    You can also use totem to play videos accelerated, but it is more power hungry and less smooth than gst-play-1.0. Still, totem will use a lot less CPU than VLC or Mplayer...

Other video player apps won't work anywhere near as efficiently. VLC, Mplayer, Google Chrome and Firefox for example, will only use software decoding (much higher CPU) since they use FFmpeg directly rather than GStreamer. See the Future Roadmap below where we will try to fix those issues.

Know Your Limits

If video playback appears frozen or causes high CPU usage, then it's likely you're trying to play a certain profile of video your CPU is a bit too old to handle (or you made a mistake in following the instructions).

The simplest answer is to run 'vainfo' which will list all the video codecs that your CPU/GPU can handle in hardware.

The complicated answer is to look up your CPU model (from 'lscpu') on http://ark.intel.com, find out what generation it is and then using that information you can check your CPU generation against this list: https://github.com/01org/intel-vaapi-driver/blob/master/README

Medium Term: A more polished quick start

The above Quick Start section has a lot of caveats. Most of them are mentioned as links to bugs that need fixing. But there is also the problem of poor MKV and TS container file support in GStreamer that needs fixing. After some or all of these issues are addressed we will be able to recommend Totem as the efficient video player of choice.

Long Term: Future Roadmap

In future we would like all video playback in Ubuntu to be accelerated by default (on modern Intel chips at least), out of the box. There are two good ways to get there. Both of them involve the FFmpeg because the FFmpeg libraries are the core decoding libraries that almost all video players and web browsers use by default.

For video player apps, we will need to modify the standard Ubuntu/Debian ffmpeg packages to add acceleration. For web browsers however, they will also each need to be modified as each browser has it's own private copy of FFmpeg libraries. So FFmpeg is the focus of all future work.

You should also know that the Intel Quick Sync Video is what we're aiming to use. Quick Sync Video is the Intel hardware feature that accelerates video handling efficiently. So if you see "QSV" referred to in software instructions then those instructions are misleading you, and actually talking about something else that indirectly uses your Intel Quick Sync Video hardware.

Future Approach #1: VA-API ("LibVA") decode

Despite what some documentation says, FFmpeg can't play video with Intel hardware acceleration yet. This fact is hidden in the support matrix where you find a "N" at the intersection of "VA-API" and "Decoder": https://trac.ffmpeg.org/wiki/HWAccelIntro

So this means your Mplayer, VLC, Chrome, Chromium and Firefox can't yet play video using hardware acceleration out-of-the-box. All we need to do is implement the "decode" side, or find someone who can. But it's a big development task so let's think about that another day.

Future Approach #2: Media SDK ("MSDK")

(This is the approach that the FFmpeg documentation rather confusingly refers to as "Intel QSV")

Intel's Media SDK is the company's professional offering with the aim of providing the highest performance and the most video features all accelerated on Intel chips.

It sounds great. You can download the SDK for free (https://software.intel.com/en-us/media-sdk), and they have also published a first open source release of it too (https://github.com/Intel-Media-SDK/MediaSDK). But there are hidden reasons why we can't use any of this yet in Ubuntu...

The difficulty with Intel's Media SDK (both the commercial and open source offering) is that it's missing important pieces. Both of them have a dependency on a custom proprietary fork of LibAV (VA-API). While this "missing" code is actually provided in the commercial download, it requires that you:

  1. Overwrite official Ubuntu files, possibly breaking your system; and
  2. Recompile all your video players yourself; and
  3. Don't redistribute what you've made.

That won't work for us right now. And we don't recommend you try. It's very complicated and can easily break your Ubuntu installation.

So what about their open source version of Media SDK on Github? Well, that won't work either. It actually has a dependency on the commercial download in order to build at all, and with that has all the same restrictions as the commercial download.

Why is this a "Future Approach" then? Because in future we might be able to use the Media SDK after Intel removes the proprietary prerequisite problem: https://github.com/Intel-Media-SDK/MediaSDK/issues/10

What about NVIDIA and AMD?

They have different acceleration architectures with different instructions. So for the sake of simplicity, NVIDIA and AMD are not covered in this document. Still, long term future efforts for all CPU/GPU types should be focussed on FFmpeg for the same reasons outlined above.

http://bbb3d.renderfarming.net/download.html