IntelQuickSyncVideo

Differences between revisions 17 and 18
Revision 17 as of 2017-06-26 04:07:49
Size: 9745
Editor: vanvugt
Comment:
Revision 18 as of 2017-06-26 04:10:24
Size: 9730
Editor: vanvugt
Comment:
Deletions are marked like this. Additions are marked like this.
Line 73: Line 73:
 3. Play it with full hardware acceleration:
 {{{
 $ gst-play-1.0 OUTFILE.mp4
 }}}
 3. Play your new file ''OUTFILE.mp4'' using the above instructions.

Hardware-Accelerated Video Playback on Intel CPUs in Ubuntu

Note this document covers Intel CPU/GPUs only.

There's a lot of complicated stuff written on this topic. This document aims to get you working efficient video playback on Ubuntu, as simply as possible, as well as outlining future plans.

It's important to know the difference between working and not working. If it's working then you should be able to play modern 4K or full HD videos with around 20% CPU usage or less on a laptop and 10% or less on a desktop. If it's much more than that then something has gone wrong. For example, in testing these instructions a properly configured Haswell desktop uses only 3% CPU to play 4K 60FPS H.264 video.

Alternate approaches do exist, but those have been observed to increase rather than decrease CPU usage when compared to the default software decoders. So remember to measure your CPU. A working solution should play the video and decrease your CPU usage significantly.

Quick Start

Here's how to make things work quickly. What "works" well is limited to a couple of particular video players. If you need web browser video, or MKV files for example, then you should look at the Future Roadmap instead.

  1. Install Ubuntu.
  2. Make sure you're using X, and not Wayland. This means either Unity 7 or GNOME is OK. Just don't use GNOME on Wayland because it's not yet working.

  3. Install the Intel acceleration driver and tools:
     $ sudo apt install vainfo i965-va-driver
  4. Check that acceleration is installed and supported:
     $ vainfo
    You should see a list of video codec profiles that your system is capable of playing with full hardware acceleration.
  5. Reboot. It's not yet clear why, but you might need to reboot before the best performance is achievable.

Option 1: Accelerated playback using GStreamer libraries

  1. Install the missing packages:
     $ sudo apt install gstreamer1.0-vaapi gstreamer1.0-libav gstreamer1.0-plugins-ugly
  2. Play your video using gst-play-1.0 for the best performance:

     $ gst-play-1.0 myvideo.mp4

    You can also use totem to play videos accelerated, but it is more power hungry and less smooth than gst-play-1.0.

Option 2: Accelerated playback using FFmpeg libraries

  1. Install the missing packages:
     $ sudo apt install mpv
  2. Play your video with mpv hardware acceleration turned on (it's not turned on by default!):

     $ mpv --hwdec vaapi myvideo.mp4

    The reason why mpv is recommended over other FFmpeg-based players is because FFmpeg hardware acceleration support is not automatically integrated and has to be implemented per application. It is known to be implemented and working properly in mpv only.

Other video player apps won't work anywhere near as efficiently. VLC, Mplayer, Google Chrome and Firefox for example, will only use software decoding (much higher CPU) since they use FFmpeg directly rather than GStreamer. See the Future Roadmap below where we will try to fix those issues.

And TS files

Video files with the .ts extension are also playable with the same hardware acceleration after a few extra steps:

  1. Install the ffmpeg command-line tool:
     $ sudo apt install -f ffmpeg
  2. Make an MP4 file (this is fast and does not change the video or audio contents, just the file format):
     $ ffmpeg -i INFILE.ts -vcodec copy -acodec copy OUTFILE.mp4
  3. Play your new file OUTFILE.mp4 using the above instructions.

Know Your Limits

If video playback appears frozen or causes high CPU usage, then it's likely you're trying to play a certain profile of video your CPU is a bit too old to handle (or you made a mistake in following the instructions).

The simplest answer is to run 'vainfo' which will list all the video codecs that your CPU/GPU can handle in hardware.

The complicated answer is to look up your CPU model (from 'lscpu') on http://ark.intel.com, find out what generation it is and then using that information you can check your CPU generation against this list or this table.

Medium Term: A more polished quick start

The above Quick Start section has a lot of caveats. Most of them are mentioned as links to bugs that need fixing. But there is also the problem of poor MKV and TS container file support in GStreamer that needs fixing. After some or all of these issues are addressed we will be able to recommend Totem as the efficient video player of choice.

Long Term: Future Roadmap

In future we would like all video playback in Ubuntu to be accelerated by default (on modern Intel chips at least), out of the box. There are two good ways to get there. Both of them involve the FFmpeg because the FFmpeg libraries are the core decoding libraries that almost all video players and web browsers use by default.

For video player apps, we will need to modify the standard Ubuntu/Debian ffmpeg packages to add acceleration. For web browsers however, they will also each need to be modified as each browser has it's own private copy of FFmpeg libraries. So FFmpeg is the focus of all future work.

You should also know that the Intel Quick Sync Video is what we're aiming to use. Quick Sync Video is the Intel hardware feature that accelerates video handling efficiently. So if you see "QSV" referred to in software instructions then those instructions are misleading you, and actually talking about something else that indirectly uses your Intel Quick Sync Video hardware. Also note that historically the video decoding functions have been referred to by Intel as Intel Clear Video HD, and this is still true for Intel technical documents. But generally the internet community refers to it under the heading of Quick Sync Video (QSV).

Future Approach #1: VA-API ("LibVA") decode

Despite what some documentation says, FFmpeg can't play video with Intel hardware acceleration yet. This fact is hidden in the support matrix where you find a "N" at the intersection of "VA-API" and "Decoder": https://trac.ffmpeg.org/wiki/HWAccelIntro

So this means your Mplayer, VLC, Chrome, Chromium and Firefox can't yet play video using hardware acceleration out-of-the-box. It initially seemed all we need to do is implement the "decode" side, or find someone who can. However even that's not enough...

FFmpeg's hwaccel architecture v1 requires per-app implementation so doesn't scale very well. Even their own ffplay tool doesn't support it yet. However rather than implementing the v1 hwaccel architecture in each app, it is probably advisable to take (or wait for) the approach the FFmpeg themselves have in mind: Hardware Acceleration (hwaccel) API v2.

Future Approach #2: Media SDK ("MSDK")

(This is the approach that the FFmpeg documentation rather confusingly refers to as "Intel QSV")

Intel's Media SDK is the company's professional offering with the aim of providing the highest performance and the most video features all accelerated on Intel chips.

It sounds great. You can download the SDK for free (https://software.intel.com/en-us/media-sdk), and they have also published a first open source release of it too (https://github.com/Intel-Media-SDK/MediaSDK). But there are hidden reasons why we can't use any of this yet in Ubuntu...

The difficulty with Intel's Media SDK (both the commercial and open source offering) is that it's missing important pieces. Both of them have a dependency on a custom proprietary fork of LibAV (VA-API). While this "missing" code is actually provided in the commercial download, it requires that you:

  1. Overwrite official Ubuntu files, possibly breaking your system; and
  2. Recompile all your video players yourself; and
  3. Don't redistribute what you've made.

That won't work for us right now. And we don't recommend you try. It's very complicated and can easily break your Ubuntu installation.

So what about their open source version of Media SDK on Github? Well, that won't work either. It actually has a dependency on the commercial download in order to build at all, and with that has all the same restrictions as the commercial download.

Why is this a "Future Approach" then? Because in future we might be able to use the Media SDK after Intel removes the proprietary prerequisite problem: https://github.com/Intel-Media-SDK/MediaSDK/issues/10

What about NVIDIA and AMD?

They have different acceleration architectures with different instructions. So for the sake of simplicity, NVIDIA and AMD are not covered in this document. Still, long term future efforts for all CPU/GPU types should be focussed on FFmpeg for the same reasons outlined above.

http://bbb3d.renderfarming.net/download.html

IntelQuickSyncVideo (last edited 2018-08-28 04:00:15 by vanvugt)