A real-time hardware codec library for RustDesk based on FFmpeg

Codec

Windows

GPU	FFmpeg ram	FFmpeg vram
intel encode	qsv	qsv
intel decode	d3d11	d3d11
nvidia encode	nvenc(nv12->d3d11)	nvenc(d3d11)
nvidia decode	d3d11	d3d11
amd encode	amf	amf
amd decode	d3d11	d3d11

Notes

• The reason for discarding the codecs using Cucontext is discussed in the following forum thread: https://forums.developer.nvidia.com/t/cuctxdestroy-causing-system-freeze-and-black-screen/290542/1. Based on the information above, there are several optimizations and changes made to the codec:

  • FFmpeg encoding AV_PIX_FMT_NV12 directly: The codec is modified to transfer AV_PIX_FMT_NV12 to AV_PIX_FMT_D3D11. This is done because FFmpeg doesn't use Cucontext if the device type is AV_HWDEVICE_TYPE_D3D11VA.
  • FFmpeg decoding with AV_HWDEVICE_TYPE_CUDA acceleration: This functionality is disabled and replaced with AV_HWDEVICE_TYPE_D3D11VA. The decoding process now utilizes D3D11VA acceleration instead of CUDA.

• VRAM encoding and decoding use FFmpeg only. SDK code is retained only for driver availability checks.

Linux

GPU	FFmpeg ram
intel encode	vaapi
intel decode	vaapi
nvidia encode	vaapi, nvnec
nvidia decode	vaapi, nvdec
amd encode	vaapi, amf
amd decode	vaapi

Issue

• vaapi: only tested on intel with va-driver-all, and hevc_vaapi encoding not supported on my pc
• remove hevc_vaapi because of possible poor quality
• amf: not tested, GPUOpen-LibrariesAndSDKs/AMF#378

MacOS

FFmpeg ram encode	FFmpeg ram decode
h265 only	Y

Android

FFmpeg ram encode
Y

System requirements

• intel

  Windows Intel(r) graphics driver since 27.20.100.8935 version.

  Hardware Platforms Supported by the Intel(R) Media SDK GPU Runtime

  https://www.intel.com/content/www/us/en/docs/onevpl/developer-reference-media-intel-hardware/1-1/overview.html

• AMD

  AMD Radeon Software Adrenalin Edition 23.1.2 (22.40.01.34) or newer

  https://github.com/GPUOpen-LibrariesAndSDKs/AMF

• nvidia

  Windows: Driver version 471.41 or higher

  https://docs.nvidia.com/video-technologies/video-codec-sdk/11.1/read-me/index.html

  https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new?ncid=em-prod-816193