After CreateVulkanVideoDecodeParser is called, nvidia-vkvideo-parser.dll begins its duties and opens up a console window during decoding.

Can this be behaviour be disabled?

I'd like to just run my test app without any superfluous windows lying about.

Thanks.

Hi, I'm trying to get yuva420p pixel format into my game engine (for RGBA videos), and so far the only way I can do it is via VP9 codec since HEVC + alpha encoders are only available on Mac.

I made some test files for yuva420p using VP9 codec (via FFMPEG), which play back fine in VLC but assert in this sample due to the missing VK_VIDEO_CODEC_OPERATION_DECODE_VP9_BIT_KHR symbol which isn't in vulkan_beta.h yet.

I presume this is coming at some point and VK_EXT_video_decode_vp9 define is currently disabled for that reason?

#ifdef VK_EXT_video_decode_vp9 { VK_VIDEO_CODEC_OPERATION_DECODE_VP9_BIT_KHR, "VP9" },

I was wondering if there is an alternative for alpha support (aside from using a separate grayscale video file), and if there is a list of codecs <-> pixel formats that are supported by Vulkan Video and the current beta drivers (and this sample). Thanks for the good work, keep it up.

1. Link broken [FFMPEG libraries for Windows] Download the following archive: https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2021-04-12-12-38/ffmpeg-N-101934-g7e241a1b73-win64-lgpl-shared.zip Then extract the archive to <APP_INSTALLED_LOC>\bin\libs\ffmpeg and add the path of the <APP_INSTALLED_LOC>\bin\libs\ffmpeg\win64\ of the application. Please make sure that <APP_INSTALLED_LOC>\bin\libs\ffmpeg\win64\bin location contains avformat-58.dll, avutil-56.dll and avcodec-58.dll shared libraries and <APP_INSTALLED_LOC>\bin\libs\ffmpeg\win64\lib contains the corresponding lib files.

2. build_windows_targets.bat doesn't work:

Determining VS version
Detected Visual Studio Version as 16 0
Generating 64-bit CMake files for Visual Studio 16 0
CMake Error: Could not create named generator Visual Studio 16 0 Win64

Generators
* Visual Studio 16 2019        = Generates Visual Studio 2019 project files.
                                 Use -A option to specify architecture.
  Visual Studio 15 2017 [arch] = Generates Visual Studio 2017 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 14 2015 [arch] = Generates Visual Studio 2015 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 12 2013 [arch] = Generates Visual Studio 2013 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 11 2012 [arch] = Generates Visual Studio 2012 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 10 2010 [arch] = Generates Visual Studio 2010 project files.
                                 Optional [arch] can be "Win64" or "IA64".
  Visual Studio 9 2008 [arch]  = Generates Visual Studio 2008 project files.
                                 Optional [arch] can be "Win64" or "IA64".
  Borland Makefiles            = Generates Borland makefiles.
  NMake Makefiles              = Generates NMake makefiles.
  NMake Makefiles JOM          = Generates JOM makefiles.
  MSYS Makefiles               = Generates MSYS makefiles.
  MinGW Makefiles              = Generates a make file for use with
                                 mingw32-make.
  Green Hills MULTI            = Generates Green Hills MULTI files
                                 (experimental, work-in-progress).
  Unix Makefiles               = Generates standard UNIX makefiles.
  Ninja                        = Generates build.ninja files.
  Ninja Multi-Config           = Generates build-<Config>.ninja files.
  Watcom WMake                 = Generates Watcom WMake makefiles.
  CodeBlocks - MinGW Makefiles = Generates CodeBlocks project files.
  CodeBlocks - NMake Makefiles = Generates CodeBlocks project files.
  CodeBlocks - NMake Makefiles JOM
                               = Generates CodeBlocks project files.
  CodeBlocks - Ninja           = Generates CodeBlocks project files.
  CodeBlocks - Unix Makefiles  = Generates CodeBlocks project files.
  CodeLite - MinGW Makefiles   = Generates CodeLite project files.
  CodeLite - NMake Makefiles   = Generates CodeLite project files.
  CodeLite - Ninja             = Generates CodeLite project files.
  CodeLite - Unix Makefiles    = Generates CodeLite project files.
  Eclipse CDT4 - NMake Makefiles
                               = Generates Eclipse CDT 4.0 project files.
  Eclipse CDT4 - MinGW Makefiles
                               = Generates Eclipse CDT 4.0 project files.
  Eclipse CDT4 - Ninja         = Generates Eclipse CDT 4.0 project files.
  Eclipse CDT4 - Unix Makefiles= Generates Eclipse CDT 4.0 project files.
  Kate - MinGW Makefiles       = Generates Kate project files.
  Kate - NMake Makefiles       = Generates Kate project files.
  Kate - Ninja                 = Generates Kate project files.
  Kate - Unix Makefiles        = Generates Kate project files.
  Sublime Text 2 - MinGW Makefiles
                               = Generates Sublime Text 2 project files.
  Sublime Text 2 - NMake Makefiles
                               = Generates Sublime Text 2 project files.
  Sublime Text 2 - Ninja       = Generates Sublime Text 2 project files.
  Sublime Text 2 - Unix Makefiles
                               = Generates Sublime Text 2 project files.

Building 64-bit Debug
Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1009: Project file does not exist.
Switch: ALL_BUILD.vcxproj

64-bit Debug build failed!

3. The instructions don't provide guidance on the environment variables to set:

cmake .. -DCMAKE_GENERATOR_PLATFORM=x64 -DCMAKE_INSTALL_PREFIX="$(pwd)/install/Debug" -DCMAKE_BUILD_TYPE=Debug
-- Building for: Visual Studio 16 2019
CMake Deprecation Warning at CMakeLists.txt:4 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.28.29910.0
-- The CXX compiler identification is MSVC 19.28.29910.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: c:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: c:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: C:/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.9_3.9.1264.0_x64__qbz5n2kfra8p0/python3.9.exe (found suitable version "3.9.4", minimum required is "3")
-- Selected Windows build type: Debug
-- Selected Windows build binary arch: wddm2_amd64_debug
-- VULKAN_VIDEO_APIS_INCLUDE path is not set. Setting the default path location to D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/include
-- LIBNVPARSER_BINARY_ROOT is at D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/bin/libs/nv_vkvideo_parser/wddm2_amd64_debug
-- VULKAN_VIDEO_PARSER_INCLUDE path is not set. Setting the default path location to D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/include/nv_vkvideo_parser
-- Trying to find Vulkan Video Parser library at D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/bin/libs/nv_vkvideo_parser/wddm2_amd64_debug
-- Vulkan Video Parser Lib: D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/bin/libs/nv_vkvideo_parser/wddm2_amd64_debug/nvidia-vkvideo-parser.lib
-- FFMPEG lib location D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/bin/libs/ffmpeg/win64/lib
-- FFMPEG include location D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/bin/libs/ffmpeg/win64/include
-- The location of the Vulkan Video driver source:  - encoder test app will not be built
-- AVCODEC_LIB-NOTFOUND
-- AVFORMAT_LIB-NOTFOUND
-- AVUTIL_LIB-NOTFOUND
-- TEGRA_TOP nor LINUX_DRIVER_BRANCH_TOP nor WINDOWS_DRIVER_BRANCH_TOP are set
Found GLSLANG libraries: D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/external/shaderc/build/install/lib/shaderc_shared.lib;glslang;OGLCompiler;OSDependent;HLSL;SPIRV;SPVRemapper;SPIRV-Tools-opt;SPIRV-Tools
-- Building demos with control flow guard
-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
AVCODEC_LIB
    linked by target "vk-video-dec-test" in directory D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/demos/vk-video-dec
AVFORMAT_LIB
    linked by target "vk-video-dec-test" in directory D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/demos/vk-video-dec
AVUTIL_LIB
    linked by target "vk-video-dec-test" in directory D:/Personal/~~~~PRIVATE~~~~/ProjVulkanVideoSamples/vk_video_decoder/demos/vk-video-dec

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

Hello, running the sample I get unresolved external symbols (the functions related to the video api) Driver Info: Lib dump: The functions doesn't seem to be present in the lib vulkandumplib.txt

EDIT : I used the DispatchLoaderDynamic to load the function addresses, but I get this error :

I'm using the latest Vulkan SDK version.

Is it related to the driver?

In order for this to serve as a useful example of how to use the vulkan video decoding APIs, I think it's most useful it it wouldn't include references to proprietary libraries. Since FFMPEG is already included as a dependency it might be more useful to use it to the parsing.

Hi all, I just spent a bunch of time trying to figure out why the colours were wrong in my engine for videos, where they were working before. I fixed all the vulkan validation errors and followed the specs for YCbCr conversion via immutable sampler being set during descriptor set layout / bindings instead of setting it during update, but still no luck.

It turns out, using HLSL and DXC needs some new parameters to make it work, but I can't figure out what it is. I tried a bunch of options in the latest HLSL 2021 build of DXC.exe, and it still gives me the same thing (building to vulkan 1.2 spirv, either to a header or to spv files and loaded from disc).

My old GLSL code using glslLandValidator works though, which is what this sample uses (or rather, builds the shaders live via shaderc).

Is there a way to get the ycbcr sampler to work properly using HLSL -> DXC -> SPIRV workflow?

The spec says: for CmdBeginVideoCodingKHR

pReferenceSlots is a pointer to an array of VkVideoReferenceSlotKHR structures specifying reference slots, used within the video command context between this vkCmdBeginVideoCodingKHR command and the vkCmdEndVideoCodingKHR commmand that
follows. Each reference slot provides a slot index and the VkVideoPictureResourceKHR
specifying the reference picture resource bound to this slot index. A slot index must not appear
more than once in pReferenceSlots in a given command.

For CmdVideoDecodeKHR:

pSetupReferenceSlot is a pointer to a VkVideoReferenceSlotKHR structure used for generating a
reconstructed reference slot and Picture Resource. pSetupReferenceSlot->slotIndex specifies the
slot index number to use as a target for producing the Reconstructed (DPB) data.
pSetupReferenceSlot must be one of the entries provided in VkVideoBeginCodingInfoKHR via
the pReferenceSlots within the vkCmdBeginVideoCodingKHR command that established the
Vulkan Video Encode Context for this command.

This to me reads that the Begin coding should contain all the slot indexes for used references and generated references.

It looks like the player reads the VkVideoCapabilitiesKHR struct, but doesn't seem to use them anywhere.

I think it should probably be making sure alignments for bs and video picture granularity are properly support.

Right now when you don't specify any arguments into the program it just lits a few vulkan extensions and terminates. It would be helpful to list which commandline arguments are required to correctly invoke the program.

Hello. I have HDR videos recorded via Shadowplay to display on my HDR monitor. In Shell.cpp when I query the available swapchain formats / colorspaces with

    uint32_t chosenFormat;

    for (chosenFormat = 0; chosenFormat < formatCount; chosenFormat++) {
        std::cout << "format = " << formats[chosenFormat].format << std::endl;
        std::cout << "colorspace = " << formats[chosenFormat].colorSpace << std::endl;
    }

and choose an appropriate combination that is available, say

    swapchain_info.imageFormat = VK_FORMAT_R16G16B16A16_SFLOAT;
    swapchain_info.imageColorSpace = VK_COLOR_SPACE_EXTENDED_SRGB_LINEAR_EXT ;

After setting the same format in VulkanVideoUtils.cpp for the ImageObject / Framebuffer / and RenderPass I can only get a Low Dynamic Range results with incorrect colors ...LDR is not what I am going for.

    swapchain_info.imageFormat = VK_FORMAT_B8G8R8A8_UNORM;
    swapchain_info.imageColorSpace = VK_COLOR_SPACE_SRGB_NONLINEAR_KHR;

Can this Decoder sample support HDR video in mp4 h265 recorded from shadowplay? What I see when playing directly from a player like VLC or MPV is a 1:1 of the HDR gameplay footage I just recorded, but with this Decoder I am way off and LDR. I am using the BETA driver 473.50 on windows 11

Build failure with "Error: Unsupported option --v-lvl-root" on macos because "vk_video_decoder/external/MoltenVK/fetchDependencies" doesn't have "--v-lvl-root" option.

NVIDIA parser reports in StdVideoH264LevelIdc the extracted value from the bytestream, not the mapped data according the specification.

See https://github.com/Igalia/GstVkVideoParser/pull/33#issuecomment-1348420461

According to https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-VkVideoSessionCreateInfoKHR-maxActiveReferencePictures-04849

the maxActiveReferencePictures should not exceed 16

I'm planning to add GStreamer capabilities to replace the FFMpeg parser in https://github.com/nvpro-samples/vk_video_samples/blob/de54a1df626828c47e265c65d6676a3f083809a1/vk_video_decoder/demos/vk-video-dec/AppDecVulkanFrame/VulkanVideoProcessor.cpp#L49

Do you prefer an option and keep FFMpeg or just like drop FFMpeg for real ?

Use the external project GstVkVideoParser to parse and create vulkan structures using GStreamer

This parser is able to provide H264 and H265 content and can be used as an alternative to libnvidia-vkvideo-parser.so

When I compile and try to run the test. the GPU is not detected (RTX 3070, cuda 11.4, vulkan version: 1.2.131.2-1)

./vk-video-dec-test -i /home/burak/Downloads/Remote_Cat_2021_ETH_Version.mp4 Enumerating instance layers: VK_LAYER_KHRONOS_validation VK_LAYER_LUNARG_standard_validation Looking for instance layers: Enumerating instance extensions: VK_KHR_device_group_creation VK_KHR_display VK_KHR_external_fence_capabilities VK_KHR_external_memory_capabilities VK_KHR_external_semaphore_capabilities VK_KHR_get_display_properties2 VK_KHR_get_physical_device_properties2 VK_KHR_get_surface_capabilities2 VK_KHR_surface VK_KHR_surface_protected_capabilities VK_KHR_xcb_surface VK_KHR_xlib_surface VK_EXT_acquire_xlib_display VK_EXT_debug_report VK_EXT_debug_utils VK_EXT_direct_mode_display VK_EXT_display_surface_counter Looking for instance extensions: VK_KHR_surface VK_KHR_xcb_surface terminate called after throwing an instance of 'std::runtime_error' what(): failed to find any capable Vulkan physical device Aborted (core dumped)