The following is a summary of debugging strategies I developped during my struggles with encoding H.264/H.265 using Vulkan Video. It is intended to be a brief and helpful guide for others suffering under Vulkan Video, more specifically with the setting of video session parameters. This is part 3 to the adventure I started writing about here.
A few months ago I was given the task to integrate an open source Vulkan video encoder into our renderer. I had never touched Vulkan before (I vaguely knew to feel fear towards it) and knew nothing about video compression, namely the H.264 and H.265/HEVC codecs. This is part 2 to the adventure I started writing about here.
Vulkan delivers exceptional performance at the cost of verbose code and meticulous resource management, making even basic rendering operations unnecessarily complex. All this verbosity kills rapid experimentation and makes simple tasks tedious.
At 3dverse, we built a visual render graph system on top of Vulkan that replaces hundreds of lines of boilerplate with connected nodes that automatically handle the low-level details.
In this post, I'll demonstrate our approach with a simple screen-clearing example, reveal the production PBR render graph powering our cloud rendering platform, and finally if you are still around, preview our new ray tracing capabilities.
In the end of 2023 Vulkan released Vulkan Video extensions which allowed for GPU-accelerated encoding of H.264 and H.265 video. This was bleeding edge stuff: these extensions provided a GPU-agnostic solution for video compression which did not exist previously. At 3dverse, our renderer was dependent on NVIDIA solely because of video encoding, so this new solution would finally make us GPU-agnostic!
However. To this day, there is very limited information online to help someone implement anything with these extensions. The following is a semi-rageful but triumphant account of working with Vulkan Video from the vantage point of an engine programmer who had no previous experience with Vulkan, or any video compression codecs.
