Lately, I've been playing with OPC-UA subscriptions to mirror a real robot's live state in its 3dverse Digital Twin. In this devlog, I'll walk through how I built a Node.js app that runs both an OPC-UA client and a Livelink headless client. The OPC-UA client receives updates of the robot state and forwards them via Livelink to synchronize its 3dverse Digital Twin.
H.265 (HEVC) is a video compression standard that delivers roughly the same quality as H.264 at half the file size. What are the basic steps taken by the H.265 codec to compress a sequence of frames? What are some of the reasons that H.265 provides 50% better compression than H.264?
A few months ago I was given the task to integrate a Vulkan video encoder into our renderer, which would compress frames in H.264/H.265. I went in knowing nearly nothing about video compression and came out with some interesting knowledge I will cover in this new series. This article goes over color-related topics such as chroma subsampling, color gamuts, color spaces and some color science math.
The following is a summary of debugging strategies I developped during my struggles with encoding H.264/H.265 using Vulkan Video. It is intended to be a brief and helpful guide for others suffering under Vulkan Video, more specifically with the setting of video session parameters.
A few months ago I was given the task to integrate an open source Vulkan video encoder into our renderer. I had never touched Vulkan before (I vaguely knew to feel fear towards it) and knew nothing about video compression, namely the H.264 and H.265/HEVC codecs.
What does it take to simulate a vast, living virtual world in real-time without tanking performance?
In this post, we dig into the lowest and most fundamental levels of our engine architecture, where we represent hundreds of thousands of entities using a blend of data-oriented design, ECS principles, and one of the most underrated tools in the programmer's toolbox: virtual memory.
Vulkan delivers exceptional performance at the cost of verbose code and meticulous resource management, making even basic rendering operations unnecessarily complex. All this verbosity kills rapid experimentation and makes simple tasks tedious.
At 3dverse, we built a visual render graph system on top of Vulkan that replaces hundreds of lines of boilerplate with connected nodes that automatically handle the low-level details.
In this post, I'll demonstrate our approach with a simple screen-clearing example, reveal the production PBR render graph powering our cloud rendering platform, and finally if you are still around, preview our new ray tracing capabilities.
Streaming real-time XR to the web sounds great, until latency ruins the experience.
In this post, we explain how we made server-side rendering not just usable, but comfortable in XR, using a few clever tricks to smooth out the delay and keep everything feeling grounded, even when the network isn’t.
In the end of 2023 Vulkan released Vulkan Video extensions which allowed for GPU-accelerated encoding of H.264 and H.265 video. This was bleeding edge stuff: these extensions provided a GPU-agnostic solution for video compression which did not exist previously. At 3dverse, our renderer was dependent on NVIDIA solely because of video encoding, so this new solution would finally make us GPU-agnostic!
However. To this day, there is very limited information online to help someone implement anything with these extensions. The following is a semi-rageful but triumphant account of working with Vulkan Video from the vantage point of an engine programmer who had no previous experience with Vulkan, or any video compression codecs.
