Source/Interface/XR

Virtual Reality

OpenXR

The VR support in Blender is based on the OpenXR standard. While being rather new, it has wide industry support and good chances to become the de facto standard for XR applications.

Layered interfacing with OpenXR runtimes

The OpenXR specification splits the workload between the application (in our case Blender) and an OpenXR runtime. This runtime implements or manages the device drivers and higher level functionality. At the time of writing, Monado, Oculus, SteamVR and Windows Mixed Reality include OpenXR compatible runtimes.

On the Blender side, we have to connect to these runtimes, i.e. we have to dynamically link to them. While we could write that linking code ourselves, the OpenXR-SDK contains an OpenXR loader, which deals with that and does provide additional sanity checks and debug helpers. It uses the runtime that is currently set as active on the user's system. That way users can have multiple runtimes installed (e.g. for multiple HMDs from different vendors), and decide which one to use in an OS wide manner.

Ghost-XR

Design Overview

• For code using this API, the most important object is a GHOST_XrContext handle. Through it, all API functions and internal state can be accessed/modified.
• Main responsibilities of the Ghost XR-context are to manage lifetimes of the OpenXR runtime connection (represented by XrInstance), the session and to delegate operations/data to the session.
• The OpenXR related graphics code, which is OS dependent, is managed through a GHOST_IXrGraphicsBinding interface, that can be implemented for the different graphics libraries supported (currently OpenGL and DirectX).
• Much of this code here has to follow the OpenXR specification and is based on the OpenXR hello_xr implentation.
• In future we may want to take some code out of the context, e.g. extension and API layer management.
• There are runtime debugging and benchmarking options (exposed through --debug-xr and --debug-xr-time).
• Error handling is described in a section below.

Why have this in Ghost?

Decision was to do the OpenXR level access through GHOST. Main reasons:

• OpenXR requires access to low level, OS dependent graphics lib data (e.g. see XrGraphicsBindingOpenGLXlibKHR)
• Some C++ features appeared handy (std::vector, RAII + exception handling, cleaner code through object methods, etc.)
• General low level nature of the OpenXR API

Comment by Julian Eisel (initial VR support developer):
After all I think much of the functionality doesn't really fit into GHOST however. I would like to address this by having a separate module (naming proposal: VAMR for virtual + augmented + mixed reality), placed in intern/. The main issue is getting this to work well with Ghost data, especially how to get the mentioned low level data out of Ghost.

Error Handling Strategy

The error handling strategy chosen uses C++ exceptions, a controversial feature. The following explains why this was seen as the best way nevertheless.

The strategy requirements were:

• If an error occurs, cleanly exit the VR session (or destroy the entire context), causing no resource leaks or side effects to the rest of Blender.
• Show a useful error message to the user.
• Don't impair readability of code too much with error handling.

Here's why I chose an exception based strategy:

• Most alternatives require early exiting functions. This early exiting has to be 'bubbled up' the call stack to the point that performs error handling. For safe code, early exit checks have to be performed everywhere and code gets really impaired by error checking. Tried this first and wasn't happy at all. Even if error handling is wrapped into macros.
• All GHOST_Xr resources are managed via RAII. So stack unwinding will cause them to be released cleanly whenever an exception is thrown.
• GHOST_Xr has a clear boundary (the Ghost C-API) with only a handful of public functions. That is the only place we need to have try-catch blocks at. (Generally, try-catch blocks at kinda random places are a bad code smell IMHO. Module boundaries are a valid place to put them.)
• Exceptions allow us to pass multiple bits of error information through mulitple layers of the call stack. This information can also be made specific with a useful error message. As of now, they conain a user error message, the OpenXR error code (if any), as well as the exact source code location the error was caught at.

So the strategy I went with works as follows:

• If a VR related error occurs within GHOST_Xr, throw an exception (GHOST_XrException currently).
• OpenXR calls are wrapped into a macro throwing an exception if the return value indicates an error.
• Useful debugging information and user messages are stored in the exceptions.
• All resources must be managed through RAII, so throwing an exception will release 'dangling' ones cleanly.
• In the GHOST C-API wrappers, the exceptions are caught and contained error information is forwarded to a custom error handling callback.
• The error handling callback is set in wm_xr.c, prior to creating the XR-Context, and implements clean destruction of the context.

Further Information

The main VR project started as a Google Summer of Code 2019 project. A rather extensive overview is available in its final report.