GLSL Cross Compilation

The GPU module supports multiple GL backends. The biggest challenge of multiple GL backends is that GLSL is different on all those platforms. It isn't possible to compile an OpenGL GLSL on Vulkan as the GLSL differs. The big differences between the GLSL's are how they handle resources.

This section of the documentation will handle how to create GLSL that can safely be cross compiled to different backends.

GPUShaderCreateInfo

Defining a new compile unit

When creating a new compile unit to contain a GPUShaderCreateInfo definition it needs to be added to the SHADER_CREATE_INFOS in gpu/CMakeLists.txt this will automatically register the definition in a registry.

Each of the compile unit should include gpu_shader_create_info.h.

Interface Info

Interfaces are data that are passed between shader stages (Vertex => Fragment stage). Attributes can be flat, smooth or no_perspective describing the interpolation mode between.

Example Interface Info

GPU_SHADER_INTERFACE_INFO(text_iface, "")
    .flat(Type::VEC4, "color_flat")
    .no_perspective(Type::VEC2, "texCoord_interp")
    .flat(Type::INT, "glyph_offset")
    .flat(Type::IVEC2, "glyph_dim")
    .flat(Type::INT, "interp_size")

Shader Info

Shader Info describes

Where to find required data (vertex_in, push constant).
Textures/samplers to use (sampler)
Where to store the final data (fragment_out)
It describes the data format between the shader stages (vertex_out).
The source code of each stage (vertex_source, fragment_source)

Shader infos can reuse other infos to reduce code duplication (additional_info would load the data from the given shader info into the new shader info.

The active GPU backend will adapt the GLSL source to generate those part of the code.

Example Shader Info

GPU_SHADER_CREATE_INFO(gpu_shader_text)
    // vertex_in define vertex buffer inputs. They will be passed to the vertex stage.
    .vertex_in(0, Type::VEC4, "pos")
    .vertex_in(1, Type::VEC4, "col")
    .vertex_in(2, Type::IVEC2, "glyph_size")
    .vertex_in(3, Type::INT, "offset")

    // vertex_out define the structure of the output of the vertex stage.
    // This would be the input of the fragment stage or geometry stage.
    .vertex_out(text_iface)

    // definition of the output of the fragment stage.
    .fragment_out(0, Type::VEC4, "fragColor")

    // Flat uniforms aren't supported anymore and should be added as push constants.
    // Note that push constants are limited to 128 bytes. Use uniform buffers
    // when more space is required.
    // Internal Matrices are automatically bound to push constants when they exists.
    // Matrices inside a uniform buffer is the responsibility of the developer. 
    .push_constant(0, Type::MAT4, "ModelViewProjectionMatrix")

    // Define a sampler location.
    .sampler(0, ImageType::FLOAT_2D, "glyph", Frequency::PASS)

    // Specify the vertex and fragment shader source.
    // dependencies can be automatically included by using `#pragma BLENDER_REQUIRE`
    .vertex_source("gpu_shader_text_vert.glsl")
    .fragment_source("gpu_shader_text_frag.glsl")

    // Add all info of the GPUShaderCreateInfo with the given name.
    // Provides limited form of inheritance.
    .additional_info("gpu_srgb_to_framebuffer_space")

    // Create info is marked to be should be compilable.
    // By default a create info is not compilable.
    // Compilable shaders are compiled when using shader builder.
    .do_static_compilation(true);

Shader Source Order

Shader source order does not follow the order of the methods call made to the create info. Instead it follows this fixed order:

Standard Defines: GPU module defines (GPU_SHADER, GPU_VERTEX_SHADER, OS, GPU vendor, and extensions) and MSL glue.
Create Info Defines: .define.
Typedef Sources: .typedef_source.
Resources Declarations: .sampler, .image, .uniform_buf and .storage_buf.
Layout Declarations: .geometry_layout, .local_group_size.
Interface Declarations: .vertex_in, .vertex_out, .fragment_out, .fragment_out.
Main Dependencies: All files inside #pragma BLENDER_REQUIRE directives.
Main: Shader stage source file .vertex_source, .fragment_source, .geometry_source or .compute_source.
NodeTree Dependencies: All files needed by the nodetree functions. Only for shaders from Blender Materials.
NodeTree: Definition of the nodetree functions (ex: nodetree_surface()). Only for shaders from Blender Materials.

Buffer Structures

Previously structs that were used on CPU/GPU would be written twice. Once using the CPU data types and once that uses the GPU data types. Developers were responsible to keep those structures consistent.

Shared structs can be defined in 'shader_shared' header files. For example the GPU_shader_shared.h. These headers can be included in C and CPP compile units.

/* In GPU_shader_shared.h */
struct MyStruct {
  float4x4 modelMatrix;
  float4 colors[3];
  bool1 do_fill;
  float dim_factor;
  float thickness;
  float _pad;
};
BLI_STATIC_ASSERT_ALIGN(MyStruct, 16)

Developer is still responsible to layout the struct (alignment and padding) so it can be used on the GPU. See these rules about correctly packing members.

GPU_SHADER_CREATE_INFO(my_shader)
  .typedef_source("GPU_shader_shared.h")
  .uniform_buf(0, "MyStruct", "my_struct");

This will create a uniform block binding at location 0 with content of type MyStruct named my_struct. The struct members can then be accessed also using my_struct (ex: my_struct.thickness).

Uniform and storage buffers can also be declared as array of struct like this:

GPU_SHADER_CREATE_INFO(my_shader)
  .typedef_source("GPU_shader_shared.h")
  .storage_buf(0, "MyStruct", "my_struct[]");

A shader create info can contain multiple 'typedef_source'. They are included only once and before any resource declaration (see gpu shader source ordering).

Geometry shaders

To be completed. Due to specific requirements of certain gpu backend input and output parameters of this stage should always use a named structure.

Compute shaders

For compute shaders the workgroup size must be defined. This can be done by calling the local_group_size function. This function accepts 1-3 parameters to define the local working group size for the x, y and z dimension.

GPU_SHADER_CREATE_INFO(draw_hair_refine_compute)
    /* ... */
    /* define a local group size where x=1 and y=1, z isn't defined. Missing parameters would fallback
     * to the platform default value. For OpenGL 4.3 this is also 1. */
    .local_group_size(1, 1)
    .compute_source("common_hair_refine_comp.glsl")
    /* ... */

C & C++ Code sharing

Code that needs to be shared between CPU and GPU implementation can also be put into 'shader_shared' header files. However, only a subset of C and C++ syntax is allowed for cross compilation to work:

No arrays except as input parameters.
No parameter reference & and likewise out and inout.
No pointers or references.
No iterators.
No namespace.
No template.
Use float suffix by default for float literals to avoid double promotion in C++.
Functions working on floats (ex: round(), exp(), pow() ...) might have different implementation on CPU and GPU.

See #103026 for more detail.

You can also declare enum inside these files. They will be correctly translated by our translation layer for GLSL compilation. However some rules to apply:

Always use u suffix for enum values. GLSL do not support implicit cast and enums are treated as uint under the hood.
Define all enum values. This is to simplify our custom pre-processor code.
(C++ only) Always use uint32_t as underlying type (enum eMyEnum : uint32_t).
(C only) do NOT use enum types inside UBO/SSBO structs and use uint instead (because enum size is implementation dependent in C).

ShaderBuilder

Using the CMAKE option WITH_GPU_SHADER_BUILDER=On will precompile each shader to make sure that the syntax is correcty and that all the generated code compiles and links with the main shader code.

Only shaders that are part of the SHADER_CREATE_INFOS and .do_static_compilation(true) is set, will be compiled. Enabling this option would reduce compile roundtrips when developing shaders as during compile time the shaders are validated, compiled and linked on the used platform.

Special Notes

ShaderBuilder is still in development and there are some limitations.

It produces a lot of linker warnings on MacOS #95047.
When using ShaderBuilder WITH_COMPILER_ASAN should be turned off due to runtime errors making compilation fail.
When using ShaderBuilder WITH_USD should be turned off due to linking errors.

https://devtalk.blender.org/t/build-with-dwith-gpu-shader-builder-on-failed/23105

Building Blender

Develop

Process

More

Contents

Source/EEVEE & Viewport/GPU Module/GLSL Cross Compilation

GLSL Cross Compilation

GPUShaderCreateInfo

Defining a new compile unit

Interface Info

Example Interface Info

Shader Info

Example Shader Info

Shader Source Order

Buffer Structures

Geometry shaders

Compute shaders

C & C++ Code sharing

ShaderBuilder

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Building Blender

Develop

Process

More

Contents

Source/EEVEE & Viewport/GPU Module/GLSL Cross Compilation

GLSL Cross Compilation

GPUShaderCreateInfo

Defining a new compile unit

Interface Info

Example Interface Info

Shader Info

Example Shader Info

Shader Source Order

Buffer Structures

Geometry shaders

Compute shaders

C & C++ Code sharing

ShaderBuilder