User:Jbakker/projects/EEVEE Compositing

EEVEE Compositing

Currently compositing using EEVEE scenes is limited. One of the reasons is that the render passes supported by EEVEE are not complete. This design will describe the render passes that is technical possible that would allow pass based compositing using EEVEE.

Scoping

Cryptomatte, Denoising Data and AOVs are out of scope for this design. We can add them later, but as they need special attention it will be better to create specific design proposal for them.

Requirements/Goal

The main requirement for this project is:

• It must be possible to construct the combined pass by mixing the individual render passes in the compositor.

This requirement directly enables many compositing tasks like color correction, material coloring, relighting etc.

Feasibility

In order to test the feasibility of this proposal we executed a proof of concept. This Proof of concept can be found at {D6331}. Here we demonstrated that it is possible to extract render passes and that these passes are usable in the compositor. It allows us to detail this proposal.

Existing render passes

This is the list of the render passes that we currently provide.

• Combined
• Data
  • Z
  • Depth
  • Normal
• Screen Space Effects
  • Subsurface Light
  • Subsurface Color
  • Ambient Occlusion

Required render passes

• Material
  • Diffuse Color
  • Diffuse Light (Direct+Indirect)
  • Glossy Color: (Note that the principled BSDF nodes do not support Glossy Color, only light) When using a Principled BSDF with a glossy component pure white (1, 1, 1) will be set. When it doesn't have a glossy component pure black (0, 0, 0) will be set.
  • Glossy Light
  • Emission
  • Refraction Color
  • Refraction Light
• World
  • Environment
• Volume
• Screen Space effects
  • Bloom: Is an additive pass with the changes that the bloom effect has to the render result.

Additional render passes

• Data
  • Material ID: Material ID could be loaded in the as a uniform in a specific fragment shader and rendered into a texture. Only the pixel center sample will be recorded.
• Screen space effects
  • Depth of Field: Depth of field could be an additive layer that is created by diffing the input and output of the DoF. Currently not sure how useful it is to have this as a render pass. we could use the defocus node in the compositor to do a similar effect.

Known limitations

• To construct these render passes we need to allocate memory on the GPU during the whole render time. This is a float RGB texture when rendering with many samples or a half RGB texture when rendering with fewer samples. Hardware limits (additional GPU RAM) depends on the the implementation of the OpenGL Driver. It could be that scenes would not be able to render on low-end cards due to memory limitations and how advanced the resource management in the driver is. When enabling 20 passes for a HD render with many samples it needs around of 500MB GPU Memory to store the results. In the future we could optimize this by re-rendering the full frame with different passes enabled.
• Z, Material ID and Depth of Field will not be active in the viewport rendering. We could add a proposal for supporting Material ID.
• `Ambient Occlusion is (also) mixed in the different light passes. We could remove them from the light passes, but that would make it harder to composite the combined pass from the individual passes.

Open Tasks

• Shadow pass is currently included in the different light passes. Do we want to separate shadow from the material light.
• Object ID is difficult to add due to that the meshes are sorted based on Material. I haven't looked into detail if we can use similar tricks like we do for the selection_id.
• Transmission still needs research how (and with what support) we could add transparency.
• Depth of Field needs more thought as an additive layer might not be use full. Perhaps sample based effect needs to be added next to the screen space effect or just use the compositor defocus node.