Tools/Tests/Setup

Automated Tests

Blender uses the CMake testing framework to run Python and GTest tests. Only part of Blender's code base is covered, developers are highly encouraged to add more tests.

Setup

Tests require Blender to be built, see the official build instructions. This document assumes the directory layout to be:

~/blender-git/blender  # Blender source directory
~/blender-git/build    # Blender build directory

Running the tests is then done with make test:

cd ~/blender-git/blender
make test

GTest tests need Blender to be built with WITH_GTESTS enabled.

Downloading Test Files

Additional binary test files are needed for running the Blender tests. These are available through an SVN repository, separate from the Blender source repository.

make test will automatically download these files the first time it runs.

To download the test files manually:

cd ~/blender-git
mkdir lib
cd lib
svn checkout https://svn.blender.org/svnroot/bf-blender/trunk/lib/tests

Updating Test Files

Test files change along with the Blender source code. make update will update tests to the latest version if you have downloaded them once, together with the Blender source code.

cd ~/blender-git/blender
make update

Running Tests

make test runs all available tests. For fine control, use ctest directly.

ctest must be run from the build directory.

cd ~/blender-git/build

Some example commands are:

# List available tests
ctest -N
# Run all tests with name matching "alembic"
ctest -R alembic
# Show verbose output for a test, to get more information on why it failed
ctest -VV -R alembic
# Run tests in parallel, and show verbose output for failed tests
ctest -j 4 --output-on-failure

When building with Visual Studio or Xcode, the build configuration needs to be specified for every ctest command, for example:

ctest -C Release -R alembic

Some GTest tests bundle together all tests within a Blender module. It's possible to run only specific test within a module as follows. Find the name of the module test, for example by listing all tests with ctest -N.

  ...
  Test  #5: bf_blenkernel_tests
  ...

Then run the test with verbose output:

ctest -R bf_blenkernel_tests -V

This shows the exact command that is run for each test. You can then copy-paste that command to run it yourself.

Running Tests on ASAN builds

Running tests on ASAN builds is tricky, because ASAN will fail most of them due to false/unrelated memory leak detection. However, completely 'silencing' ASAN reports using the exitcode=0 ASAN_OPTIONS is a (very) bad idea, as it will also hide many actual issues, including segfaults!

The first step is to remove memleaks errors which happen in third party libraries, see the ASAN page for details.

The guardedalloc test intentionally attempts to allocate an invalid amount of memory and expects to fail and return NULL in that case. allocator_may_return_null=true is required for this test to behave as expected.

ASAN_OPTIONS="allocator_may_return_null=true" LSAN_OPTIONS="print_suppressions=false:suppressions=/path/to/blender/tools/config/analysis/lsan.supp" ctest -j 4 --output-on-failure

No LSAN Suppressions In case LSAN suppressions do not work for some reason, it is also possible to pass leak_check_at_exit=false to ASAN_OPTIONS. However, doing this will also hide valid memleaks reports, which should be avoided as much as possible.

Adding Tests

Tests can be written in Python or C++. For tests that can easily be written in Python, this is preferred. Lower-level tests can be written in C++ files next to the sources under test. See the language-specific pages for more info.

Render Tests

Inspection

Render and OpenGL drawing tests produce a HTML file to show the result of running regression tests, comparing renders and screenshots against a reference image. These can be found at:

~/blender-git/build/tests/report.html

Adding or Updating Tests

When adding a new test, or when there are benign test failures due to intentional changes, the reference images can be created or updated as follows:

cd ~/blender-git/build
# Update reference images for cycles tests
BLENDER_TEST_UPDATE=1 ctest -R cycles

After this all the tests should pass, and .blend files and references images can be committed.

Cycles GPU

For testing Cycles GPU rendering, CYCLES_TEST_DEVICES in the CMake configuration is a list of devices to test. By default it is only CPU, but CUDA, OPTIX, ONEAPI and HIP can be added.

EEVEE and Workbench

WITH_OPENGL_RENDER_TESTS enables EEVEE and Workbench tests, which are currently disabled by default due to different outputs of different GPUs. They render the same .blend files as Cycles.

Modifier Tests

Similar to render tests, modifier tests compare the output to a reference mesh. Updating the reference mesh for new test or on changes is done in the same way:

cd ~/blender-git/build
BLENDER_TEST_UPDATE=1 ctest -R modifiers