User:Ideasman42/BlenderLibsForLinux

Building Linux Libs

This page contains notes on maintaining Linux libraries, it is more a working set of notes and not official documentation.

Things to do on Fresh CentOS7

  • run: vi /etc/sysconfig/network-scripts/ifcfg-enp0s3 where enp0s3 may change based on the system. Change line ONBOOT=yes (was no). Reboot.
  • Follow steps from:
 https://wiki.blender.org/wiki/Building_Blender/Linux/CentOS7ReleaseEnvironment

Add public SSH key

Needed to log into the guest from the host. 

mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
vi ~/.ssh/authorized_keys
# Paste in public key.

~/.bash_profile: 

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
  . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH

# Custom Things.

# WARNING: when we have to support multiple different environments at once, 
# this is no longer a good idea as switching to end environment from another will cause problems.

if [ -f /etc/rocky-release ]; then
  source scl_source enable gcc-toolset-11
else
  source scl_source enable devtoolset-9
fi

# Handy single character path shortcuts.
alias B="cd /src/blender"
alias L="cd /src/lib/linux_x86_64"
alias D="cd /src/build_linux/deps"

SSH Into the Guest OS

While not essential, in general I find it much more convenient to SSH into the system, it allows copy/paste on-screen text, larger terminal display and doesn't lock the cursor to the window as VirtualBox does.

GUI-Less VirtualBox (host)

Running GUI-less is handy, allows to quit and restart the graphical session without interrupting the VM.

The VM can be started from a terminal as follows: 

VBoxHeadless --startvm "Redhat7" & disown

Run TMUX (guest)

Attach or join TMUX session, useful in case the SSH session disconnects (if the terminal is closed for e.g.). 

TERM=xterm-256color tmux new -As RedHat7


Mount the guest VM's filesystem (from the host)

This is handy because it allows using development environment and tools from the host system, accessing files can be a bit slow, otherwise this is convenient. 

sshfs -p 3022 [email protected]:/ /mnt/ext

Port forwarding needs to be set to forward 3022 on the host to 22 on the guest from virtualbox.

Notes from Sybren

What I do to release new builds is:

  1. SSH into my CentOS7 VM.
  2. Make sure the Blender sources are good to go (right branch, necessary patches applied).
  3. cd deps and make -j12 until done.
  4. on the real machine, run rsync_from_buildbot.sh the library (will post below) for every library that should be updated.
  5. do a test build of Blender and check nothing broke.
  6. submit the lot via SVN.

The rsync_from_buildbot.sh script: 


#!/bin/bash
set -ex

TARGET="$1"
shift

exec rsync \
  buildbot:/home/username/buildbot-builder/linux_glibc217_x86_64_cmake/lib/linux_x86_64/$TARGET/ \
  ./linux_centos7_x86_64/$TARGET/ \
  --exclude '__pycache__' \
  --delete --delete-excluded \
  -vaHAX "$@"

Fish command to perform an SVN-ADD: 

function svnadd -a SUBDIR -d "Add new files and remove deleted files"
  # This is the Fish way of doing `SVN_STATUS=$(<svn status $SUBDIR)`
  begin; set -l IFS; set SVN_STATUS (svn status $SUBDIR); end

  echo "$SVN_STATUS" | grep '^? ' | sed 's/^.//' | xargs --no-run-if-empty -n 1 svn add
  echo "$SVN_STATUS" | grep '^! ' | sed 's/^.//' | xargs --no-run-if-empty -n 1 svn rm
end

Modify SVN Config

~/.subversion/config

Otherwise it will also ignore .a and .so files, which is super annoying when you do the library maintenance work. 

global-ignores = .libs __pycache__ *.rej *~ #*# .#* .*.swp .DS_Store

Notes from Sergey

  • Sometimes needs clean build.
  • We don't do reproducable builds, so manually delete libraries before copying new ones over.
  • SVN Add can ignore ".a so make sure those are added (by default these get ignored).
  • Deps and linking issue: Some deps pick up system libraries. This can cause make deps to fail when making deps on the development environment.
  • Building libraries for future releases.
  • Don't start on new libs until stable release has been released (and the current libs are tagged).
  • build_files/build_environment/linux/linux-centos7-setup.sh Script should be kept small, make everything part of "make deps".
  • build_files/build_environment/cmake/versions.cmake (is source of truth for deps), if comments need to be added for deps (what they are used for, any general notes).
  • document why deps are needed - nice to have.
  • Multiple Python's versions are kept in SVN (why?, easy bisecting).

Submitting new Libraries

Update the task for libraries https://developer.blender.org/T101403

Don't Remove/Add New Libraries

When updating libraries DO NOT use: 

svn rm path/
# Copy the updated library over.
svn add path/

Doing so makes updating SVN fail if the directories contain any temporary files or cache (most common with Python's __pycache__), see:

See links where users ran into this problem:

Instead, delete the directory, copy over the new one, then use svn st to list files to be added/removed.

TODO: write a script that automates this step.


Patch Building DOES NOT use the latest libraries?

This can trip you up when testing if new functionality works with newly added/updated libs. The patch builder has logic to use the same version of libs when the patch was submitted.

The Solution is to use the branch-build functionality, instead of building patches.

Building Libraries

Link Time Optimization (LTO)

Link time optimization causes static libraries not to link when developers build with different GCC versions.

For example: 

/python/lib/libpython3.10.a’ generated with LTO version 11.2 instead of the expected 12.0
compilation terminated.
lto-wrapper: fatal error: /usr/bin/c++ returned 1 exit status
compilation terminated.

So while the builds will work (and pass tests) on the VM, this should not be used.

Building One Library at a time with Multiple Jobs

Building a single library at a time makes it much easier to debug when something goes wrong as the output isn't mixed with unrelated libraries.

Building with a single thread has the down-side that each library is build with one job, leaving cores idle and taking a very long time. There is also the problem of some large libraries being a bottleneck causing long build times when other small libraries have finished building.

CMake doesn't provide an obvious way to handle this.

CD'ing into each build directory and running make -j $(nproc) is an option but requires manual intervention.

The solution is to run make and pass MAKEFLAGS, so the top-level make is single threaded, but sub-processes use multiple jobs. 

make MAKEFLAGS="-j$(nproc)"

The ninja build system (used by meson) will already use multiple jobs, so it doesn't need special handling.

Testing Libraries

Changes to libraries should rebuild Blender and pass all tests.

Slow rebuilds from GPU Kernels

A make release build is very slow as it builds all GPU kernels, these are not used or necessary for testing make full build is sufficient for running tests.


Rebuilding All Libraries

Instead of removing the deps directory, remove everything except for the downloads. 

rm -rf $(find /src/build_linux/deps -mindepth 1 -maxdepth 1 -type d -not -name packages)
Retrieved from "https://wiki.blender.org/w/index.php?title=User:Ideasman42/BlenderLibsForLinux&oldid=37188"