[solved] Building OBS in Linux: “recompile with -fPIC”

Why do I need to change the pilot in command anyway?

tl;dr: if and only if your problem is the wrong pyenv being used when building from AUR, you can use the CMake hint set(Python_FIND_VIRTUALENV ONLY) in src/obs-studio-git/deps/obs-scripting/obspython/CMakeLists.txt

The issue: `PyExc_OverflowError’ can not be used when making a shared object; recompile with -fPIC

I wanted to update OBS to the latest version in AUR, which tracks the git version of OBS. I’m sure I installed the git version for a reason…

[2020-12-31T15:59:11+0000] [PACMAN] Running 'pacman -U /tmp/trizen-robert/obs-studio-git/obs-studio-git-26.1.0.r30.gd46e8b03c-1-x86_64.pkg.tar.zst'

I have a vague memory of installing it to use a version with the virtual camera while that was still experimental? Anyway, it’s what I have, and I ran into an issue when I went to update it:

[ 21%] Built target libobs
Consolidate compiler generated dependencies of target obs-frontend-api
[ 21%] Linking CXX shared library libobs-frontend-api.so
Installing OBS rundir
[ 21%] Built target obs-frontend-api
Consolidate compiler generated dependencies of target obs-scripting
[ 21%] Linking CXX shared library libobs-scripting.so
/usr/bin/ld: /home/robert/.pyenv/versions/3.9.5/lib/libpython3.9.a(pythonrun.o): warning: relocation against `_Py_UnhandledKeyboardInterrupt' in read-only section `.text'
/usr/bin/ld: /home/robert/.pyenv/versions/3.9.5/lib/libpython3.9.a(longobject.o): relocation R_X86_64_PC32 against symbol `PyExc_OverflowError' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[2]: *** [deps/obs-scripting/CMakeFiles/obs-scripting.dir/build.make:221: deps/obs-scripting/libobs-scripting.so.29] Error 1
make[1]: *** [CMakeFiles/Makefile2:946: deps/obs-scripting/CMakeFiles/obs-scripting.dir/all] Error 2
make: *** [Makefile:156: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
:: Unable to build obs-studio-git - makepkg exited with code: 4

Judging by a search of the OBS discord history, this is a common issue that people run into. The user @Tuna pointed out to me that the -fPIC issue isn’t with OBS, but with the library it’s linking to- python, in this case.

Installing another Python

Helpfully, I have pyenv installed, which I think I installed back when I was doing the OCR automation of the Deep Rock Galactic videos as there were quite a few python packages that I didn’t necessarily want ‘cluttering up’ my system python.

Installing a particular version of python is easy easy as pyenv install <version>. I have 3.9.5 installed as my system version, so I installed 3.9.4 with a shared python library: env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.4, then created a pyenv based on that: pyenv virtualenv 3.9.4 obsbuild, and activated it via pyenv activate obsbuild.

I then rebuilt, and…

-- Found Python: /home/robert/.pyenv/shims/python3.9 (found version "3.9.4") found components: Interpreter Development Development.Module Development.Embed
-- OBS:  ENABLED    obspython
-- OBS:  obs-scripting -> Python 3.9.4 found.

/usr/bin/ld: /home/robert/.pyenv/versions/3.9.5/lib/libpython3.9.a(longobject.o): relocation R_X86_64_PC32 against symbol `PyExc_OverflowError' can not be used when making a shared object; recompile with -fPIC

(snip)

(obsbuild) $ python --version
Python 3.9.4

Huh. It was detecting 3.9.4 (the one with a shared library), but still using 3.9.5?

A Useful CMake Hint

OBS uses CMake for building, and so I went looking for a way to make it use the ‘correct’ python version. Via an answer on SO, I found a useful CMake hint: Python_FIND_VIRTUALENV. By setting this to ONLY, the behaviour it uses is: “Only the virtual environment is used to look-up for the interpreter.” Just the badger!

Built! Let’s run it…

$ obs
obs: error while loading shared libraries: libpython3.9.so.1.0: cannot open shared object file: No such file or directory

Pointing to the library

Interestingly, even being in the ‘obsbuild’ venv wasn’t enough to let that build of OBS find the library.

We can set LD_LIBRARY_PATH to the directory of the python library (in my case ~/.pyenv/versions/3.9.4/lib/), and OBS should run:

LD_LIBRARY_PATH=/home/robert/.pyenv/versions/3.9.4/lib/:$LD_LIBRARY_PATH obs

Well, it runs until it segfaults?

A cautionary note on rabbit / shark holes, or: The Fallacy of Apparent Progress

When I write things up here, I like to include a section where there is reflection on the process. It’s useful for me and it might be useful for other people too, to know the thought process involved- the why as well as the how.

I started on this path because I was writing up another post in the Blender text addon (QTE) series, which was a release. It being a release and a public (GitHub) one at that, I wanted it to look the part, which meant recording a demo. So I opened up OBS and was setting up scenes when I noticed guides were missing, so I figured I’d update OBS, and that’s what kicked off this post.

There are elements of perfectionism here — “I don’t want to put something out for the world to see that gives a poor impression / is less than my ‘best’ — and another thing that I can’t call the term for to mind, but I’ll call ‘The Fallacy of Apparent Progress’, best illustrated with an image I’ve borrowed before:

The thought process goes something along the lines of:

“I’ll keep working on this thing because I’m making progress in that area, even though it’s not really related or is disproportionate effort for the effect on the original purpose.”

The Fallacy of Apparent Progress (or similar)

The point is it’s worth assessing what you’re doing and why you’re doing it. When the OBS I had built segfaulted, I could have chased that up, but instead I installed the stable version from the community repo. After updating StreamFX addon to the experimental version for OBS 28 from AUR (and noting the comment about branch), I had a working OBS.

Looking back, while I learned a bit about CMake directives I could have avoided the linker/python/library issues altogether, which may have been a better use of my time!