xpra compile issue: ‘error: �AV_PIX_FMT_VAAPI_MOCO� undeclared’ (etc)

Symbols, symbols everywhere

Update: thanks to totaam for committing changes that solved the compile issue, this is sorted now!

---

I have been using xpra for what feels like over a decade now, as a ‘screen for GUI applications’. I use still use it as part of my Youtube workflow (covered elsewhere!), so that I can upload videos without my PC being on / across reboots.

With a recent version of xpra I lost the ability to use my scrollwheel. I had my distro’s version installed (4.2-rc0 at time of writing) but it was suggested that I should install the latest version in case there was a bug which has since been fixed.

As a side note, this is an excellent time to read up on the jwz xscreensaver d�b�cle of a couple years ago. The issues between upstream devs, who don’t want to be bombarded with queries about long-fixed bugs in old versions; distros who want to provide a stable base; and users who just want software which ‘works’ are quite interesting- albeit with the usual drama / heated discussions.

Anyway, I can’t compile xpra:

Cythonizing xpra/codecs/dec_avcodec2/decoder.pyx
running install
running build
running build_py
copying xpra/build_info.py -> build/lib.linux-x86_64-3.9/xpra
running build_ext
building 'xpra.codecs.dec_avcodec2.decoder' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ixpra/codecs/dec_avcodec2 -I/home/robert/.pyenv/versions/3.9.5/include/python3.9 -c xpra/codecs/dec_avcodec2/decoder.c -o build/temp.linux-x86_64-3.9/xpra/codecs/dec_avcodec2/decoder.o -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wall -Werror -fPIC
xpra/codecs/dec_avcodec2/decoder.c: In function �__pyx_pf_4xpra_6codecs_12dec_avcodec2_7decoder_7Decoder_init_context�:
xpra/codecs/dec_avcodec2/decoder.c:8446:23: error: assignment discards �const� qualifier from pointer target type [-Werror=discarded-qualifiers]
 8446 |   __pyx_v_self->codec = avcodec_find_decoder(__pyx_v_CodecID);
      |                       ^
xpra/codecs/dec_avcodec2/decoder.c: In function �__pyx_pymod_exec_decoder�:
xpra/codecs/dec_avcodec2/decoder.c:17957:37: error: �AV_PIX_FMT_VAAPI_MOCO� undeclared (first use in this function); did you mean �AV_PIX_FMT_VAAPI�?
17957 |   __pyx_t_1 = __Pyx_PyInt_From_long(AV_PIX_FMT_VAAPI_MOCO); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 353, __pyx_L1_error)
      |                                     ^~~~~~~~~~~~~~~~~~~~~
      |                                     AV_PIX_FMT_VAAPI
xpra/codecs/dec_avcodec2/decoder.c:17957:37: note: each undeclared identifier is reported only once for each function it appears in
xpra/codecs/dec_avcodec2/decoder.c:17969:37: error: �AV_PIX_FMT_VAAPI_IDCT� undeclared (first use in this function); did you mean �AV_PIX_FMT_VAAPI�?
17969 |   __pyx_t_1 = __Pyx_PyInt_From_long(AV_PIX_FMT_VAAPI_IDCT); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 354, __pyx_L1_error)
      |                                     ^~~~~~~~~~~~~~~~~~~~~
      |                                     AV_PIX_FMT_VAAPI
xpra/codecs/dec_avcodec2/decoder.c:17981:37: error: �AV_PIX_FMT_VAAPI_VLD� undeclared (first use in this function); did you mean �AV_PIX_FMT_VAAPI�?
17981 |   __pyx_t_1 = __Pyx_PyInt_From_long(AV_PIX_FMT_VAAPI_VLD); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 355, __pyx_L1_error)
      |                                     ^~~~~~~~~~~~~~~~~~~~
      |                                     AV_PIX_FMT_VAAPI
cc1: all warnings being treated as errors
error: command '/usr/bin/gcc' failed with exit code 1

There’s two types of error here:

• a warning about a const pointer target not being used as such (I think, I don’t speak much C)
• undeclared identifiers

Thanks to the helpful totaam in #xpra, we eventually figured that my ffmpeg libraries are not the right ones for xpra. Initially it seemed as though they were out of date; but actually xpra builds against version 4.4 of ffmpeg and I am running a git release ahead of that release, which deprecated some pixel formats:

I will open have opened a basic WIP PR so that xpra can be compiled with ffmpeg > 4.4.