Using OBS-STUDIO PPA

ERROR:

CMake Error at CMakeLists.txt:106 (find_package): By not providing "Findlibobs.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "libobs", but CMake did not find one.

Could not find a package configuration file provided by "libobs" with any of the following names:

libobsConfig.cmake
libobs-config.cmake

Add the installation prefix of "libobs" to CMAKE_PREFIX_PATH or set "libobs_DIR" to a directory containing one of the above files. If "libobs" provides a separate development package or SDK, be sure it has been installed.

I was able to successfully build the project and moved over the .so and data files to OBS, but unfortunately OBS will no longer run.

Removing obs-backgroundremoval.so from the plugins directory allows OBS to run again.

Excerpt from crash report:

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [2894]

Application Specific Information:
detected buffer overflow

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib             	0x00007fff2046210d __chk_fail_overflow + 16
1   libsystem_c.dylib             	0x00007fff2046274c __memcpy_chk + 18
2   libobs.0.dylib                	0x00000001076ac06b obs_register_source_s + 299
3   obs-backgroundremoval.so      	0x0000000028fa3b15 obs_module_load + 21
4   libobs.0.dylib                	0x00000001076aae0e obs_init_module + 78
5   libobs.0.dylib                	0x00000001076abc80 load_all_callback + 64
6   libobs.0.dylib                	0x00000001076abb7e obs_find_modules + 2046
7   libobs.0.dylib                	0x00000001076ab363 obs_load_all_modules + 35
8   com.obsproject.obs-studio     	0x00000001049c87ce OBSBasic::OBSInit() + 734
9   com.obsproject.obs-studio     	0x00000001049a4cd9 OBSApp::OBSInit() + 681
10  com.obsproject.obs-studio     	0x00000001049a8d28 main + 5032
11  libdyld.dylib                 	0x00007fff2052c621 start + 1

• pkgver_hyphen variable definition requires a tag on the latest git commit (which is not always the case). This is easily fixed by either adding a tag for each commit, or something like:

pkgver_hyphen=$(git describe --exact-match --tags $(git rev-list --tags --max-count=1))

• Also a minor issue in the build file where SIPNet_Softmax.onnx is not included in the data folder. I THINK it has worked by changing line 242 in the CmakeLists.txt to read install(FILES "data/SINet_Softmax_simple.onnx"

Hi, This is what happens when I try to build on Archlinux following the README instructions:

[[email protected] build]$ cmake .. && cmake --build . && cmake --install .
CMake Warning (dev) at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (Libobs)
  does not match the name of the calling package (LibObs).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  external/FindLibObs.cmake:89 (find_package_handle_standard_args)
  CMakeLists.txt:50 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Could NOT find Libobs (missing: LIBOBS_INCLUDE_DIR) 
CMake Error at external/FindLibObs.cmake:111 (message):
  Could not find the libobs library
Call Stack (most recent call first):
  CMakeLists.txt:50 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".

If I edit external/FindLibObs.cmake and change find_package_handle_standard_args(Libobs ....) to find_package_handle_standard_args(LibObs ....) I still get

[[email protected] build]$ cmake .. && cmake --build . && cmake --install .
CMake Error at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find LibObs (missing: LIBOBS_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  external/FindLibObs.cmake:89 (find_package_handle_standard_args)
  CMakeLists.txt:50 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".

I do have libobs under /usr/lib/pkgconfig where libobs.pc is found but if I export LIBOBS_INCLUDE_DIR="/usr/lib/pkgconfig/" the output doesn't change so I'm probably not doing it right. Any suggestions?

Good day.

Thank you for the recent updates. I'm still unable to use the plugin on OSX. So far:

• With the compiled plugin: onnxruntime cannot be found. If create a symlink to 1.5 from 1.7 the app crashes.
• I tried compiling on my machine again. With the updated code and instructions I'm able to run

cmake .. \
-DobsLibPath=/Applications/OBS.app/Contents/Frameworks \
-DobsIncludePath="../../obs-studio/libobs" \
-DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" \
-DLIBOBS_LIB="../../obs-studio/build/libobs/libobs.dylib"

but I DO have to specify DLIBOBS... attributes which are absent in the README. Not sure what I'm missing in my setup.

• Running cmake --build . fails with :

cmake --build .
Scanning dependencies of target obs-backgroundremoval
[ 33%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
make[2]: *** No rule to make target `<path>/obs-studio/build/libobs/libobs.dylib', needed by `obs-backgroundremoval.so'.  Stop.
make[1]: *** [CMakeFiles/obs-backgroundremoval.dir/all] Error 2
make: *** [all] Error 2

• I looked online and saw that other plugins are compiled using make -j4. I executed that and it worked(?):

Scanning dependencies of target obs-backgroundremoval
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
[100%] Linking CXX shared module obs-backgroundremoval.so
[100%] Built target obs-backgroundremoval

However after I copy the plugin, OBS fails to start:

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [19910]

Application Specific Information:
detected buffer overflow

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib             	0x00007fff67a3dfca __chk_fail_overflow.cold.1 + 16
1   libsystem_c.dylib             	0x00007fff67a3b214 __chk_fail_overflow + 9
2   libsystem_c.dylib             	0x00007fff67a3b848 __memcpy_chk + 18
3   libobs.0.dylib                	0x0000000110daf06b obs_register_source_s + 299
4   obs-backgroundremoval.so      	0x0000000028444925 obs_module_load + 21
5   libobs.0.dylib                	0x0000000110dade0e obs_init_module + 78
6   libobs.0.dylib                	0x0000000110daec80 load_all_callback + 64
7   libobs.0.dylib                	0x0000000110daeb7e obs_find_modules + 2046
8   libobs.0.dylib                	0x0000000110dae363 obs_load_all_modules + 35
9   com.obsproject.obs-studio     	0x000000010e0d47ce OBSBasic::OBSInit() + 734
10  com.obsproject.obs-studio     	0x000000010e0b0cd9 OBSApp::OBSInit() + 681
11  com.obsproject.obs-studio     	0x000000010e0b4d28 main + 5032
12  libdyld.dylib                 	0x00007fff6796ecc9 start + 1

And that's not ideal. Please advise.

I've had to opportunity to use this plugin now on several occasions to feed the virtual camera output into webex meetings, and it's been working well, but like many others I find that cpu usage is a bottleneck that effects performance. Tweaking and/or trying out different models could improve that. But another thing I was thinking of, and I'm not sure how feasible it is, but maybe an option could be created to reprocess the mask every X frames?

I'm guessing the work of the models reprocessing what the mask should be is the big hitter on cpu. Right now I can run about 15fps max against my cpu's (with some frame droppage), which isn't the smoothest experience as far as watching lips move on a web meeting, etc, But even rendering at 5fps the mask seems to hold up well. So I was thinking, what if for example there was an option to only reprocess the mask every X frames. Then, if you wanted the existing experience, you could set it to1, but it might be possible in my case to set it to 5 and be able to render 30fps video while in the background the models are only actually having to process 6fps. Again, I haven't reviewed the code to get an idea of how feasible it would be to implement, just throwing another idea out there on a way to maybe improve performance.

I get this:

info: [obs-backgroundremoval] Destroy scalers.
info: [obs-backgroundremoval] Model location (null)
error: [obs-backgroundremoval] Unable to get model filename from plugin.
info: User added filter 'Background Removal' (background_removal) to source 'cam1'
info: [obs-backgroundremoval] Destroy scalers.
info: [obs-backgroundremoval] Initialize scalers. Size 640 x 480
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 3) >= this->size() (which is 0)
[1]    34840 abort (core dumped)  obs

Any ideas? seems it cant find the models? what can I do?

Did you test out adding transparency with an alpha channel instead of setting the color?

There is an Alpha YUV format. So it seems like it wouldn't be too hard to add a parameter for transparency, convert to an AYUV, apply the mask, then convert back to the proper output format.

Thoughts? I can fork and give it a shot, but wanted to see if you had tried going direct to transparent already.

Hi Roy,

first of all, thank you so much for delving into the merits of developing a so much anticipated and much needed plugin for the awesome OBS.

However, with regards to the build instructions I would like to be so bold as to mention several issues which I experienced during my struggle to get an error free build leaving me with frustration and finally giving up on the endeavour.

First of all, I had an issue in which Homebrew would not install correctly on the version of macOS Catalina which I’m running. However, I made it to the point that I could install the mentioned prerequisites.

Secondly, and this is actually missing in you description, cmake is not part of the Xcode development environment which I’m using, but has to be brewed separately. Using the Homebrew package of cmake results in a SIGILL error which I didn’t try to track down. Instead I used the CMake Mac application which I downloaded from cmake.org.

Last, but not least, using the CMake application to generate the make files I encountered another error which pinpointed me to the necessity to have the OBS source code available to build the libobs module. However, trying to build this module again left me with errors which I didn’t spend time to investigate further upon.

All in all the prerequisites for building your background removal module seems to require refinement and also seems to be more complex for green field build environments.

Therefore I’d like to ask you if it would be possible for you to make available a binary build of the module together with the model data in a compressed file which users could download and install more easily. I’d be happy to support you in your efforts by making a donation via your preferred platform.

Kind regards, all the best, stay safe and healthy, and greetings from Switzerland

Mati

This commit patches for using the GPU on Linux as detailed in https://github.com/royshil/obs-backgroundremoval/issues/56

Additionally it expands on the installation instructions for Ubuntu.

The default instructions did not work for me on ubuntu 20.04 with obs studio ppa because obs did not detect the plugin installation. Upon investigation it turns out that the obs ppa is utilizing the /lib/obs-plugins which is not in the CMakeLists.txt. So I wrote a patch for it:

--- CMakeLists.txt      2021-09-07 14:43:11.033428381 +0200
+++ CMakeLists-patched.txt      2021-09-07 14:44:30.217512505 +0200
@@ -239,13 +239,13 @@
        file(GLOB onnxmodels data/*.onnx)
 
        install(TARGETS ${CMAKE_PROJECT_NAME}
-               LIBRARY DESTINATION "${CMAKE_INSTALL_FULL_LIBDIR}/obs-plugins")
+               LIBRARY DESTINATION "/lib/obs-plugins")
 
        install(FILES ${locale_files}
-               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
+               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
 
        install(FILES ${onnxmodels}
-               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/")
+               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/")
 endif()
 # --- End of section ---

And now obs studio successfully display the filter, however trying to add the filter it breaks with a segfault and this is what I see in the core dump:

StacktraceAddressSignature: /usr/bin/obs:11:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28+ad6b0:/usr/lib/obs-plugins/obs->
StacktraceTop:
 __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:314
 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator=(std::__cxx11::basic_string<ch>
 filter_update(void*, obs_data*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
 filter_create(obs_data*, obs_source*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
 ?? () from /lib/libobs.so.0

Versions are exact match (27.0.1 atm). The core dump has a path in it which is /usr//lib/obs-plugins and that does not look right to me. Ideas are welcomed.

PS: The default debian sources libobs0 and obs-studio did not work either because the filter is not detected at all so I'm sure it's using some other plugins directory like the ppa does. Either case I need the 26+ (deb one is 25) of obs due to its virtual camera (that plugin is a pain to maintain manually and breaks frequently).

Hello, With my MAC I had a problem with the plugin Background Removal. I have installed OBS v. 28.1.2 and after I put the plugin's files in /Applications/OBS.app/Contents/. When I use a filter on the video capture I don't see the specific option in the application.Why, OBS does not take into consideration the plugin download. Thanks for your help.

System

• Mac 12.6.1
• OBS 28.1.2
• OBS Background Removal 0.4.0

Problem

Removing the background and adding a color or chroma key to remove the background causes OBS to crash. I does not output any logs when it crashes

OBS 28.1.2 (mac) log file uploaded at 2022-12-15, 08:33:47

08:32:43.079: [macOS] Permission for audio device access granted.
08:32:43.089: [macOS] Permission for video device access granted.
08:32:43.110: [macOS] Permission for accessibility denied.
08:32:43.112: [macOS] Permission for screen capture denied.
08:32:43.112: Rosetta translation used: false
08:32:43.112: CPU Name: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
08:32:43.112: CPU Speed: 2600MHz
08:32:43.112: Physical Cores: 6, Logical Cores: 12
08:32:43.112: Physical Memory: 16384MB Total
08:32:43.112: OS Name: macOS
08:32:43.112: OS Version: Version 12.6.1 (Build 21G217)
08:32:43.112: Kernel Version: 21.6.0
08:32:43.114: hotkeys-cocoa: Using layout 'com.apple.keylayout.US'
08:32:43.115: Current Date/Time: 2022-12-15, 08:32:43
08:32:43.115: Browser Hardware Acceleration: true
08:32:43.115: Qt Version: 6.3.1 (runtime), 6.3.1 (compiled)
08:32:43.115: Portable mode: false
08:32:43.380: OBS 28.1.2 (mac)
08:32:43.380: ---------------------------------
08:32:43.382: ---------------------------------
08:32:43.382: audio settings reset:
08:32:43.382: 	samples per sec: 48000
08:32:43.382: 	speakers:        2
08:32:43.382: 	max buffering:   960 milliseconds
08:32:43.382: 	buffering type:  dynamically increasing
08:32:43.388: ---------------------------------
08:32:43.388: Initializing OpenGL...
08:32:44.698: Loading up OpenGL on adapter ATI Technologies Inc. AMD Radeon Pro 5300M OpenGL Engine
08:32:44.698: OpenGL loaded successfully, version 4.1 ATI-4.8.101, shading language 4.10
08:32:45.141: ---------------------------------
08:32:45.141: video settings reset:
08:32:45.141: 	base resolution:   1920x1080
08:32:45.141: 	output resolution: 1920x1080
08:32:45.141: 	downscale filter:  Bicubic
08:32:45.141: 	fps:               30/1
08:32:45.141: 	format:            NV12
08:32:45.141: 	YUV mode:          Rec. 709/Partial
08:32:45.141: NV12 texture support not available
08:32:45.141: P010 texture support not available
08:32:45.142: Audio monitoring device:
08:32:45.142: 	name: Default
08:32:45.142: 	id: default
08:32:45.142: ---------------------------------
08:32:45.167: No AJA devices found, skipping loading AJA UI plugin
08:32:45.167: Failed to initialize module 'aja-output-ui'
08:32:45.187: No AJA devices found, skipping loading AJA plugin
08:32:45.187: Failed to initialize module 'aja'
08:32:45.192: Failed to load 'en-US' text for module: 'decklink-captions'
08:32:45.197: Failed to load 'en-US' text for module: 'decklink-output-ui'
08:32:45.199: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
08:32:45.199: Failed to initialize module 'decklink'
08:32:45.674: [VideoToolbox encoder]: Adding VideoToolbox encoders
08:32:45.836: [obs-browser]: Version 2.18.7
08:32:45.836: [obs-browser]: CEF Version 103.0.5060.134 (runtime), 103.61.23+g2fe5a76+chromium-103.0.5060.134 (compiled)
08:32:45.888: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.0.1 | RPC Version: 1)
08:32:45.888: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.3.1 | Qt version (run-time): 6.3.1
08:32:45.888: [obs-websocket] [obs_module_load] Linked ASIO Version: 101201
08:32:45.898: [obs-websocket] [obs_module_load] Module loaded.
08:32:45.915: os_dlopen(/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib->/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib): dlopen(/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib, 0x0101): tried: '/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib' (no such file)
08:32:45.915: 
08:32:45.915: [vlc-video]: Couldn't find VLC installation, VLC video source disabled
08:32:45.945: Failed to load 'en-US' text for module: 'obs-backgroundremoval.so'
08:32:45.945: [obs-backgroundremoval] plugin loaded successfully (version 0.3.0)
08:32:45.945: ---------------------------------
08:32:45.945:   Loaded Modules:
08:32:45.945:     obs-backgroundremoval.so
08:32:45.945:     vlc-video
08:32:45.945:     text-freetype2
08:32:45.945:     rtmp-services
08:32:45.945:     obs-x264
08:32:45.945:     obs-websocket
08:32:45.945:     obs-vst
08:32:45.945:     obs-transitions
08:32:45.945:     obs-outputs
08:32:45.945:     obs-filters
08:32:45.945:     obs-ffmpeg
08:32:45.945:     obs-browser
08:32:45.945:     mac-virtualcam
08:32:45.945:     mac-videotoolbox
08:32:45.945:     mac-syphon
08:32:45.945:     mac-capture
08:32:45.945:     mac-avcapture
08:32:45.945:     image-source
08:32:45.945:     frontend-tools
08:32:45.945:     decklink-output-ui
08:32:45.945:     decklink-captions
08:32:45.945:     coreaudio-encoder
08:32:45.945: ---------------------------------
08:32:45.946: ==== Startup complete ===============================================
08:32:46.031: All scene data cleared
08:32:46.031: ------------------------------------------------
08:32:46.115: coreaudio: Device 'Microsoft Teams Audio' [48000 Hz] initialized
08:32:46.115: [Loaded global audio device]: 'Desktop Audio'
08:32:46.133: coreaudio: Device 'MacBook Pro Microphone' [48000 Hz] initialized
08:32:46.133: [Loaded global audio device]: 'Mic/Aux'
08:32:46.135: adding 21 milliseconds of audio buffering, total audio buffering is now 21 milliseconds (source: Desktop Audio)
08:32:46.135: 
08:32:46.153: Video Capture Device: Selected device 'FaceTime HD Camera (Built-in)'
08:32:46.156: Video Capture Device: Using preset 1280x720
08:32:46.243: [obs-backgroundremoval] Destroy scalers.
08:32:46.310: [obs-backgroundremoval] Model mediapipe.onnx input 0: name input_1:0 shape (4 dim) 1 x 96 x 160 x 3
08:32:46.310: [obs-backgroundremoval] Model mediapipe.onnx output 0: name segment:0 shape (4 dim) 1 x 96 x 160 x 2
08:32:46.310: [obs-backgroundremoval] Allocated 46080 sized float-array for input 0
08:32:46.311: [obs-backgroundremoval] Allocated 30720 sized float-array for output 0
08:32:46.477: Switched to scene 'Scene'
08:32:46.481: ------------------------------------------------
08:32:46.481: Loaded scenes:
08:32:46.481: - scene 'Scene':
08:32:46.489:     - source: 'Image' (image_source)
08:32:46.489:     - source: 'Video Capture Device' (av_capture_input_v2)
08:32:46.489:         - filter: 'Background Removal' (background_removal)
08:32:46.489:         - filter: 'Color Key' (color_key_filter_v2)
08:32:46.489: ------------------------------------------------
08:32:47.880: [obs-backgroundremoval] Destroy scalers.
08:32:47.880: [obs-backgroundremoval] Initialize scalers. Size 1280 x 720

Apple made Vision.framework and it uses GPU so that the background removal by Vision.framework on macOS is super efficient.

I implemented the code with Vision.framework and thus you can use Apple GPU for the background removal now.

You can install the latest binary from here https://github.com/umireon/obs-backgroundremoval/releases/tag/v0.4.4-umireon

The inference device shows CPU but it actually uses GPU when Vision (Apple) was selected.

I changed config of my camera by applying flips (vertical, horizontal, clockwise and counterclockwise) both from camera config itself and from OBS filters, but it plugin failed to respond to those changes. E.g. the torso of my body shown in the opposite direction.

Can you please add support to account for those geometric transformations?

In Ubuntu 22.04 when I try to cmake --build . following the installation instructions I get this error:

error: ‘OrtSessionOptionsAppendExecutionProvider_DML’ was not declared in this scope; did you mean ‘OrtSessionOptionsAppendExecutionProvider_CPU’? 215 | Ort::ThrowOnError(OrtSessionOptionsAppendExecutionProvider_DML(sessionOptions, 0)); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | OrtSessionOptionsAppendExecutionProvider_CPU gmake[2]: *** [CMakeFiles/obs-backgroundremoval.dir/build.make:76: CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/obs-backgroundremoval.dir/all] Error 2 gmake: *** [Makefile:156: all] Error 2

Some Idea to solve it? Thanks