An OBS plugin for removing background in portrait images (video), making it easy to replace the background when screen recording.

Overview

OBS Plugin: Background Removal

Download

Check out the latest releases for downloads and install instructions.

Introduction

This plugin is meant to make it easy to replace the background in portrait images and video. It is using a neural network to predict the mask of the portrait and remove the background pixels. It's easily composable with other OBS plugins to replace the background with e.g. an image or a transparent color.

The model used for background detection is SINet: https://arxiv.org/abs/1911.09099 The pre-trained model weights were taken from: https://github.com/anilsathyan7/Portrait-Segmentation/tree/master/SINet

Some more information about how I built it: https://www.morethantechnical.com/2021/04/15/obs-plugin-for-portrait-background-removal-with-onnx-sinet-model/

Building

The plugin was built and tested on Mac OSX, Windows and Ubuntu Linux. Help is appreciated in building on other OSs and formalizing the one-click installers.

Prerequisites for building

Mac OSX

Install dependencies

You may use homebrew:

$ brew install opencv onnxruntime

Or - you may also build a (very minimal) version of OpenCV and ONNX Runtime for static-linking, instead of the homebrew ones:

<root>/build/ $ ../scripts/makeOpenCV_osx.sh
<root>/build/ $ ../scripts/makeOnnxruntime_osx.sh

Static linking should be more robust across versions of OSX, as well as building for 10.13.

Finding libobs

If you install the desktop OBS app (https://obsproject.com/download) you already have the binaries for libobs (e.g. /Applications/OBS.app/Contents/Frameworks/libobs.0.dylib) But you don't have the headers - so clone the main obs repo e.g. git clone --single-branch -b 26.1.2 [email protected]:obsproject/obs-studio.git (match the version number to your OBS install. Right now on OSX it's 26.1.2)

Build

$ mkdir build && cd build
$ cmake .. -DobsLibPath=/Applications/OBS.app/Contents/Frameworks -DobsIncludePath=~/Downloads/obs-studio/libobs
$ cmake --build .

Install

Add it to your OBS install, e.g.

$ cp obs-backgroundremoval.so /Applications/OBS.app/Contents/PlugIns
$ mkdir -p /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval/
$ cp ../data/SINet_Softmax.onnx /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval/

Linux / Ubuntu

Install dependencies

$ apt install -y libobs-dev libopencv-dev language-pack-en wget git build-essential cmake
$ wget https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-1.7.0.tgz
$ tar xzvf onnxruntime-linux-x64-1.7.0.tgz --strip-components=1 -C /usr/local/ --wildcards "*/include/*" "*/lib*/"

Build and install

$ mkdir build && cd build
$ cmake .. && cmake --build . && cmake --install .

Windows

We will use static linking (as much as possible) to aviod having to lug around .DLLs with the plugin.

Install OpenCV via vcpkg:

$ mkdir build
$ cd build
$ git clone https://github.com/microsoft/vcpkg
$ cd vcpkg
$ .\bootstrap-vcpkg.bat
$ .\vcpkg.exe install opencv[core]:x64-windows-static

Unzip an ONNX runtime release: https://github.com/microsoft/onnxruntime/releases to e.g. Downloads. You should have a directory like <Downloads>\Microsoft.AI.MachineLearning.1.7.2\.

Clone the OBS repo, Downloads\ $ git clone [email protected]:obsproject/obs-studio.git, to e.g. Downloads. Checkout tag 26.1.1: Downloads\obs-studio\ $ git checkout 26.1.1

Build the plugin:

$ cmake .. -DobsPath="<Downloads>\obs-studio\" -DOnnxruntime_INCLUDE_HINT=<Downloads>\Microsoft.AI.MachineLearning.1.7.2\build\native\include -DOnnxruntime_DIR=<Downloads>\Microsoft.AI.MachineLearning.1.7.2\runtimes\win-x64\_native\static
$ cmake --build . --config Release
Issues
  • Can now Install. [Commit bedcaf9. Refer to @Zahrun] Ubuntu 22.04 LTS

    Can now Install. [Commit bedcaf9. Refer to @Zahrun] Ubuntu 22.04 LTS

    Using OBS-STUDIO PPA

    ERROR:

    CMake Error at CMakeLists.txt:106 (find_package): By not providing "Findlibobs.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "libobs", but CMake did not find one.

    Could not find a package configuration file provided by "libobs" with any of the following names:

    libobsConfig.cmake
    libobs-config.cmake
    

    Add the installation prefix of "libobs" to CMAKE_PREFIX_PATH or set "libobs_DIR" to a directory containing one of the above files. If "libobs" provides a separate development package or SDK, be sure it has been installed.

    opened by ShadowStorm0 15
  • OBS Studio 26.1.2 will not open (MacOS)

    OBS Studio 26.1.2 will not open (MacOS)

    I was able to successfully build the project and moved over the .so and data files to OBS, but unfortunately OBS will no longer run.

    Removing obs-backgroundremoval.so from the plugins directory allows OBS to run again.

    Excerpt from crash report:

    Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
    Exception Codes:       0x0000000000000001, 0x0000000000000000
    Exception Note:        EXC_CORPSE_NOTIFY
    
    Termination Signal:    Illegal instruction: 4
    Termination Reason:    Namespace SIGNAL, Code 0x4
    Terminating Process:   exc handler [2894]
    
    Application Specific Information:
    detected buffer overflow
    
    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0   libsystem_c.dylib             	0x00007fff2046210d __chk_fail_overflow + 16
    1   libsystem_c.dylib             	0x00007fff2046274c __memcpy_chk + 18
    2   libobs.0.dylib                	0x00000001076ac06b obs_register_source_s + 299
    3   obs-backgroundremoval.so      	0x0000000028fa3b15 obs_module_load + 21
    4   libobs.0.dylib                	0x00000001076aae0e obs_init_module + 78
    5   libobs.0.dylib                	0x00000001076abc80 load_all_callback + 64
    6   libobs.0.dylib                	0x00000001076abb7e obs_find_modules + 2046
    7   libobs.0.dylib                	0x00000001076ab363 obs_load_all_modules + 35
    8   com.obsproject.obs-studio     	0x00000001049c87ce OBSBasic::OBSInit() + 734
    9   com.obsproject.obs-studio     	0x00000001049a4cd9 OBSApp::OBSInit() + 681
    10  com.obsproject.obs-studio     	0x00000001049a8d28 main + 5032
    11  libdyld.dylib                 	0x00007fff2052c621 start + 1
    
    opened by benmurden 15
  • PKGBUILD archlinux minor issues

    PKGBUILD archlinux minor issues

    • pkgver_hyphen variable definition requires a tag on the latest git commit (which is not always the case). This is easily fixed by either adding a tag for each commit, or something like:

    pkgver_hyphen=$(git describe --exact-match --tags $(git rev-list --tags --max-count=1))

    • Also a minor issue in the build file where SIPNet_Softmax.onnx is not included in the data folder. I THINK it has worked by changing line 242 in the CmakeLists.txt to read install(FILES "data/SINet_Softmax_simple.onnx"
    opened by krisezra87 11
  • Not building on Archlinux

    Not building on Archlinux

    Hi, This is what happens when I try to build on Archlinux following the README instructions:

    [[email protected] build]$ cmake .. && cmake --build . && cmake --install .
    CMake Warning (dev) at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
      The package name passed to `find_package_handle_standard_args` (Libobs)
      does not match the name of the calling package (LibObs).  This can lead to
      problems in calling code that expects `find_package` result variables
      (e.g., `_FOUND`) to follow a certain pattern.
    Call Stack (most recent call first):
      external/FindLibObs.cmake:89 (find_package_handle_standard_args)
      CMakeLists.txt:50 (find_package)
    This warning is for project developers.  Use -Wno-dev to suppress it.
    
    -- Could NOT find Libobs (missing: LIBOBS_INCLUDE_DIR) 
    CMake Error at external/FindLibObs.cmake:111 (message):
      Could not find the libobs library
    Call Stack (most recent call first):
      CMakeLists.txt:50 (find_package)
    
    
    -- Configuring incomplete, errors occurred!
    See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".
    

    If I edit external/FindLibObs.cmake and change find_package_handle_standard_args(Libobs ....) to find_package_handle_standard_args(LibObs ....) I still get

    [[email protected] build]$ cmake .. && cmake --build . && cmake --install .
    CMake Error at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
      Could NOT find LibObs (missing: LIBOBS_INCLUDE_DIR)
    Call Stack (most recent call first):
      /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
      external/FindLibObs.cmake:89 (find_package_handle_standard_args)
      CMakeLists.txt:50 (find_package)
    
    
    -- Configuring incomplete, errors occurred!
    See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".
    

    I do have libobs under /usr/lib/pkgconfig where libobs.pc is found but if I export LIBOBS_INCLUDE_DIR="/usr/lib/pkgconfig/" the output doesn't change so I'm probably not doing it right. Any suggestions?

    opened by morrolinux 10
  • Still having build issues on OSX

    Still having build issues on OSX

    Good day.

    Thank you for the recent updates. I'm still unable to use the plugin on OSX. So far:

    • With the compiled plugin: onnxruntime cannot be found. If create a symlink to 1.5 from 1.7 the app crashes.
    • I tried compiling on my machine again. With the updated code and instructions I'm able to run
    cmake .. \
    -DobsLibPath=/Applications/OBS.app/Contents/Frameworks \
    -DobsIncludePath="../../obs-studio/libobs" \
    -DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" \
    -DLIBOBS_LIB="../../obs-studio/build/libobs/libobs.dylib"
    

    but I DO have to specify DLIBOBS... attributes which are absent in the README. Not sure what I'm missing in my setup.

    • Running cmake --build . fails with :
    cmake --build .
    Scanning dependencies of target obs-backgroundremoval
    [ 33%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
    [ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
    make[2]: *** No rule to make target `<path>/obs-studio/build/libobs/libobs.dylib', needed by `obs-backgroundremoval.so'.  Stop.
    make[1]: *** [CMakeFiles/obs-backgroundremoval.dir/all] Error 2
    make: *** [all] Error 2
    
    • I looked online and saw that other plugins are compiled using make -j4. I executed that and it worked(?):
    Scanning dependencies of target obs-backgroundremoval
    [ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
    [ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
    [100%] Linking CXX shared module obs-backgroundremoval.so
    [100%] Built target obs-backgroundremoval
    

    However after I copy the plugin, OBS fails to start:

    Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
    Exception Codes:       0x0000000000000001, 0x0000000000000000
    Exception Note:        EXC_CORPSE_NOTIFY
    
    Termination Signal:    Illegal instruction: 4
    Termination Reason:    Namespace SIGNAL, Code 0x4
    Terminating Process:   exc handler [19910]
    
    Application Specific Information:
    detected buffer overflow
    
    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0   libsystem_c.dylib             	0x00007fff67a3dfca __chk_fail_overflow.cold.1 + 16
    1   libsystem_c.dylib             	0x00007fff67a3b214 __chk_fail_overflow + 9
    2   libsystem_c.dylib             	0x00007fff67a3b848 __memcpy_chk + 18
    3   libobs.0.dylib                	0x0000000110daf06b obs_register_source_s + 299
    4   obs-backgroundremoval.so      	0x0000000028444925 obs_module_load + 21
    5   libobs.0.dylib                	0x0000000110dade0e obs_init_module + 78
    6   libobs.0.dylib                	0x0000000110daec80 load_all_callback + 64
    7   libobs.0.dylib                	0x0000000110daeb7e obs_find_modules + 2046
    8   libobs.0.dylib                	0x0000000110dae363 obs_load_all_modules + 35
    9   com.obsproject.obs-studio     	0x000000010e0d47ce OBSBasic::OBSInit() + 734
    10  com.obsproject.obs-studio     	0x000000010e0b0cd9 OBSApp::OBSInit() + 681
    11  com.obsproject.obs-studio     	0x000000010e0b4d28 main + 5032
    12  libdyld.dylib                 	0x00007fff6796ecc9 start + 1
    

    And that's not ideal. Please advise.

    opened by mctrafik 10
  • Performance Idea - option to reprocess mask every X frames

    Performance Idea - option to reprocess mask every X frames

    I've had to opportunity to use this plugin now on several occasions to feed the virtual camera output into webex meetings, and it's been working well, but like many others I find that cpu usage is a bottleneck that effects performance. Tweaking and/or trying out different models could improve that. But another thing I was thinking of, and I'm not sure how feasible it is, but maybe an option could be created to reprocess the mask every X frames?

    I'm guessing the work of the models reprocessing what the mask should be is the big hitter on cpu. Right now I can run about 15fps max against my cpu's (with some frame droppage), which isn't the smoothest experience as far as watching lips move on a web meeting, etc, But even rendering at 5fps the mask seems to hold up well. So I was thinking, what if for example there was an option to only reprocess the mask every X frames. Then, if you wanted the existing experience, you could set it to1, but it might be possible in my case to set it to 5 and be able to render 30fps video while in the background the models are only actually having to process 6fps. Again, I haven't reviewed the code to get an idea of how feasible it would be to implement, just throwing another idea out there on a way to maybe improve performance.

    opened by iBenMoore 8
  • crash after adding it to the cam

    crash after adding it to the cam

    I get this:

    info: [obs-backgroundremoval] Destroy scalers.
    info: [obs-backgroundremoval] Model location (null)
    error: [obs-backgroundremoval] Unable to get model filename from plugin.
    info: User added filter 'Background Removal' (background_removal) to source 'cam1'
    info: [obs-backgroundremoval] Destroy scalers.
    info: [obs-backgroundremoval] Initialize scalers. Size 640 x 480
    terminate called after throwing an instance of 'std::out_of_range'
      what():  vector::_M_range_check: __n (which is 3) >= this->size() (which is 0)
    [1]    34840 abort (core dumped)  obs
    

    Any ideas? seems it cant find the models? what can I do?

    opened by r3k2 8
  • What about an alpha channel?

    What about an alpha channel?

    Did you test out adding transparency with an alpha channel instead of setting the color?

    There is an Alpha YUV format. So it seems like it wouldn't be too hard to add a parameter for transparency, convert to an AYUV, apply the mask, then convert back to the proper output format.

    Thoughts? I can fork and give it a shot, but wanted to see if you had tried going direct to transparent already.

    opened by troypesola 8
  • Incomplete build description and missing dependencies

    Incomplete build description and missing dependencies

    Hi Roy,

    first of all, thank you so much for delving into the merits of developing a so much anticipated and much needed plugin for the awesome OBS.

    However, with regards to the build instructions I would like to be so bold as to mention several issues which I experienced during my struggle to get an error free build leaving me with frustration and finally giving up on the endeavour.

    First of all, I had an issue in which Homebrew would not install correctly on the version of macOS Catalina which I’m running. However, I made it to the point that I could install the mentioned prerequisites.

    Secondly, and this is actually missing in you description, cmake is not part of the Xcode development environment which I’m using, but has to be brewed separately. Using the Homebrew package of cmake results in a SIGILL error which I didn’t try to track down. Instead I used the CMake Mac application which I downloaded from cmake.org.

    Last, but not least, using the CMake application to generate the make files I encountered another error which pinpointed me to the necessity to have the OBS source code available to build the libobs module. However, trying to build this module again left me with errors which I didn’t spend time to investigate further upon.

    All in all the prerequisites for building your background removal module seems to require refinement and also seems to be more complex for green field build environments.

    Therefore I’d like to ask you if it would be possible for you to make available a binary build of the module together with the model data in a compressed file which users could download and install more easily. I’d be happy to support you in your efforts by making a donation via your preferred platform.

    Kind regards, all the best, stay safe and healthy, and greetings from Switzerland

    Mati

    opened by MatiMax 8
  • Linux GPU Usage

    Linux GPU Usage

    This commit patches for using the GPU on Linux as detailed in https://github.com/royshil/obs-backgroundremoval/issues/56

    Additionally it expands on the installation instructions for Ubuntu.

    opened by mproffitt 7
  • Compatibility issue with the obs-studio ppa

    Compatibility issue with the obs-studio ppa

    The default instructions did not work for me on ubuntu 20.04 with obs studio ppa because obs did not detect the plugin installation. Upon investigation it turns out that the obs ppa is utilizing the /lib/obs-plugins which is not in the CMakeLists.txt. So I wrote a patch for it:

    --- CMakeLists.txt      2021-09-07 14:43:11.033428381 +0200
    +++ CMakeLists-patched.txt      2021-09-07 14:44:30.217512505 +0200
    @@ -239,13 +239,13 @@
            file(GLOB onnxmodels data/*.onnx)
     
            install(TARGETS ${CMAKE_PROJECT_NAME}
    -               LIBRARY DESTINATION "${CMAKE_INSTALL_FULL_LIBDIR}/obs-plugins")
    +               LIBRARY DESTINATION "/lib/obs-plugins")
     
            install(FILES ${locale_files}
    -               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
    +               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
     
            install(FILES ${onnxmodels}
    -               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/")
    +               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/")
     endif()
     # --- End of section ---
    
    

    And now obs studio successfully display the filter, however trying to add the filter it breaks with a segfault and this is what I see in the core dump:

    StacktraceAddressSignature: /usr/bin/obs:11:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28+ad6b0:/usr/lib/obs-plugins/obs->
    StacktraceTop:
     __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:314
     std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator=(std::__cxx11::basic_string<ch>
     filter_update(void*, obs_data*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
     filter_create(obs_data*, obs_source*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
     ?? () from /lib/libobs.so.0
    

    Versions are exact match (27.0.1 atm). The core dump has a path in it which is /usr//lib/obs-plugins and that does not look right to me. Ideas are welcomed.

    PS: The default debian sources libobs0 and obs-studio did not work either because the filter is not detected at all so I'm sure it's using some other plugins directory like the ppa does. Either case I need the 26+ (deb one is 25) of obs due to its virtual camera (that plugin is a pain to maintain manually and breaks frequently).

    opened by chickencoding123 5
  • MacOS under Build instructions there is an error with the cmake command

    MacOS under Build instructions there is an error with the cmake command

    Here is the error when running the cmake command: CMake Error: The source directory "/Users/" does not appear to contain CMakeLists.txt. I'm assuming that the command needs to be run from a specific working directory. Please modify the command so that includes the full path.

    opened by pfs7 0
  • Problems with plugin innstallation in Ubuntu 22.04, OBS-Studio 27.2.4

    Problems with plugin innstallation in Ubuntu 22.04, OBS-Studio 27.2.4

    Dear Roy,

    Thanks for the great plugin.

    I don't think this is an issue with the code itself. I think it is a problem with the installation, perhaps we are missing some details or functions.

    Following strictly the instructions, result in an error and incompatibilities with libobs versions.
    It seems that apt only has the older version. I spend a lot of time trying to learn by myself how to solve this problem.
    I failed, so I would like to ask for guidance.

    I think the issue is that the plugin obs-backgroundremoval doesn't build properly in Ubuntu 22.04 with the obs-studio 27.2.4

    The issue of missing Findlibobs.cmake and subsequent other Find functions, I tried to solve, but there was always something that I couldn't solve. Perhaps you could double-check to see if there is really a problem.

    BELOW ARE THE CONFIGURATION AND ACTIONS:

    System is Ubuntu 22.04 5.15.0-40-generic

    OBS-Studio 27.2.4 was installed CUDA-Toolkit, CUDA-DNN was installed libopencv was installed

    OBS-Studio headers were installed using: git clone --single-branch -b 27.2.4 [email protected]:obsproject/obs-studio.git ALTHOUGH I DON'T KNOW HOW THESE HEADERS INTERACT WITH THE BUILDING OF THE PLUGIN!!

    ONNXRUNTIME CPU installed using the provided commands: $ export ONNX_VERSION=1.10.0 $ sudo apt install cuda libcudnn8 $ wget https://github.com/microsoft/onnxruntime/releases/download/v${ONNX_VERSION}/onnxruntime-linux-x64-gpu-${ONNX_VERSION}.tgz $ sudo tar xzvf onnxruntime-linux-x64-gpu-${ONNX_VERSION}.tgz --strip-components=1 -C /usr/local/ --wildcards "/include/" "/lib/"

    OTHER CONFIGURATION cmake version 3.22.1 libopencv 4.5.6_dfsg-9ubuntu4 nvidia-cudnn/jammy,now 8.2.4.15~cuda11.4 nvidia-cuda-toolkit/jammy,now 11.5.1-1ubuntu1

    When installing dependencies: sudo apt install -y libobs-dev libopencv-dev language-pack-en wget git build-essential cmake libsimde-dev

    there is a problem: The following packages have unmet dependencies: obs-studio : Conflicts: libobs0 but 27.2.3+dfsg1-1 is to be installed E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

    So, I proceeded without libobs-dev sudo apt install -y libopencv-dev language-pack-en wget git build-essential cmake libsimde-dev

    OTHER CONFIGURATION build-essential is already the newest version (12.9ubuntu3). cmake is already the newest version (3.22.1-1ubuntu1). language-pack-en is already the newest version (1:22.04+20220415). wget is already the newest version (1.21.2-2ubuntu1). libopencv-dev is already the newest version (4.5.4+dfsg-9ubuntu4). libsimde-dev is already the newest version (0.7.2-6). git is already the newest version (1:2.34.1-1ubuntu1.2). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

    LIBOBS IN /USR This is what I find about installed libobs:

    find /usr |grep -i libobs /usr/share/obs/libobs /usr/share/obs/libobs/deinterlace_yadif_2x.effect /usr/share/obs/libobs/default.effect /usr/share/obs/libobs/format_conversion.effect /usr/share/obs/libobs/default_rect.effect /usr/share/obs/libobs/deinterlace_base.effect /usr/share/obs/libobs/deinterlace_linear_2x.effect /usr/share/obs/libobs/deinterlace_yadif.effect /usr/share/obs/libobs/area.effect /usr/share/obs/libobs/solid.effect /usr/share/obs/libobs/deinterlace_linear.effect /usr/share/obs/libobs/deinterlace_discard.effect /usr/share/obs/libobs/bicubic_scale.effect /usr/share/obs/libobs/deinterlace_discard_2x.effect /usr/share/obs/libobs/repeat.effect /usr/share/obs/libobs/bilinear_lowres_scale.effect /usr/share/obs/libobs/opaque.effect /usr/share/obs/libobs/deinterlace_blend.effect /usr/share/obs/libobs/premultiplied_alpha.effect /usr/share/obs/libobs/lanczos_scale.effect /usr/share/obs/libobs/deinterlace_blend_2x.effect /usr/lib/libobs-frontend-api.so /usr/lib/libobs-frontend-api.so.0 /usr/lib/x86_64-linux-gnu/pkgconfig/libobs.pc /usr/lib/libobs-opengl.so.0.0 /usr/lib/libobs.so.0 /usr/lib/libobs.so /usr/lib/libobsglad.so /usr/lib/libobs-scripting.so /usr/lib/libobs-opengl.so /usr/lib/cmake/LibObs /usr/lib/cmake/LibObs/LibObsConfig.cmake /usr/lib/cmake/LibObs/LibObsConfigVersion.cmake /usr/lib/cmake/LibObs/LibObsTarget.cmake /usr/lib/cmake/LibObs/LibObsTarget-none.cmake /usr/lib/libobs-opengl.so.0 /usr/lib/libobsglad.so.0 /usr/lib/libobs-frontend-api.so.0.0

    This is about ONNXRUNTIME find /usr |grep -i onnx /usr/local/lib/libonnxruntime_providers_cuda.so /usr/local/lib/libonnxruntime_providers_tensorrt.so /usr/local/lib/libonnxruntime_providers_shared.so /usr/local/lib/libonnxruntime.so /usr/local/lib/libonnxruntime.so.1.10.0 /usr/local/include/onnxruntime_run_options_config_keys.h /usr/local/include/onnxruntime_c_api.h /usr/local/include/onnxruntime_session_options_config_keys.h /usr/local/include/onnxruntime_cxx_api.h /usr/local/include/onnxruntime_cxx_inline.h

    THEN

    gh repo clone royshil/obs-backgroundremoval TO CREATE THE LOCAL COPY

    THEN $ cd obs-backgroundremoval $ sudo ldconfig # required if you have previously had a different version of onnxruntime installed $ mkdir build && cd build $ cmake -DWITH_CUDA=ON .. && cmake --build . && sudo cmake --install .

    THIS IS THE FINAL ERROR: cmake -DWITH_CUDA=ON .. -- Found Git: /usr/bin/git (found version "2.34.1") -- The C compiler identification is GNU 11.2.0 -- The CXX compiler identification is GNU 11.2.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Error at CMakeLists.txt:106 (find_package): By not providing "Findlibobs.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "libobs", but CMake did not find one.

    Could not find a package configuration file provided by "libobs" with any of the following names:

    libobsConfig.cmake
    libobs-config.cmake
    

    Add the installation prefix of "libobs" to CMAKE_PREFIX_PATH or set "libobs_DIR" to a directory containing one of the above files. If "libobs" provides a separate development package or SDK, be sure it has been installed.

    -- Configuring incomplete, errors occurred!

    Any guidance would be welcomed, Roy

    Thanks in advance.

    opened by ny2292000 1
  • How can I use this in GPU mode?

    How can I use this in GPU mode?

    I tried setting the Inference device to "GPU - DirectML", but it did not seem to have effect on the CPU usage. Do I need some packages like CUDA installed to make it work?

    opened by claell 0
  • Chroma key with feather filter creates some color spill for me

    Chroma key with feather filter creates some color spill for me

    I want to use this plugin with a virtual background. For that, it is suggested to use a chroma key filter.

    However, together with the "Feather blend silhouette" option of this, the chroma key filter creates a color spill on the edges, which doesn't look nice:

    grafik

    If I don't use the "Feather blend silhouette", the edges don't look good. I didn't find an option to later add the feather blend (for example, after or in chroma key filter).

    Ideally, one could simply make the background transparent in this filter without the need of additional chroma key filter in the chain. Unfortunately, currently, the color picker doesn't give me that option.

    Do you know a good solution or possibly can you add some?

    For now, I just reduce the strength of "Feather blend silhouette" as a workaround.

    opened by claell 0
  • Idea: GPU Only version or Use as Effect Filter?

    Idea: GPU Only version or Use as Effect Filter?

    Hi guys,

    I was wonder if it would be possible to use background removal as an Effect Filter? The RTX version of this works this way I believe and while I know it's probably no easy feat I figured I'd ask if it's possible to use it as an Effect Filter instead of a Video Filter, since Video Filters can't be applied to source mirrors or nested scenes. I'd really love it so I could achieve this without an RTX card. Please let me know if this is possible!

    Thank you so much for all your hard work!

    opened by MetacLeod 0
Releases(v0.4.0)
  • v0.4.0(Nov 3, 2021)

    In this release

    • Upgrade to OBS v27.1.3 (latest) support
    • New realtime model Robust Video Matting (https://github.com/PeterL1n/RobustVideoMatting)
    • Calculate mask every X frame - save on CPU cycles! (set to 2 for a 50% performance gain with unnoticeable quality drop)
    • Some internal abstraction of the models to make it easy to add more models in the future
    • More advancement on CUDA - but still no stable version... need help on this plz.

    I'm taking the releases off of "Pre-Release" since it seems 10,000s of people were able to successfully use the plugin.

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    • Archlinux: install the package e.g. pacman -U obs-backgroundremoval*.tar.zst

    What's Changed

    • (rough) use CUDA by @packysauce in https://github.com/royshil/obs-backgroundremoval/pull/29
    • Process Mask every X frames of video by @royshil in https://github.com/royshil/obs-backgroundremoval/pull/31
    • Add "obs" path suffix to allow compilation with OBS Studio 27.1.3 (ppa) by @fubar-coder in https://github.com/royshil/obs-backgroundremoval/pull/45

    New Contributors

    • @packysauce made their first contribution in https://github.com/royshil/obs-backgroundremoval/pull/29
    • @royshil made their first contribution in https://github.com/royshil/obs-backgroundremoval/pull/31
    • @fubar-coder made their first contribution in https://github.com/royshil/obs-backgroundremoval/pull/45

    Full Changelog: https://github.com/royshil/obs-backgroundremoval/compare/v0.3.0-beta...v0.4.0

    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.4.0-win64.zip(49.85 MB)
    obs-backgroundremoval-macosx.zip(55.39 MB)
  • v0.3.0-beta(Jun 17, 2021)

    In this release

    • Added 2 new lightweight segmentation models: Google Mediapipe "Meet" model, and the "Selfie Segmentation" model, both from https://github.com/PINTO0309/PINTO_model_zoo
    • Upgrade to OBS v27.0.1 (following OBS production releases on OSX and Windows)
    • Upgrade onnxruntime on OSX to v1.7.2
    • Fix issue with RGB vs BGR background color flip

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    • Archlinux: install the package e.g. pacman -U obs-backgroundremoval*.tar.zst
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.3.0_beta-1-x86_64.pkg.tar.zst(30.72 MB)
    obs-backgroundremoval-macosx.zip(42.08 MB)
    obs-backgroundremoval-win64.zip(59.74 MB)
  • v0.2.5-beta(May 25, 2021)

    In this release

    • Archlinux build and package

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    • Archlinux: install the package e.g. pacman -U obs-backgroundremoval*.tar.zst
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.2.5_beta-1-x86_64.pkg.tar.zst(30.03 MB)
    obs-backgroundremoval-macosx.zip(41.10 MB)
    obs-backgroundremoval-win64.zip(59.01 MB)
  • v0.2.4-beta(May 11, 2021)

    In this release

    • Adding the MODNet segmentation model. Slightly better in segmentation and resources.
    • Dropdown to choose model (SINet or MODNet)

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(41.10 MB)
    obs-backgroundremoval-win64.zip(59.01 MB)
  • v0.2.3-beta(May 7, 2021)

    In this release

    • Optimized ONNX model with slightly better runtime/resources performance (5-10%)
    • Fix to DirectML build (including the DLL)

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(18.60 MB)
    obs-backgroundremoval-win64.zip(13.31 MB)
  • v0.2.2-beta(May 4, 2021)

    In this release

    • Better versioning of the plugin
    • GPU support on Windows through DirectML (any available GPU)

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio)
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(18.27 MB)
    obs-backgroundremoval-win64.zip(12.99 MB)
  • v0.2.1-beta(May 3, 2021)

    In this release

    • Fixing the problem of down&up scaling the original image. Instead - apply the mask to the original image size, without reducing its resolution.

    Install

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio)
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/ -Linux: TBD
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(18.27 MB)
    obs-backgroundremoval-win64.zip(4.64 MB)
  • 0.2-beta(May 2, 2021)

    This release features

    • Omni color space handling
    • Slightly better performance

    Install

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio)
    • Mac OSX: Copy obs-backgroundremoval.so to /Applications/OBS.app/Contents/PlugIns. Copy SInet_Softmax.onnx to /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval -Linux: TBD
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.2beta-1-x86_64.pkg.tar.zst(3.84 KB)
    obs-backgroundremoval-win64.zip(4.34 MB)
    obs-backgroundremoval.so(39.69 MB)
    SINet_Softmax.onnx(428.53 KB)
  • 0.1-beta(Apr 26, 2021)

    In this release

    • Windows support
    • Linux (Ubuntu) support
    • More streamlined build
    • Colorspaces conversions

    To install

    • Windows: Copy .dlls to e.g. C:\Program Files\obs-studio\obs-plugins\64bit, and .onnx file to e.g. C:\Program Files\obs-studio\data\obs-plugins\obs-backgroundremoval
    • MacOSX: Copy .so file to /Applications/OBS.app/Contents/PlugIns and .onnx file to /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval

    Linux install TBD

    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval.dll(3.76 MB)
    obs-backgroundremoval.so(170.69 KB)
    onnxruntime.dll(7.62 MB)
    SINet_Softmax.onnx(428.53 KB)
  • 0.1-alpha(Apr 23, 2021)

    This is an alpha pre-release just to provide the pre-built binaries, to avoid having to build them. The plugin binary and model are supplied and they should be copied to the OBS plugins directory manually, e.g.

    $ cp obs-backgroundremoval.so /Applications/OBS.app/Contents/PlugIns
    $ cp SINet_Softmax.onnx /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval/
    

    OpenCV and ONNXRuntime are still needed for the plugin to run. Use homebrew to install them:

    $ brew install opencv onnxruntime
    

    In case homebrew doesn't install v1.7.2 of onnxruntime - i've included the dylib.

    Source code(tar.gz)
    Source code(zip)
    libonnxruntime.1.7.2.dylib(12.01 MB)
    obs-backgroundremoval.so(218.32 KB)
    SINet_Softmax.onnx(428.53 KB)
Owner
Roy Shilkrot
Roy Shilkrot
An OBS plugin for Zoom/Meet-like virtual background feature.

OBS Virtual background plugin English / Japanese OBS Virtual background plugin is a plugin for OBS. This plugin allows you to crop the background area

KOUNOIKE Yuusuke 163 Aug 9, 2022
"Zero-copy" Linux screen capture plugin for OBS that uses libdrm and dmabuf

linux-kmsgrab plugin for OBS Introduction This plugin is a proof-of-concept libdrm-based screen capture for OBS. It uses DMA-BUF to import CRTC frameb

Ivan Avdeev 51 Jun 19, 2022
OBS Plugin to capture CS:GO with Trusted Mode enabled

OBS Plugin to capture CS:GO with Trusted Mode enabled Ever since Valve introduced Trusted Mode you could no longer capture Counter-Strike: Global Offe

Gregor Steiner 101 Jul 15, 2022
OBS Plugin with image that reacts to sound source.

OBS Image Reaction Plugin Image that reacts to sound source. Installing binaries Download binaries from Releases. For Windows, Move the contents of pl

null 28 Aug 1, 2022
audio monitor filter for OBS Studio

Audio Monitor dock and filter for OBS Studio Plugin for OBS Studio to add Audio Monitor dock and filter. It allows you to put the audio of a OBS sourc

Exeldro 187 Jul 30, 2022
Software for making digital voice work without the smart hub in the way

Software for making digital voice work without the smart hub in the way

Charlie 2 Jun 7, 2022
Block all ads in vídeo, áudio, banner, anti-skip

NoAdSpotify Block spotify ad This is an updated and simplified version of the project: BlockTheSpot Last updated: 6th June 2021 Last tested version: 1

null 13 Apr 7, 2022
Example for transmit video + audio to tv via hackRF

Этот код может передавать звук и изображение на телевизор через hackRF В этом проекте использовался код из следующих репозиториев: https://github.com/

null 8 Jun 14, 2022
Single file C library for decoding MPEG1 Video and MP2 Audio

PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer Single-file MIT licensed library for C/C++ See pl_mpeg.h for the documentation. Why?

Dominic Szablewski 566 Jul 27, 2022
cute_dsp is a C API for various DSP effects suitable for video games

cute_dsp is a C API for various DSP effects suitable for video games and meant to interface directly with the cute_sound library created by Randy Gaul

Matt Rosen 13 May 29, 2021
PeakEater is a free open-source VST3/AU waveshaper plugin

PeakEater Free open source VST3/AU wave shaping plugin for macOS and Windows. View Demo · Download · Report Bug Table of Contents About The Project Fe

Vladyslav Voinov 42 Aug 2, 2022
BYOD is a guitar distortion plugin with a customisable signal chain that allows users to create their own guitar distortion effects.

BYOD is a guitar distortion plugin with a customisable signal chain that allows users to create their own guitar distortion effects. The plugin contains a wide variety of distortion effects from analog modelled circuits to purely digital creations, along with some musical tone-shaping filters, and a handful of other useful processing blocks.

null 155 Aug 7, 2022
ChowKick is a kick drum synthesizer plugin based on creative modelling of old-school drum machine circuits

ChowKick is a kick drum synthesizer plugin based on creative modelling of old-school drum machine circuits. MIDI input to the plugin triggers a pulse with a parameterized size and shape. The pulse is then passed into a resonant filter which can be tuned to a specific frequency, or matched to the frequency of the incoming MIDI notes.

null 103 Aug 6, 2022
A weird distortion and lowpass plugin

Delta Limiter Building ./waf clean configure build Documentation pandoc -s user_manual.md -o user_manual.html pandoc -s user_manual.md -o user_manual.

Hannes Braun 2 Jan 26, 2022
SuperCollider as a VST3 plugin

PluginCollider PluginCollider is an experimental fork of https://github.com/supercollider/SuperColliderAU and it is still not complete since it only b

Pascal Gauthier 24 Aug 8, 2022
Wolf MidiCurve is a MIDI remapping plugin.

Wolf MidiCurve is a MIDI remapping plugin. It can be built as an LV2 or VST plugin and as a standalone Jack application.

Wolf Plugins 7 Nov 18, 2021
Audio Plugin API

Audio Plugin API

null 1.1k Aug 1, 2022
Easy and efficient audio synthesis in C++

Tonic Fast and easy audio synthesis in C++. Prefer coding to patching? Love clean syntax? Care about performance? That's how we feel too, and why we m

null 466 Aug 9, 2022
Audacity is an easy-to-use, multi-track audio editor and recorder for Windows, Mac OS X, GNU/Linux and other operating systems

Audacity is an easy-to-use, multi-track audio editor and recorder for Windows, Mac OS X, GNU/Linux and other operating systems. Audacity is open source software licensed under GPL, version 2 or later.

Audacity 7.8k Aug 5, 2022