An OBS plugin for removing background in portrait images (video), making it easy to replace the background when screen recording.

Overview

OBS Plugin: Background Removal

Download

Check out the latest releases for downloads and install instructions.

Introduction

This plugin is meant to make it easy to replace the background in portrait images and video. It is using a neural network to predict the mask of the portrait and remove the background pixels. It's easily composable with other OBS plugins to replace the background with e.g. an image or a transparent color.

The model used for background detection is SINet: https://arxiv.org/abs/1911.09099 The pre-trained model weights were taken from: https://github.com/anilsathyan7/Portrait-Segmentation/tree/master/SINet

Some more information about how I built it: https://www.morethantechnical.com/2021/04/15/obs-plugin-for-portrait-background-removal-with-onnx-sinet-model/

Building

The plugin was built and tested on Mac OSX, Windows and Ubuntu Linux. Help is appreciated in building on other OSs and formalizing the one-click installers.

Prerequisites for building

Mac OSX

Install dependencies

You may use homebrew:

$ brew install opencv onnxruntime

Or - you may also build a (very minimal) version of OpenCV and ONNX Runtime for static-linking, instead of the homebrew ones:

<root>/build/ $ ../scripts/makeOpenCV_osx.sh
<root>/build/ $ ../scripts/makeOnnxruntime_osx.sh

Static linking should be more robust across versions of OSX, as well as building for 10.13.

Finding libobs

If you install the desktop OBS app (https://obsproject.com/download) you already have the binaries for libobs (e.g. /Applications/OBS.app/Contents/Frameworks/libobs.0.dylib) But you don't have the headers - so clone the main obs repo e.g. git clone --single-branch -b 26.1.2 [email protected]:obsproject/obs-studio.git (match the version number to your OBS install. Right now on OSX it's 26.1.2)

Build

$ mkdir build && cd build
$ cmake .. -DobsLibPath=/Applications/OBS.app/Contents/Frameworks -DobsIncludePath=~/Downloads/obs-studio/libobs
$ cmake --build .

Install

Add it to your OBS install, e.g.

$ cp obs-backgroundremoval.so /Applications/OBS.app/Contents/PlugIns
$ mkdir -p /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval/
$ cp ../data/SINet_Softmax.onnx /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval/

Linux / Ubuntu

Install dependencies

$ apt install -y libobs-dev libopencv-dev language-pack-en wget git build-essential cmake
$ wget https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-1.7.0.tgz
$ tar xzvf onnxruntime-linux-x64-1.7.0.tgz --strip-components=1 -C /usr/local/ --wildcards "*/include/*" "*/lib*/"

Build and install

$ mkdir build && cd build
$ cmake .. && cmake --build . && cmake --install .

Windows

We will use static linking (as much as possible) to aviod having to lug around .DLLs with the plugin.

Install OpenCV via vcpkg:

$ mkdir build
$ cd build
$ git clone https://github.com/microsoft/vcpkg
$ cd vcpkg
$ .\bootstrap-vcpkg.bat
$ .\vcpkg.exe install opencv[core]:x64-windows-static

Unzip an ONNX runtime release: https://github.com/microsoft/onnxruntime/releases to e.g. Downloads. You should have a directory like <Downloads>\Microsoft.AI.MachineLearning.1.7.2\.

Clone the OBS repo, Downloads\ $ git clone [email protected]:obsproject/obs-studio.git, to e.g. Downloads. Checkout tag 26.1.1: Downloads\obs-studio\ $ git checkout 26.1.1

Build the plugin:

$ cmake .. -DobsPath="<Downloads>\obs-studio\" -DOnnxruntime_INCLUDE_HINT=<Downloads>\Microsoft.AI.MachineLearning.1.7.2\build\native\include -DOnnxruntime_DIR=<Downloads>\Microsoft.AI.MachineLearning.1.7.2\runtimes\win-x64\_native\static
$ cmake --build . --config Release
Comments
  • Can now Install. [Commit bedcaf9. Refer to @Zahrun] Ubuntu 22.04 LTS

    Can now Install. [Commit bedcaf9. Refer to @Zahrun] Ubuntu 22.04 LTS

    Using OBS-STUDIO PPA

    ERROR:

    CMake Error at CMakeLists.txt:106 (find_package): By not providing "Findlibobs.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "libobs", but CMake did not find one.

    Could not find a package configuration file provided by "libobs" with any of the following names:

    libobsConfig.cmake
    libobs-config.cmake
    

    Add the installation prefix of "libobs" to CMAKE_PREFIX_PATH or set "libobs_DIR" to a directory containing one of the above files. If "libobs" provides a separate development package or SDK, be sure it has been installed.

    opened by ShadowStorm0 17
  • OBS Studio 26.1.2 will not open (MacOS)

    OBS Studio 26.1.2 will not open (MacOS)

    I was able to successfully build the project and moved over the .so and data files to OBS, but unfortunately OBS will no longer run.

    Removing obs-backgroundremoval.so from the plugins directory allows OBS to run again.

    Excerpt from crash report:

    Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
    Exception Codes:       0x0000000000000001, 0x0000000000000000
    Exception Note:        EXC_CORPSE_NOTIFY
    
    Termination Signal:    Illegal instruction: 4
    Termination Reason:    Namespace SIGNAL, Code 0x4
    Terminating Process:   exc handler [2894]
    
    Application Specific Information:
    detected buffer overflow
    
    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0   libsystem_c.dylib             	0x00007fff2046210d __chk_fail_overflow + 16
    1   libsystem_c.dylib             	0x00007fff2046274c __memcpy_chk + 18
    2   libobs.0.dylib                	0x00000001076ac06b obs_register_source_s + 299
    3   obs-backgroundremoval.so      	0x0000000028fa3b15 obs_module_load + 21
    4   libobs.0.dylib                	0x00000001076aae0e obs_init_module + 78
    5   libobs.0.dylib                	0x00000001076abc80 load_all_callback + 64
    6   libobs.0.dylib                	0x00000001076abb7e obs_find_modules + 2046
    7   libobs.0.dylib                	0x00000001076ab363 obs_load_all_modules + 35
    8   com.obsproject.obs-studio     	0x00000001049c87ce OBSBasic::OBSInit() + 734
    9   com.obsproject.obs-studio     	0x00000001049a4cd9 OBSApp::OBSInit() + 681
    10  com.obsproject.obs-studio     	0x00000001049a8d28 main + 5032
    11  libdyld.dylib                 	0x00007fff2052c621 start + 1
    
    opened by benmurden 15
  • PKGBUILD archlinux minor issues

    PKGBUILD archlinux minor issues

    • pkgver_hyphen variable definition requires a tag on the latest git commit (which is not always the case). This is easily fixed by either adding a tag for each commit, or something like:

    pkgver_hyphen=$(git describe --exact-match --tags $(git rev-list --tags --max-count=1))

    • Also a minor issue in the build file where SIPNet_Softmax.onnx is not included in the data folder. I THINK it has worked by changing line 242 in the CmakeLists.txt to read install(FILES "data/SINet_Softmax_simple.onnx"
    opened by krisezra87 11
  • Not building on Archlinux

    Not building on Archlinux

    Hi, This is what happens when I try to build on Archlinux following the README instructions:

    [[email protected] build]$ cmake .. && cmake --build . && cmake --install .
    CMake Warning (dev) at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
      The package name passed to `find_package_handle_standard_args` (Libobs)
      does not match the name of the calling package (LibObs).  This can lead to
      problems in calling code that expects `find_package` result variables
      (e.g., `_FOUND`) to follow a certain pattern.
    Call Stack (most recent call first):
      external/FindLibObs.cmake:89 (find_package_handle_standard_args)
      CMakeLists.txt:50 (find_package)
    This warning is for project developers.  Use -Wno-dev to suppress it.
    
    -- Could NOT find Libobs (missing: LIBOBS_INCLUDE_DIR) 
    CMake Error at external/FindLibObs.cmake:111 (message):
      Could not find the libobs library
    Call Stack (most recent call first):
      CMakeLists.txt:50 (find_package)
    
    
    -- Configuring incomplete, errors occurred!
    See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".
    

    If I edit external/FindLibObs.cmake and change find_package_handle_standard_args(Libobs ....) to find_package_handle_standard_args(LibObs ....) I still get

    [[email protected] build]$ cmake .. && cmake --build . && cmake --install .
    CMake Error at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
      Could NOT find LibObs (missing: LIBOBS_INCLUDE_DIR)
    Call Stack (most recent call first):
      /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
      external/FindLibObs.cmake:89 (find_package_handle_standard_args)
      CMakeLists.txt:50 (find_package)
    
    
    -- Configuring incomplete, errors occurred!
    See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".
    

    I do have libobs under /usr/lib/pkgconfig where libobs.pc is found but if I export LIBOBS_INCLUDE_DIR="/usr/lib/pkgconfig/" the output doesn't change so I'm probably not doing it right. Any suggestions?

    opened by morrolinux 10
  • Still having build issues on OSX

    Still having build issues on OSX

    Good day.

    Thank you for the recent updates. I'm still unable to use the plugin on OSX. So far:

    • With the compiled plugin: onnxruntime cannot be found. If create a symlink to 1.5 from 1.7 the app crashes.
    • I tried compiling on my machine again. With the updated code and instructions I'm able to run
    cmake .. \
    -DobsLibPath=/Applications/OBS.app/Contents/Frameworks \
    -DobsIncludePath="../../obs-studio/libobs" \
    -DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" \
    -DLIBOBS_LIB="../../obs-studio/build/libobs/libobs.dylib"
    

    but I DO have to specify DLIBOBS... attributes which are absent in the README. Not sure what I'm missing in my setup.

    • Running cmake --build . fails with :
    cmake --build .
    Scanning dependencies of target obs-backgroundremoval
    [ 33%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
    [ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
    make[2]: *** No rule to make target `<path>/obs-studio/build/libobs/libobs.dylib', needed by `obs-backgroundremoval.so'.  Stop.
    make[1]: *** [CMakeFiles/obs-backgroundremoval.dir/all] Error 2
    make: *** [all] Error 2
    
    • I looked online and saw that other plugins are compiled using make -j4. I executed that and it worked(?):
    Scanning dependencies of target obs-backgroundremoval
    [ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
    [ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
    [100%] Linking CXX shared module obs-backgroundremoval.so
    [100%] Built target obs-backgroundremoval
    

    However after I copy the plugin, OBS fails to start:

    Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
    Exception Codes:       0x0000000000000001, 0x0000000000000000
    Exception Note:        EXC_CORPSE_NOTIFY
    
    Termination Signal:    Illegal instruction: 4
    Termination Reason:    Namespace SIGNAL, Code 0x4
    Terminating Process:   exc handler [19910]
    
    Application Specific Information:
    detected buffer overflow
    
    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0   libsystem_c.dylib             	0x00007fff67a3dfca __chk_fail_overflow.cold.1 + 16
    1   libsystem_c.dylib             	0x00007fff67a3b214 __chk_fail_overflow + 9
    2   libsystem_c.dylib             	0x00007fff67a3b848 __memcpy_chk + 18
    3   libobs.0.dylib                	0x0000000110daf06b obs_register_source_s + 299
    4   obs-backgroundremoval.so      	0x0000000028444925 obs_module_load + 21
    5   libobs.0.dylib                	0x0000000110dade0e obs_init_module + 78
    6   libobs.0.dylib                	0x0000000110daec80 load_all_callback + 64
    7   libobs.0.dylib                	0x0000000110daeb7e obs_find_modules + 2046
    8   libobs.0.dylib                	0x0000000110dae363 obs_load_all_modules + 35
    9   com.obsproject.obs-studio     	0x000000010e0d47ce OBSBasic::OBSInit() + 734
    10  com.obsproject.obs-studio     	0x000000010e0b0cd9 OBSApp::OBSInit() + 681
    11  com.obsproject.obs-studio     	0x000000010e0b4d28 main + 5032
    12  libdyld.dylib                 	0x00007fff6796ecc9 start + 1
    

    And that's not ideal. Please advise.

    opened by mctrafik 10
  • Performance Idea - option to reprocess mask every X frames

    Performance Idea - option to reprocess mask every X frames

    I've had to opportunity to use this plugin now on several occasions to feed the virtual camera output into webex meetings, and it's been working well, but like many others I find that cpu usage is a bottleneck that effects performance. Tweaking and/or trying out different models could improve that. But another thing I was thinking of, and I'm not sure how feasible it is, but maybe an option could be created to reprocess the mask every X frames?

    I'm guessing the work of the models reprocessing what the mask should be is the big hitter on cpu. Right now I can run about 15fps max against my cpu's (with some frame droppage), which isn't the smoothest experience as far as watching lips move on a web meeting, etc, But even rendering at 5fps the mask seems to hold up well. So I was thinking, what if for example there was an option to only reprocess the mask every X frames. Then, if you wanted the existing experience, you could set it to1, but it might be possible in my case to set it to 5 and be able to render 30fps video while in the background the models are only actually having to process 6fps. Again, I haven't reviewed the code to get an idea of how feasible it would be to implement, just throwing another idea out there on a way to maybe improve performance.

    opened by iBenMoore 8
  • crash after adding it to the cam

    crash after adding it to the cam

    I get this:

    info: [obs-backgroundremoval] Destroy scalers.
    info: [obs-backgroundremoval] Model location (null)
    error: [obs-backgroundremoval] Unable to get model filename from plugin.
    info: User added filter 'Background Removal' (background_removal) to source 'cam1'
    info: [obs-backgroundremoval] Destroy scalers.
    info: [obs-backgroundremoval] Initialize scalers. Size 640 x 480
    terminate called after throwing an instance of 'std::out_of_range'
      what():  vector::_M_range_check: __n (which is 3) >= this->size() (which is 0)
    [1]    34840 abort (core dumped)  obs
    

    Any ideas? seems it cant find the models? what can I do?

    opened by r3k2 8
  • What about an alpha channel?

    What about an alpha channel?

    Did you test out adding transparency with an alpha channel instead of setting the color?

    There is an Alpha YUV format. So it seems like it wouldn't be too hard to add a parameter for transparency, convert to an AYUV, apply the mask, then convert back to the proper output format.

    Thoughts? I can fork and give it a shot, but wanted to see if you had tried going direct to transparent already.

    opened by troypesola 8
  • Incomplete build description and missing dependencies

    Incomplete build description and missing dependencies

    Hi Roy,

    first of all, thank you so much for delving into the merits of developing a so much anticipated and much needed plugin for the awesome OBS.

    However, with regards to the build instructions I would like to be so bold as to mention several issues which I experienced during my struggle to get an error free build leaving me with frustration and finally giving up on the endeavour.

    First of all, I had an issue in which Homebrew would not install correctly on the version of macOS Catalina which I’m running. However, I made it to the point that I could install the mentioned prerequisites.

    Secondly, and this is actually missing in you description, cmake is not part of the Xcode development environment which I’m using, but has to be brewed separately. Using the Homebrew package of cmake results in a SIGILL error which I didn’t try to track down. Instead I used the CMake Mac application which I downloaded from cmake.org.

    Last, but not least, using the CMake application to generate the make files I encountered another error which pinpointed me to the necessity to have the OBS source code available to build the libobs module. However, trying to build this module again left me with errors which I didn’t spend time to investigate further upon.

    All in all the prerequisites for building your background removal module seems to require refinement and also seems to be more complex for green field build environments.

    Therefore I’d like to ask you if it would be possible for you to make available a binary build of the module together with the model data in a compressed file which users could download and install more easily. I’d be happy to support you in your efforts by making a donation via your preferred platform.

    Kind regards, all the best, stay safe and healthy, and greetings from Switzerland

    Mati

    opened by MatiMax 8
  • Linux GPU Usage

    Linux GPU Usage

    This commit patches for using the GPU on Linux as detailed in https://github.com/royshil/obs-backgroundremoval/issues/56

    Additionally it expands on the installation instructions for Ubuntu.

    opened by mproffitt 7
  • Compatibility issue with the obs-studio ppa

    Compatibility issue with the obs-studio ppa

    The default instructions did not work for me on ubuntu 20.04 with obs studio ppa because obs did not detect the plugin installation. Upon investigation it turns out that the obs ppa is utilizing the /lib/obs-plugins which is not in the CMakeLists.txt. So I wrote a patch for it:

    --- CMakeLists.txt      2021-09-07 14:43:11.033428381 +0200
    +++ CMakeLists-patched.txt      2021-09-07 14:44:30.217512505 +0200
    @@ -239,13 +239,13 @@
            file(GLOB onnxmodels data/*.onnx)
     
            install(TARGETS ${CMAKE_PROJECT_NAME}
    -               LIBRARY DESTINATION "${CMAKE_INSTALL_FULL_LIBDIR}/obs-plugins")
    +               LIBRARY DESTINATION "/lib/obs-plugins")
     
            install(FILES ${locale_files}
    -               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
    +               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
     
            install(FILES ${onnxmodels}
    -               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/")
    +               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/")
     endif()
     # --- End of section ---
    
    

    And now obs studio successfully display the filter, however trying to add the filter it breaks with a segfault and this is what I see in the core dump:

    StacktraceAddressSignature: /usr/bin/obs:11:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28+ad6b0:/usr/lib/obs-plugins/obs->
    StacktraceTop:
     __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:314
     std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator=(std::__cxx11::basic_string<ch>
     filter_update(void*, obs_data*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
     filter_create(obs_data*, obs_source*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
     ?? () from /lib/libobs.so.0
    

    Versions are exact match (27.0.1 atm). The core dump has a path in it which is /usr//lib/obs-plugins and that does not look right to me. Ideas are welcomed.

    PS: The default debian sources libobs0 and obs-studio did not work either because the filter is not detected at all so I'm sure it's using some other plugins directory like the ppa does. Either case I need the 26+ (deb one is 25) of obs due to its virtual camera (that plugin is a pain to maintain manually and breaks frequently).

    opened by chickencoding123 5
  • Troubleshooting with plugin Background Removal

    Troubleshooting with plugin Background Removal

    Hello, With my MAC I had a problem with the plugin Background Removal. I have installed OBS v. 28.1.2 and after I put the plugin's files in /Applications/OBS.app/Contents/. When I use a filter on the video capture I don't see the specific option in the application.Why, OBS does not take into consideration the plugin download. Thanks for your help.

    opened by webmasternic 2
  • Crashing consistently without log

    Crashing consistently without log

    System

    • Mac 12.6.1
    • OBS 28.1.2
    • OBS Background Removal 0.4.0

    Problem

    Removing the background and adding a color or chroma key to remove the background causes OBS to crash. I does not output any logs when it crashes

    Click here to view logs
    OBS 28.1.2 (mac) log file uploaded at 2022-12-15, 08:33:47
    
    08:32:43.079: [macOS] Permission for audio device access granted.
    08:32:43.089: [macOS] Permission for video device access granted.
    08:32:43.110: [macOS] Permission for accessibility denied.
    08:32:43.112: [macOS] Permission for screen capture denied.
    08:32:43.112: Rosetta translation used: false
    08:32:43.112: CPU Name: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    08:32:43.112: CPU Speed: 2600MHz
    08:32:43.112: Physical Cores: 6, Logical Cores: 12
    08:32:43.112: Physical Memory: 16384MB Total
    08:32:43.112: OS Name: macOS
    08:32:43.112: OS Version: Version 12.6.1 (Build 21G217)
    08:32:43.112: Kernel Version: 21.6.0
    08:32:43.114: hotkeys-cocoa: Using layout 'com.apple.keylayout.US'
    08:32:43.115: Current Date/Time: 2022-12-15, 08:32:43
    08:32:43.115: Browser Hardware Acceleration: true
    08:32:43.115: Qt Version: 6.3.1 (runtime), 6.3.1 (compiled)
    08:32:43.115: Portable mode: false
    08:32:43.380: OBS 28.1.2 (mac)
    08:32:43.380: ---------------------------------
    08:32:43.382: ---------------------------------
    08:32:43.382: audio settings reset:
    08:32:43.382: 	samples per sec: 48000
    08:32:43.382: 	speakers:        2
    08:32:43.382: 	max buffering:   960 milliseconds
    08:32:43.382: 	buffering type:  dynamically increasing
    08:32:43.388: ---------------------------------
    08:32:43.388: Initializing OpenGL...
    08:32:44.698: Loading up OpenGL on adapter ATI Technologies Inc. AMD Radeon Pro 5300M OpenGL Engine
    08:32:44.698: OpenGL loaded successfully, version 4.1 ATI-4.8.101, shading language 4.10
    08:32:45.141: ---------------------------------
    08:32:45.141: video settings reset:
    08:32:45.141: 	base resolution:   1920x1080
    08:32:45.141: 	output resolution: 1920x1080
    08:32:45.141: 	downscale filter:  Bicubic
    08:32:45.141: 	fps:               30/1
    08:32:45.141: 	format:            NV12
    08:32:45.141: 	YUV mode:          Rec. 709/Partial
    08:32:45.141: NV12 texture support not available
    08:32:45.141: P010 texture support not available
    08:32:45.142: Audio monitoring device:
    08:32:45.142: 	name: Default
    08:32:45.142: 	id: default
    08:32:45.142: ---------------------------------
    08:32:45.167: No AJA devices found, skipping loading AJA UI plugin
    08:32:45.167: Failed to initialize module 'aja-output-ui'
    08:32:45.187: No AJA devices found, skipping loading AJA plugin
    08:32:45.187: Failed to initialize module 'aja'
    08:32:45.192: Failed to load 'en-US' text for module: 'decklink-captions'
    08:32:45.197: Failed to load 'en-US' text for module: 'decklink-output-ui'
    08:32:45.199: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
    08:32:45.199: Failed to initialize module 'decklink'
    08:32:45.674: [VideoToolbox encoder]: Adding VideoToolbox encoders
    08:32:45.836: [obs-browser]: Version 2.18.7
    08:32:45.836: [obs-browser]: CEF Version 103.0.5060.134 (runtime), 103.61.23+g2fe5a76+chromium-103.0.5060.134 (compiled)
    08:32:45.888: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.0.1 | RPC Version: 1)
    08:32:45.888: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.3.1 | Qt version (run-time): 6.3.1
    08:32:45.888: [obs-websocket] [obs_module_load] Linked ASIO Version: 101201
    08:32:45.898: [obs-websocket] [obs_module_load] Module loaded.
    08:32:45.915: os_dlopen(/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib->/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib): dlopen(/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib, 0x0101): tried: '/Applications/VLC.app/Contents/MacOS/lib/libvlccore.dylib' (no such file)
    08:32:45.915: 
    08:32:45.915: [vlc-video]: Couldn't find VLC installation, VLC video source disabled
    08:32:45.945: Failed to load 'en-US' text for module: 'obs-backgroundremoval.so'
    08:32:45.945: [obs-backgroundremoval] plugin loaded successfully (version 0.3.0)
    08:32:45.945: ---------------------------------
    08:32:45.945:   Loaded Modules:
    08:32:45.945:     obs-backgroundremoval.so
    08:32:45.945:     vlc-video
    08:32:45.945:     text-freetype2
    08:32:45.945:     rtmp-services
    08:32:45.945:     obs-x264
    08:32:45.945:     obs-websocket
    08:32:45.945:     obs-vst
    08:32:45.945:     obs-transitions
    08:32:45.945:     obs-outputs
    08:32:45.945:     obs-filters
    08:32:45.945:     obs-ffmpeg
    08:32:45.945:     obs-browser
    08:32:45.945:     mac-virtualcam
    08:32:45.945:     mac-videotoolbox
    08:32:45.945:     mac-syphon
    08:32:45.945:     mac-capture
    08:32:45.945:     mac-avcapture
    08:32:45.945:     image-source
    08:32:45.945:     frontend-tools
    08:32:45.945:     decklink-output-ui
    08:32:45.945:     decklink-captions
    08:32:45.945:     coreaudio-encoder
    08:32:45.945: ---------------------------------
    08:32:45.946: ==== Startup complete ===============================================
    08:32:46.031: All scene data cleared
    08:32:46.031: ------------------------------------------------
    08:32:46.115: coreaudio: Device 'Microsoft Teams Audio' [48000 Hz] initialized
    08:32:46.115: [Loaded global audio device]: 'Desktop Audio'
    08:32:46.133: coreaudio: Device 'MacBook Pro Microphone' [48000 Hz] initialized
    08:32:46.133: [Loaded global audio device]: 'Mic/Aux'
    08:32:46.135: adding 21 milliseconds of audio buffering, total audio buffering is now 21 milliseconds (source: Desktop Audio)
    08:32:46.135: 
    08:32:46.153: Video Capture Device: Selected device 'FaceTime HD Camera (Built-in)'
    08:32:46.156: Video Capture Device: Using preset 1280x720
    08:32:46.243: [obs-backgroundremoval] Destroy scalers.
    08:32:46.310: [obs-backgroundremoval] Model mediapipe.onnx input 0: name input_1:0 shape (4 dim) 1 x 96 x 160 x 3
    08:32:46.310: [obs-backgroundremoval] Model mediapipe.onnx output 0: name segment:0 shape (4 dim) 1 x 96 x 160 x 2
    08:32:46.310: [obs-backgroundremoval] Allocated 46080 sized float-array for input 0
    08:32:46.311: [obs-backgroundremoval] Allocated 30720 sized float-array for output 0
    08:32:46.477: Switched to scene 'Scene'
    08:32:46.481: ------------------------------------------------
    08:32:46.481: Loaded scenes:
    08:32:46.481: - scene 'Scene':
    08:32:46.489:     - source: 'Image' (image_source)
    08:32:46.489:     - source: 'Video Capture Device' (av_capture_input_v2)
    08:32:46.489:         - filter: 'Background Removal' (background_removal)
    08:32:46.489:         - filter: 'Color Key' (color_key_filter_v2)
    08:32:46.489: ------------------------------------------------
    08:32:47.880: [obs-backgroundremoval] Destroy scalers.
    08:32:47.880: [obs-backgroundremoval] Initialize scalers. Size 1280 x 720
    
    opened by mbruno-kr 0
  • GPU support on macOS by Vision.framework

    GPU support on macOS by Vision.framework

    Apple made Vision.framework and it uses GPU so that the background removal by Vision.framework on macOS is super efficient.

    I implemented the code with Vision.framework and thus you can use Apple GPU for the background removal now.

    You can install the latest binary from here https://github.com/umireon/obs-backgroundremoval/releases/tag/v0.4.4-umireon

    スクリーンショット 2022-11-30 12 27 09

    The inference device shows CPU but it actually uses GPU when Vision (Apple) was selected.

    opened by umireon 0
  • Support for flips and rotations from camera config

    Support for flips and rotations from camera config

    I changed config of my camera by applying flips (vertical, horizontal, clockwise and counterclockwise) both from camera config itself and from OBS filters, but it plugin failed to respond to those changes. E.g. the torso of my body shown in the opposite direction.

    Can you please add support to account for those geometric transformations?

    opened by southaustralia 0
  • ‘OrtSessionOptionsAppendExecutionProvider_DML’ was not declared in this scope

    ‘OrtSessionOptionsAppendExecutionProvider_DML’ was not declared in this scope

    In Ubuntu 22.04 when I try to cmake --build . following the installation instructions I get this error:

    error: ‘OrtSessionOptionsAppendExecutionProvider_DML’ was not declared in this scope; did you mean ‘OrtSessionOptionsAppendExecutionProvider_CPU’? 215 | Ort::ThrowOnError(OrtSessionOptionsAppendExecutionProvider_DML(sessionOptions, 0)); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | OrtSessionOptionsAppendExecutionProvider_CPU gmake[2]: *** [CMakeFiles/obs-backgroundremoval.dir/build.make:76: CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/obs-backgroundremoval.dir/all] Error 2 gmake: *** [Makefile:156: all] Error 2

    Some Idea to solve it? Thanks

    opened by LuisGFer 2
Releases(v0.4.0)
  • v0.4.0(Nov 3, 2021)

    In this release

    • Upgrade to OBS v27.1.3 (latest) support
    • New realtime model Robust Video Matting (https://github.com/PeterL1n/RobustVideoMatting)
    • Calculate mask every X frame - save on CPU cycles! (set to 2 for a 50% performance gain with unnoticeable quality drop)
    • Some internal abstraction of the models to make it easy to add more models in the future
    • More advancement on CUDA - but still no stable version... need help on this plz.

    I'm taking the releases off of "Pre-Release" since it seems 10,000s of people were able to successfully use the plugin.

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    • Archlinux: install the package e.g. pacman -U obs-backgroundremoval*.tar.zst

    What's Changed

    • (rough) use CUDA by @packysauce in https://github.com/royshil/obs-backgroundremoval/pull/29
    • Process Mask every X frames of video by @royshil in https://github.com/royshil/obs-backgroundremoval/pull/31
    • Add "obs" path suffix to allow compilation with OBS Studio 27.1.3 (ppa) by @fubar-coder in https://github.com/royshil/obs-backgroundremoval/pull/45

    New Contributors

    • @packysauce made their first contribution in https://github.com/royshil/obs-backgroundremoval/pull/29
    • @royshil made their first contribution in https://github.com/royshil/obs-backgroundremoval/pull/31
    • @fubar-coder made their first contribution in https://github.com/royshil/obs-backgroundremoval/pull/45

    Full Changelog: https://github.com/royshil/obs-backgroundremoval/compare/v0.3.0-beta...v0.4.0

    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.4.0-win64.zip(49.85 MB)
    obs-backgroundremoval-macosx.zip(55.39 MB)
  • v0.3.0-beta(Jun 17, 2021)

    In this release

    • Added 2 new lightweight segmentation models: Google Mediapipe "Meet" model, and the "Selfie Segmentation" model, both from https://github.com/PINTO0309/PINTO_model_zoo
    • Upgrade to OBS v27.0.1 (following OBS production releases on OSX and Windows)
    • Upgrade onnxruntime on OSX to v1.7.2
    • Fix issue with RGB vs BGR background color flip

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    • Archlinux: install the package e.g. pacman -U obs-backgroundremoval*.tar.zst
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.3.0_beta-1-x86_64.pkg.tar.zst(30.72 MB)
    obs-backgroundremoval-macosx.zip(42.08 MB)
    obs-backgroundremoval-win64.zip(59.74 MB)
  • v0.2.5-beta(May 25, 2021)

    In this release

    • Archlinux build and package

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    • Archlinux: install the package e.g. pacman -U obs-backgroundremoval*.tar.zst
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.2.5_beta-1-x86_64.pkg.tar.zst(30.03 MB)
    obs-backgroundremoval-macosx.zip(41.10 MB)
    obs-backgroundremoval-win64.zip(59.01 MB)
  • v0.2.4-beta(May 11, 2021)

    In this release

    • Adding the MODNet segmentation model. Slightly better in segmentation and resources.
    • Dropdown to choose model (SINet or MODNet)

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(41.10 MB)
    obs-backgroundremoval-win64.zip(59.01 MB)
  • v0.2.3-beta(May 7, 2021)

    In this release

    • Optimized ONNX model with slightly better runtime/resources performance (5-10%)
    • Fix to DirectML build (including the DLL)

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio), for example: > Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(18.60 MB)
    obs-backgroundremoval-win64.zip(13.31 MB)
  • v0.2.2-beta(May 4, 2021)

    In this release

    • Better versioning of the plugin
    • GPU support on Windows through DirectML (any available GPU)

    Install:

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio)
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(18.27 MB)
    obs-backgroundremoval-win64.zip(12.99 MB)
  • v0.2.1-beta(May 3, 2021)

    In this release

    • Fixing the problem of down&up scaling the original image. Instead - apply the mask to the original image size, without reducing its resolution.

    Install

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio)
    • Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/ -Linux: TBD
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-macosx.zip(18.27 MB)
    obs-backgroundremoval-win64.zip(4.64 MB)
  • 0.2-beta(May 2, 2021)

    This release features

    • Omni color space handling
    • Slightly better performance

    Install

    • Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio)
    • Mac OSX: Copy obs-backgroundremoval.so to /Applications/OBS.app/Contents/PlugIns. Copy SInet_Softmax.onnx to /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval -Linux: TBD
    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval-0.2beta-1-x86_64.pkg.tar.zst(3.84 KB)
    obs-backgroundremoval-win64.zip(4.34 MB)
    obs-backgroundremoval.so(39.69 MB)
    SINet_Softmax.onnx(428.53 KB)
  • 0.1-beta(Apr 26, 2021)

    In this release

    • Windows support
    • Linux (Ubuntu) support
    • More streamlined build
    • Colorspaces conversions

    To install

    • Windows: Copy .dlls to e.g. C:\Program Files\obs-studio\obs-plugins\64bit, and .onnx file to e.g. C:\Program Files\obs-studio\data\obs-plugins\obs-backgroundremoval
    • MacOSX: Copy .so file to /Applications/OBS.app/Contents/PlugIns and .onnx file to /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval

    Linux install TBD

    Source code(tar.gz)
    Source code(zip)
    obs-backgroundremoval.dll(3.76 MB)
    obs-backgroundremoval.so(170.69 KB)
    onnxruntime.dll(7.62 MB)
    SINet_Softmax.onnx(428.53 KB)
  • 0.1-alpha(Apr 23, 2021)

    This is an alpha pre-release just to provide the pre-built binaries, to avoid having to build them. The plugin binary and model are supplied and they should be copied to the OBS plugins directory manually, e.g.

    $ cp obs-backgroundremoval.so /Applications/OBS.app/Contents/PlugIns
    $ cp SINet_Softmax.onnx /Applications/OBS.app/Contents/Resources/data/obs-plugins/obs-backgroundremoval/
    

    OpenCV and ONNXRuntime are still needed for the plugin to run. Use homebrew to install them:

    $ brew install opencv onnxruntime
    

    In case homebrew doesn't install v1.7.2 of onnxruntime - i've included the dylib.

    Source code(tar.gz)
    Source code(zip)
    libonnxruntime.1.7.2.dylib(12.01 MB)
    obs-backgroundremoval.so(218.32 KB)
    SINet_Softmax.onnx(428.53 KB)
Owner
Roy Shilkrot
Roy Shilkrot
An OBS plugin for Zoom/Meet-like virtual background feature.

OBS Virtual background plugin English / Japanese OBS Virtual background plugin is a plugin for OBS. This plugin allows you to crop the background area

KOUNOIKE Yuusuke 186 Dec 25, 2022
"Zero-copy" Linux screen capture plugin for OBS that uses libdrm and dmabuf

linux-kmsgrab plugin for OBS Introduction This plugin is a proof-of-concept libdrm-based screen capture for OBS. It uses DMA-BUF to import CRTC frameb

Ivan Avdeev 62 Dec 31, 2022
OBS Plugin to capture CS:GO with Trusted Mode enabled

OBS Plugin to capture CS:GO with Trusted Mode enabled Ever since Valve introduced Trusted Mode you could no longer capture Counter-Strike: Global Offe

Gregor Steiner 105 Nov 17, 2022
OBS Plugin with image that reacts to sound source.

OBS Image Reaction Plugin Image that reacts to sound source. Installing binaries Download binaries from Releases. For Windows, Move the contents of pl

null 39 Jan 4, 2023
audio monitor filter for OBS Studio

Audio Monitor dock and filter for OBS Studio Plugin for OBS Studio to add Audio Monitor dock and filter. It allows you to put the audio of a OBS sourc

Exeldro 222 Dec 18, 2022
Software for making digital voice work without the smart hub in the way

Software for making digital voice work without the smart hub in the way

Charlie 3 Nov 7, 2022
Block all ads in vídeo, áudio, banner, anti-skip

NoAdSpotify Block spotify ad This is an updated and simplified version of the project: BlockTheSpot Last updated: 6th June 2021 Last tested version: 1

null 14 Nov 5, 2022
Example for transmit video + audio to tv via hackRF

Этот код может передавать звук и изображение на телевизор через hackRF В этом проекте использовался код из следующих репозиториев: https://github.com/

null 9 Sep 27, 2022
Single file C library for decoding MPEG1 Video and MP2 Audio

PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer Single-file MIT licensed library for C/C++ See pl_mpeg.h for the documentation. Why?

Dominic Szablewski 605 Dec 27, 2022
cute_dsp is a C API for various DSP effects suitable for video games

cute_dsp is a C API for various DSP effects suitable for video games and meant to interface directly with the cute_sound library created by Randy Gaul

Matt Rosen 14 Jan 2, 2023
PeakEater is a free open-source VST3/AU waveshaper plugin

PeakEater Free open source VST3/AU wave shaping plugin for macOS and Windows. View Demo · Download · Report Bug Table of Contents About The Project Fe

Vladyslav Voinov 45 Dec 17, 2022
BYOD is a guitar distortion plugin with a customisable signal chain that allows users to create their own guitar distortion effects.

BYOD is a guitar distortion plugin with a customisable signal chain that allows users to create their own guitar distortion effects. The plugin contains a wide variety of distortion effects from analog modelled circuits to purely digital creations, along with some musical tone-shaping filters, and a handful of other useful processing blocks.

null 229 Jan 5, 2023
ChowKick is a kick drum synthesizer plugin based on creative modelling of old-school drum machine circuits

ChowKick is a kick drum synthesizer plugin based on creative modelling of old-school drum machine circuits. MIDI input to the plugin triggers a pulse with a parameterized size and shape. The pulse is then passed into a resonant filter which can be tuned to a specific frequency, or matched to the frequency of the incoming MIDI notes.

null 117 Dec 16, 2022
A weird distortion and lowpass plugin

Delta Limiter Building ./waf clean configure build Documentation pandoc -s user_manual.md -o user_manual.html pandoc -s user_manual.md -o user_manual.

Hannes Braun 3 Sep 19, 2022
SuperCollider as a VST3 plugin

PluginCollider PluginCollider is an experimental fork of https://github.com/supercollider/SuperColliderAU and it is still not complete since it only b

Pascal Gauthier 33 Dec 23, 2022
Wolf MidiCurve is a MIDI remapping plugin.

Wolf MidiCurve is a MIDI remapping plugin. It can be built as an LV2 or VST plugin and as a standalone Jack application.

Wolf Plugins 8 Oct 21, 2022
Audio Plugin API

Audio Plugin API

null 1.3k Jan 3, 2023
Easy and efficient audio synthesis in C++

Tonic Fast and easy audio synthesis in C++. Prefer coding to patching? Love clean syntax? Care about performance? That's how we feel too, and why we m

null 482 Dec 26, 2022
Audacity is an easy-to-use, multi-track audio editor and recorder for Windows, Mac OS X, GNU/Linux and other operating systems

Audacity is an easy-to-use, multi-track audio editor and recorder for Windows, Mac OS X, GNU/Linux and other operating systems. Audacity is open source software licensed under GPL, version 2 or later.

Audacity 8.7k Dec 31, 2022