Command line C++ and Python VSTi Host library with MFCC, FFT, RMS and audio extraction and .wav writing.

Overview
       ______               _          ___  ___
       | ___ \             | |         |  \/  |
       | |_/ /___ _ __   __| | ___ _ __| .  . | __ _ _ __
       |    // _ \ '_ \ / _` |/ _ \ '__| |\/| |/ _` | '_ \
       | |\ \  __/ | | | (_| |  __/ |  | |  | | (_| | | | |
       \_| \_\___|_| |_|\__,_|\___|_|  \_|  |_/\__,_|_| |_|

* *  Command Line VSTi Audio, Features and Parameter Renderer  * *
build
Build Status

RenderMan

Renderman is a command line VSTi host written in C++ with Python bindings using JUCE and Maximilian libraries for the backend. It is designed with ease of use in mind to extract audio and features from VSTi plugins. It has a fast growing list of features, including setting, getting parameters from synthesiers, setting whole patches, getting random patches, obtaining MFCCS, FFT, audio data and much more.

A usage example in the form of an IPython notebook can be found here.

Here are some quick gifs demonstrating a miniscule amount of the availble features, go towards the bottom of the README to see the full API:

Loading Plugins

Here we load a plugin. On Linux it's an .so file, for MacOS it will be a .vst or .au.

Getting Parameter Descriptions

We can obtain the available parameters that are used and that can be modified by doing the following.

Getting a Randomised Patch For a Synthesiser

We can easily get a randomised for a given synth by using the PatchGenerator class.

Plotting Rendered Audio Frames

We can plot the output audio frames easily by using matplotlib.

Building / Installation

MacOS

If you haven't already, get brew. The last time I checked the command to install was simply this:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Next, get the boost headers.

brew install boost-python3

You can also install boost manually if for some reason you don't want to use brew, see here.

Now just open the Xcode project in the Builds directory and build it! There is a bug in the JUCE projucer app which means the generated shared object will be suffixed with a dylib. This means python wont be able to import the module. Until this bug is fixed, change directory into the Builds/MacOSX/build/<Debug/Release> (depending on your Xcode scheme,) and run:

mv librenderman.so.dylib librenderman.so

IMPORTANT: The project is linked with libpython3.8 and libboost_python38 and the appropriate include directories. If you have a different version of python installed, run python3-config --includes --ldflags to obtain the library and include paths, and update the XCode project to point to the correct locations

Linux

Firstly, you will need the boost library (specifically the python headers) for this code to compile.

Ubuntu:

sudo apt-get install libboost-all-dev

Arch:

sudo pacman -Ss boost

Fedora:

sudo yum install boost-devel

If your distribution's package manager doesn't have boost, you can get the headers from here.

Juce itself has a list of dependancies for Linux; it's a very big library - if you don't know it you should definitely take some time out to check it out! Depending on your distribution and setup you may already have some / all of the following libraries. If you are on Ubuntu, the following commands will install your dependancies. Find the respective packages for other distros using google please!

sudo apt-get -y install llvm
sudo apt-get -y install clang
sudo apt-get -y install libfreetype6-dev
sudo apt-get -y install libx11-dev
sudo apt-get -y install libxinerama-dev
sudo apt-get -y install libxrandr-dev
sudo apt-get -y install libxcursor-dev
sudo apt-get -y install mesa-common-dev
sudo apt-get -y install libasound2-dev
sudo apt-get -y install freeglut3-dev
sudo apt-get -y install libxcomposite-dev
sudo apt-get -y install libcurl4-gnutls-dev

Well done! You've made it this far! Should you still have problems, which is always a possibility with Linux, a good place to start is the JUCE forums, particularly here and here. Feel free to drop me a note with an error and I'll happily scratch my head over it but you may get better results in the forums!

So to now build the library for Linux, change to the right directory and run make:

cd Builds/LinuxMakefile/
make

Windows - VisualStudio2019

Download and Install boost

Download from https://www.boost.org/users/download/ extract to c:\boost_1_74_0

cd c:\boost_1_74_0
bootstrap.bat
.\b2 --toolset=msvc-14.0 --build-type=complete --prefix=C:\Boost install

Download and Install python 3.7.9

Download from https://www.python.org/downloads/release/python-379/

Open RenderMan.sln in VisualStudios2019

When prompt to retarget projects

select Windows SDK Version: 8.1 platform toolset: No Upgrade

add Includes to search path Project > Properties > C/C++ > General

Additional Include Directories

C:\Boost\include\boost-1_74
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\include

add Libs to search path

Project > Properties > Configuration Properties > VC++ Directories

Library Directories

add

C:\Boost\lib
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\libs

build

after build rename

RenderMan\Builds\VisualStudio2019\x64\Debug\Dynamic Library\renderman.dll

to

librenderman.pyd

*put the pyd file in the root of the python program

Does It Work?

Change directory to where the .so file is and run:

python

Once in interactive mode, run:

import librenderman as rm

If this doesn't spit out errors, congratulations! Enjoy RenderMan. :)

Note you'll need to have the .so binary in the same directory as the Python project or where you call the interative Python shell from. To use it system wide it will need to be added to the PYTHONPATH environment variable. Soon I'll get distutils sorted so there is an easier installation method.

API

class RenderEngine

The constructor takes three arguments, the sample rate, the buffer size and fft size. Good defualt values if you don't really care that much are 44100, 512, 512 repsectively.

__init__(int sample_rate,
         int buffer_size,
         int fft_size)

Supply a full path to the plugin to this function to load the vst. It will return true if there is a successful loading.

bool load_plugin(string plugin_path)

We can set a synth's patch by taking a list of tuples and set the parameters at the int index to the float value. The PatchGenerator class can generate random patches with ease for a given synth.

void set_patch(list_of_tuples(int, float) patch)

Get the current patch.

list_of_tuples(int, float) get_patch()

Take a midi note (middle C is 40,) a velocity (0 - 127,) and the note length and recording / rendering length and create the features to be extracted!

void render_patch(int   midi_note_pitch,
                  int   midi_note_velocity,
                  float note_length_seconds,
                  float render_length_seconds)

Get MFCC features as a list of lists. The first length will be dictated by fft size divided by four, and the second length with be 13, which is the amount of coefficients.

list_of_lists get_mfcc_frames()

Get the int amount of parameters for the loaded plugin.

int get_plugin_parameter_size()

Get a description of each parameter.

string get_plugin_parameters_description()

Override a parameter to always be the supplied value. The float is normalised (0 - 1).

override_plugin_parameter(int   index,
                          float value)

Remove an overriden plugin parameter.

remove_overriden_plugin_parameter(int index)

Get a list of floats which is the audio from the rendering session.

list get_audio_frames()

Write the current patch to a wav file at the specified relative or absolute path. This will overwrite existing files and is only a preview; it is mono and currently not quite loud enough.

void write_to_wav(string path)

Get a list of root mean squared frames derived from the audio samples. Each frame is a root mean squared of an amount of samples equal to the fft size divided by four.

list_of_floats get_rms_frames()
class PatchGenerator

This class is used to generate patches for a given engine.

The constructor takes an argument of a RenderEngine that has succesfully loaded a plugin.

__init__(RenderEngine engine)

We can obtain a random value (from a real uniform distribution) that for a parameter at a specified index.

tuple(int, float) get_random_parameter(int index)

We can get a completely random patch sampled from PRNG in the same way above.

list_of_tuples(int, float) get_random_patch()

Contributors

I want to express my deep gratitude to jgefele. It is very touching that people want to use this code let alone contribute to it - thanks!

Context and Contact

This library is a work in progress for my final year project where I am using Neural Networks to generate synth patches to create desired sounds. For example, I wrote a toy VSTi and learnt synthesiser using a neural network to sound match target sounds here. Everything this library has facilitates that but I recognise there may be more applications so if there are any feature requests please drop me a line on here or leonfedden (at) gmail.com :)

Finally, doing something cool with this library? Let me know what you are up to! And if this code was useful for you then please kindly drop me a GitHub star so more developers will trust and use this code.

If you use this code for academic works, feel free to reference this code through the DOI here: DOI

Thanks for reading this far, you rock!

Issues
  • ImportError: dynamic module does not define module export function (PyInit_librenderman)

    ImportError: dynamic module does not define module export function (PyInit_librenderman)

    Thanks for making this!

    I installed boost-python, successfully built the XCode project, renamed the built file to remove the ".dylib", but when I run the "Does It Work?" check, I get...

    $ python
    Python 3.5.3 | packaged by conda-forge | (default, Feb 10 2017, 07:09:50) 
    [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import librenderman as rm
    JUCE v5.2.0
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ImportError: dynamic module does not define module export function (PyInit_librenderman)
    >>>
    

    Any suggestions on how one might fix this?

    In the current directory I have...

    $ ls
    librenderman.so
    

    Running Mac OS X 10.13.3 (High Sierra), XCode 9.2, Python 3.5.2 (Anaconda)

    opened by drscotthawley 16
  • Allow effect plugins, update to Python 3, fix windows

    Allow effect plugins, update to Python 3, fix windows

    A bunch of changes:

    1. Added a function to accept input audio and feed it into the VST plugin
    2. Updated to use Python 3 instead of 2
    3. Contributed by Lior Hakim: Build instructions for Windows
    opened by shenberg 9
  • Windows Support with JUCE 5.4.7

    Windows Support with JUCE 5.4.7

    Addressing https://github.com/fedden/RenderMan/issues/12

    First of all, thanks for this amazing project. I was really happy to see your Medium post on deep learning synth presets a while back and am even happier to see IRCAM researchers use Renderman on their https://github.com/acids-ircam/flow_synthesizer.

    This commit adds Windows support. I wanted to avoid changing the version of JUCE, but somehow the JuceLibraryCode/modules kept getting overwritten, and I had to give in. Unfortunately 5.4.x introduced many issues:

    The return signature for pluginFormatManager.createPluginInstance seems to have changed from a simple pointer to a unique_ptr. These made me change the way RenderEngine gets deallocated. if (plugin) { plugin->releaseResources(); plugin.release(); } I also had to create boilerplate for copying RenderEngineWrapper. RenderEngineWrapper(const RenderEngineWrapper&) {}

    My Visual Studio 2019 projucer properties include a lot of "hard-coded" paths: C:/tools/boost_1_71_0 C:/tools/VST3_SDK C:/Python37 C:/Python37renderman

    Originally I tried using environment variables in the Projucer. My Windows environment variable for RENDERMAN_PYTHON_PATH might be C:\Python37 and so my header search paths would be $(RENDERMAN_PYTHON_PATH)\include but Visual Studio didn't seem to recognize those vars. So I gave up and just hard-coded the paths.

    But I can use Renderman on Windows!

    If you want to try this yourself but attempt NOT updating JUCE, here's my advice: Install C:/tools/boost_1_71_0, C:/Python37, and C:/Python37renderman as suggested in the updated README.

    Create a new property sheet for Visual Studio 2019 in the Projucer. Extra Linker Flags: -DBOOST_PYTHON_STATIC_LIB

    Header Search Paths: C:\tools\boost_1_71_0; C:\Python37\include

    Extra Library Search Paths: C:\tools\boost_1_71_0\lib64-msvc-14.2; C:\Python37\libs

    Post-build Command: md C:\\Python37renderman\\DLLs; move "x64\\Release\\Dynamic Library\\librenderman.dll" "C:\\Python37renderman\\DLLs\\librenderman.pyd"; copy /B C:\\tools\\boost_1_71_0\\lib64-msvc-14.2\\boost_python* "C:\\Python37renderman\\DLLs\\"

    C:\Python37renderman\Scripts\activate.bat, then, python and import librenderman as rm

    opened by DBraun 8
  • JUCE 5.2.0  VST SDK 3.6.7 (+Travis CI)

    JUCE 5.2.0 VST SDK 3.6.7 (+Travis CI)

    I had troubles building RenderMan the JUCE version doesn't work with VST SDK >= 3.6.6 and the Steinberg download page just links 3.6.8. I just found later in the JUCE forums some links to older SDKs, but the work had already been done.

    I thought it might be useful to include (a stripped down version of) the exact VST SDK version compatible with the JUCE version in use. So I included the VST SDK 3.6.7 (latest JUCE doesn't work with 3.6.8 yet) and upgraded to the latest JUCE version 5.2.0.

    opened by jgefele 3
  • Compatible with Python3.6 on MacOS.

    Compatible with Python3.6 on MacOS.

    Use RenderMan-py36.jucer to build and follow instructions outlined at https://github.com/fedden/RenderMan/issues/7#issuecomment-375697319.

    This is a bit of a hack, but it works for people who want to use MacOS and Python 3 for the project.

    I think in the future this solution could be changed by writing a script that automatically finds python and boost locations and edits the RenderMan.jucer XML file.

    opened by harritaylor 1
  • Problems building on Mac OS

    Problems building on Mac OS

    Hi, when I try to build Renderman on Mac OS Big Sur 11.6 from the xcode file, it does not work. I get the error "'base/source/baseiids.cpp' file not found". I installed boost and it worked properly. What software am I missing that is hindering the application from installing?

    opened by dafaronbi 0
  • Couldn't build on macOS

    Couldn't build on macOS

    Since I couldn't get RenderMan to build on Windows or Ubuntu, I finally tried it on macOS. I get the error, 'boost/python.hpp' file not found. I've tried it on two different versions of macOS (11 and 12) with two different versions of xcode. I did run brew install Python on both.

    opened by inhahe 9
  • Possible sound quality loss when using VST3 plugin

    Possible sound quality loss when using VST3 plugin

    I have tested a bunch of plugins on Renderman, and I found the sound is different with which is generated in REAPER when I use VST3 plugin.

    I am using Surge plugin https://github.com/surge-synthesizer/surge The preset is INIT.

    This is the spectrum of the audio exported by REAPER: image

    This is the spectrum of that generated by Renderman with the same plugin parameters: image

    it hears like the original audio go through a low-pass filter or something

    opened by Centauria 1
  • Couldn't build on Windows

    Couldn't build on Windows

    I already had c:\Boost, so I renamed it to c:\boost.2 and attempted to follow the instructions for building Boost. When running bootstrap.bat, I get an error:

    C:\boost_1_74_0>bootstrap
    Building Boost.Build engine
    
    Failed to build Boost.Build engine.
    Please consult bootstrap.log for further diagnostics.
    

    The contents of bootstrap.log are:

    LOCALAPPDATA=C:\Users\inhah\AppData\Local
    Found with vswhere C:\Program Files\Microsoft Visual Studio\2022\Preview
    ###
    ### "Unknown toolset: vcunk"
    ###
    ### You can specify the toolset as the argument, i.e.:
    ###     .\build.bat msvc
    ###
    ### Toolsets supported by this script are: borland, como, gcc,
    ###     gcc-nocygwin, intel-win32, metrowerks, mingw,
    ###     vc12, vc14, vc141, vc142
    ###
    ### If you have Visual Studio 2017 installed you will need to either update
    ### the Visual Studio 2017 installer or run from VS 2017 Command Prompt
    ### as we where unable to detect your toolset installation.
    ###
    

    So, I checked to see if the Boost I already had in c:\boost.2 was version 1.74, and it was, so I just renamed it to c:\Boost

    Then I followed the rest of the instructions I got 160 warnings and 6 errors. The 6 errors are:

    Error	C2872	'ssize_t': ambiguous symbol (compiling source file ..\..\Source\source.cpp)	RenderMan_DynamicLibrary	c:\program files (x86)\microsoft visual studio\shared\python37_64\include\pyport.h	84	
    Error	C2872	'ssize_t': ambiguous symbol (compiling source file ..\..\Source\PatchGenerator.cpp)	RenderMan_DynamicLibrary	c:\program files (x86)\microsoft visual studio\shared\python37_64\include\pyport.h	84	
    Error	C2872	'ssize_t': ambiguous symbol (compiling source file ..\..\Source\RenderEngine.cpp)	RenderMan_DynamicLibrary	c:\program files (x86)\microsoft visual studio\shared\python37_64\include\pyport.h	84	
    Error	C1189	#error:  "Mixing a dll boost library with a static runtime is a really bad idea..." (compiling source file ..\..\Source\PatchGenerator.cpp)	RenderMan_DynamicLibrary	C:\Boost\include\boost-1_74\boost\config\auto_link.hpp	434	
    Error	C1189	#error:  "Mixing a dll boost library with a static runtime is a really bad idea..." (compiling source file ..\..\Source\RenderEngine.cpp)	RenderMan_DynamicLibrary	C:\Boost\include\boost-1_74\boost\config\auto_link.hpp	434	
    Error	C1189	#error:  "Mixing a dll boost library with a static runtime is a really bad idea..." (compiling source file ..\..\Source\source.cpp)	RenderMan_DynamicLibrary	C:\Boost\include\boost-1_74\boost\config\auto_link.hpp	434	
    

    The full output with the warnings too is here: renderman errors.txt

    One thing that may be important is that the first time I tried running bootstrap.bat, it started working, but I aborted it because it said it couldn't find some tools, so I figured I should install visual studio 14 and then run its vcvarsall.bat and then run bootstrap.bat again. I never got it to work again though. I even deleted the whole c:\boost_1_74_0 and extracted it again and tried again, still wouldn't work.

    opened by inhahe 6
  • How to load VST plug-in for Windows?

    How to load VST plug-in for Windows?

    Hi, I have built the application on Windows using another Fork, but I have not figure out how to load a VST plug-in under Windows10. Usually, on Windows, a VST plug-in is a dll file. I tried e.g. the following code: plugin_path = 'C:/Users/lenovo/Desktop/AnacondaCodeEnv/music_synth_renderman/VST/Kontakt.dll' eng.load_plugin(str(plugin_path)) But it does not work, only crashes program.

    So I want to know, how to load a VST plug-in for Windows? Or even is this possible at the first place, because the application was not only built for OSX and Linux, but also only designed for these two platforms?

    Thanks!

    EDIT: Oh, I know partially the reason. If the VST plug-in is in VST3 format (.vst3 extension), then no problem to load. If a plug-in is in .dll extension, then it is a VST2 format. But I am still confused, because the MacOS built version has no problem to load "Kontakt Player 6", which is only available as VST2 plug-in (file Kontakt.component in MacOS).

    opened by ZhaoJY1 0
Releases(v1.0.0)
Owner
Leon Fedden
AI @mogees Site: https://leonfedden.com/ Blog: https://medium.com/@LeonFedden Twitter: https://twitter.com/LeonFedden LinkedIn: https://www.linkedin.com/i
Leon Fedden
Experiments using the RPI Zero GPU for FFT (1D and 2D)

RPI0_GPU_FFT Experiments using the RPI Zero GPU for FFT/IFFT 1D/2D For an input 4194304 (1D), the GPU was around 7X faster than np.fft.fft and np.fft.

Ricardo de Azambuja 27 Jun 20, 2022
Audio plugin host

Carla Plugin Host What is Carla? Carla is a fully-featured audio plugin host, with support for many audio drivers and plugin formats. It's open source

Filipe Coelho 1.1k Jun 26, 2022
a small C library for x86 CPU detection and feature extraction

libcpuid libcpuid provides CPU identification for the x86 (and x86_64). For details about the programming API, you might want to take a look at the pr

Veselin Georgiev 329 May 31, 2022
A small single-file library for sprite outline extraction and simplification for C/C++

Sproutline A small single-file library for sprite outline extraction and simplification for C/C++. Input: Sprite with an alpha channel. Output: All th

ands 75 Jun 19, 2022
MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files supporting textures, skeletons, and all provided vertex data.

MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files. A dynamic model is one that is animated or is spawned in during the game.

Montague 31 Jun 20, 2022
A Semi Automatic Chessboard Corner Extraction Class

This program realizes semi-automatic chessboard corner extraction, the interface is simple, and the accuracy of corner extraction is guaranteed.

null 1 Oct 6, 2021
DLL Exports Extraction BOF with optional NTFS transactions.

DLL Exports Extraction BOF What is this? This is a Cobalt Strike BOF file, meant to use two or three arguments (path to DLL, and/or a third argument [

null 32 May 30, 2022
Tools for working with Wwise file types (only extraction at the moment)

Wwise Audio Tools This repository is for a static and dynamic library as well as a simple command line tool used to convert Wwise WEM files to OGG fil

RED Modding tools 5 May 26, 2022
Tuibox - A single-header terminal UI (TUI) library, capable of creating mouse-driven, interactive applications on the command line.

tuibox tuibox ("toybox") is a single-header terminal UI library, capable of creating mouse-driven, interactive applications on the command line. It is

Andrew 8 May 31, 2022
Wave Function Collapse library in C, plus a command-line tool

wfc Single-file Wave Function Collapse library in C, plus a command-line tool License: MIT Version: 0.01 This is an early version that supports the ov

krychu 290 Jun 22, 2022
A simple DPDK application that calculates stats for dropped and forwarded packets depending on the command line.

The DPDK Stats Description A small DPDK application that increments a basic packet counter for TCP or UDP packets depending on what's specified in the

Christian Deacon 10 Jun 15, 2022
The command line app automatically determines your location using GeoIP and adjusts the color temperature depending on time

go-sct A color temperature setting library and CLI that operates in a similar way to f.lux and Redshift. The command line app automatically determines

Florine Sueur 1.1k Jan 25, 2022
Quick reference on command line tools and techniques

1. Introduction 1.1. Scope 1.2. Background 1.3. Purpose 1.4. Next steps 2. Basics 2.1. Common commands 2.2. Shortcuts 2.2.1. Navigation 2.2.2. Editing

Utsav Barman 323 Jun 3, 2022
OS X command line tool to inject Frameworks and dylibs on mach-o binaries (iOS & Mac Apps).

macho-inject OS X command line tool to inject Frameworks and dylibs on mach-o binaries. It does the injection of the framework and the codesigning. It

Jon Gabilondo 3 Feb 12, 2022
fx is a workspace tool manager. It allows you to create consistent, discoverable, language-neutral and developer friendly command line tools.

fx is a workspace tool manager. It allows you to create consistent, discoverable, language-neutral and developer friendly command line tools.

null 18 Jun 2, 2022
Lightweight C++ command line option parser

Release versions Note that master is generally a work in progress, and you probably want to use a tagged release version. Version 3 breaking changes I

null 3.1k Jul 3, 2022
The command line interface for Piccolo

Piccolo programming language A fun, easy to embed high-level programming language. This repo contains the code for the Piccolo CLI. The core Piccolo c

null 7 Feb 14, 2022
Simple command-line program for sharing the display image on a local network.

XCast is a simple server/client command line program for sharing screen under X. With XCast you have the ability to either pull the display from a rem

Vito Čuček 3 Dec 12, 2021
LKM Command Line Parsing - Parte 2

LKM-Command-Line-Parsing ?? Hoje iremos falar sobre Command Line Parsing diretamente no LKM. Esse é a segunda parte da nossa série de estudos para Roo

0xi386 6 May 11, 2022