Lightweight Embedded Audio Framework

Related tags

Audio LEAF
Overview

LEAF

LEAF (Lightweight Embedded Audio Framework) is a C library for audio synthesis and processing created by Jeff Snyder, Mike Mulshine, and Matt Wang at Princeton University's New Instrument Research Lab. It was originally called OOPS when we started writing it in 2017, so you may see references to it under that name as well.

The library consists of a set of high-level audio synthesis components (Oscillators, Filters, Envelopes, Delays, Reverbs, and other Utilities).

Our primary use case is embedded audio computing on 32-bit ARM microcontrollers that can run "bare-metal" (without an OS), such as the STM32f4, STM32f7, and STM32H7. The code, however, is general enough to be used in many other situations as well. We have included a JUCE VST/AU generating template to test the library (2), and the python script we use to generate wavetables.

Many of these algorithms are sourced from other projects, especially the STK (Sound Toolkit) library and various discussions on the music-DSP mailing list. We also owe a lot to open source computer programming languages, such as C-sound, ChucK, PureData, and Supercollider.

Other interesting projects to check out that similarly target embedded applicatons are: TeensyAudio (C++), Hoxton Owl (C++), Axoloti (C), and Mutable Instruments (C++).

Some notes about LEAF:

(1) LEAF has no dependencies on any other code, except for the standard math.h functions like sinf().

(2) Use of standard malloc and calloc are avoided, and a custom memory pool implementation is included instead. This allows dynamic memory allocation/deallocation within a fixed block size, with more control over where the memory is located (very useful for embedded systems).

(3) The included JUCE template is intended to simplify using LEAF for audio software development aimed at computers or mobile devices. features an easily reconfigurable UI and simple block and tick setup to test the library components. Of course, if you intend to use the provided JUCE plugin project, you need to get JUCE and the Projucer ( https://www.juce.com/get-juce ). Check out the first tutorial to get started - it's fun and easy!

(4) if you are looking to add LEAF to a System Workbench (SW4STM32) project (the free IDE for developing STM32 embedded firmware) then follow this guide to include LEAF: https://docs.google.com/document/d/1LtMFigQvnIOkRCSL-UVge4GM91woTmVkidlzzgtCjdE/edit?usp=sharing If you don't want to deal with using leaf as a git submodule, you can also just drop the .c and .h files from LEAF's Src and Inc folders into your own Src and Inc folders, that will work as well - it'll just be a little harder to update things to newer versions of LEAF later on.

///

LEAF conventions:

We call the psuedo-objects in LEAF "objects" because it's simpler to say, even though technically they are just structs with associated functions.

Objects types start with a lowercase t: like tCycle, tSawtooth, tRamp, tEnvelopeFollower

All function names start with the object type name, followed by an underscore, then the function name in camel-case: like tCycle_setFreq(), tRamp_setDest().

The first input parameter of all LEAF functions associated with an object type is the object instance it is operating on.

LEAF assumes a global sample rate (passed into LEAF when the library itself is initialized). You can change this sample rate whenever you want, and all objects that use the global sample rate will see the change.

All LEAF objects operate on single-precision float input and output data. There are sometimes double-precision operations inside LEAF objects, but only when the precision is deemed to actually be necessary for the operation. Float literal notation (like 0.12f instead of 0.12) is used to force computation using single precision when it would otherwise be ambigious, to make LEAF more efficient for processors that have a single-precision FPU, or where the FPU computes single-precision faster than double-precision.

Audio inputs and outputs are assumed to be between -1.0 and 1.0

Internally, LEAF objects that the user defines globally (by writing something like "tCycle mySine") are actually just pointers. The structs for the data of the LEAF object is created in the mempool when the init function is called on that pointer (as in tCycle_init(&mySine)). For instance, when you create a tCycle, you are actually just creating a pointer, and when you call tCycle_init() on that tCycle, you are then creating a struct that has real data inside the mempool. This is done to make the footprint of the LEAF objects very small outside of their designated mempools, so that a large number of LEAF object pointers can exist globally even if only a few complete objects can exist in memory at a single time.

All LEAF objects must have an init() function and a free() function -- these will create the objects inside the default mempool, as well as an initToPool() and freeFromPool() function -- these will create the objects inside a user-defined specific mempool that is not the default.

LEAF objects assume that they will be "ticked" once per sample, and generally take single sample input and produce single sample output. The alternative would be to have the user pass in an array and have the objects operate on the full array, which could have performance advantages if SIMD instructions are available on the processor, but would have disadvantages in flexibility of use. If an audio object requires some kind of buffer to operate on (such as a pitch detector) it will collect samples in its sample-by-sample tick function and store them in its own internal buffer.

////

Example of using LEAF:

//in your user code, create an instance of a master leaf object. This exists so that in the case of a plugin environment or other situation with shared resources, you can still have separate instances of the LEAF library that won't conflict.
LEAF leaf

//then create instances of whatever LEAF objects you want
tCycle mySine;


//also, create a memory pool array where you can store the data for the LEAF objects. It should be an array of chars. Note that you can also define multiple mempool objects in different memory locations in you want to, and initialize different LEAF objects in different places. However, LEAF needs at least one mempool, so the one you pass into the LEAF_init function is considered the default location for any LEAF-related memory, unless you specifically put something elsewhere by using an initToPool() function instead of init(). LEAF object store a pointer to which mempool they are in, so if you initToPool it will remember where you put it and the free() function will free it from the correct location.

#define MEM_SIZE 500000
char myMemory[MEM_SIZE];


//we'll assume your code has some kind of audio buffer that is transmitting samples to an audio codec or an operating system's audio driver. In this example, let's define this here.

#define AUDIO_BUFFER_SIZE 128
float audioBuffer[AUDIO_BUFFER_SIZE];


//then initialize the whole LEAF library (this only needs to be done once, it sets global parameters like the default mempool and the sample rate)
//the parameters are: master leaf instance, sample rate, audio buffer size in samples, name of mempool array, size of mempool array, and address of a function to generate a random number. In this case, there is a function called randomNumber that exists elsewhere in the user code that generates a random floating point number from 0.0 to 1.0. We ask the user to pass in a random number function because LEAF has no dependencies, and users developing on embedded systems may want to use a hardware RNG, for instance.

LEAF_init(&leaf, 48000, AUDIO_BUFFER_SIZE, myMemory, MEM_SIZE, &randomNumber);


//now initialize the object you want to use, in this case the sine wave oscillator you created above. You need to also pass in the instance of the master leaf object (only needed for initializing objects).

tCycle_init(&mySine, &leaf);


//set the frequency of the oscillator (defaults to zero). In a real use case, you'd probably want to be updating this to new values in the audio frame based on knob positions or midi data or other inputs, but here we'll assume it stays fixed.

tCycle_setFreq(&mySine, 440.0);


//now, in your audio callback (a function that will be called every audio frame, to compute the samples needed to fill the audio buffer) tick the LEAF audio object to generate or process audio samples. 

void audioFrame()
{
  for (int i = 0; i < AUDIO_BUFFER_SIZE; i++)
  {
    audioBuffer[i] = tCycle_tick(&mySine);
  }
}
Comments
  • oscillator optimization?

    oscillator optimization?

    I was thinking about a way to remove the if, else if chain from the square,tri,etc wavetable selection. how about if table and tableinterpolate (w) get calculated in setFreq, and the waveform is calculated as

        out = (triangle[c->oct][idx] * c->w) + (triangle[c->oct+1][idx] * (1.0f - c->w));
    

    or

        out = triangle[c->oct+1][idx] + (triangle[c->oct][idx] - triangle[c->oct+1]) * c->w;
    

    clip the frequency at 20 and 20480 (just for the purpose of table interpolation) and you don't need the first and last if/else cases.

    this in setFreq

    c->w = freq * INV_20;
    for(c->oct = 0; c->w > 2.0f; c->oct++)
    	c->w = 0.5f * c->w;
    c->w = 2.0f - c->w;
    

    this should keep the pipeline from dumping by eliminating if statements.

    opened by tomerbe 1
  • License?

    License?

    Thanks for sharing! Would you be willing to put an open source license on this project? GitHub makes it easy and if you don't have a preference, I think the MIT License might be easiest for people to generally use.

    opened by timburks 1
  • try to redo the wavefolder

    try to redo the wavefolder

    I just read some interesting papers on digital simulation of analog WaveFolding.

    https://www.researchgate.net/publication/321985944_Virtual_Analog_Models_of_the_Lockhart_and_Serge_Wavefolders?_iepl%5BviewId%5D=xXZPsd9eJ3jHwUNttlf7t7An&_iepl%5Bcontexts%5D%5B0%5D=projectUpdatesLog&_iepl%5BtargetEntityId%5D=PB%3A321985944&_iepl%5BinteractionType%5D=publicationTitle

    Would be cool to add a "tLockhartWaveFolder" module in LEAF based on their code.

    They have an example they implemented in gen~, so we can base it on that: https://www.researchgate.net/publication/320957932_Lockhart_Wavefolder_Demo_V3

    /ATNIALIASED LOCKHART WAVEFOLDER/

    // Custom Lambert-W Function Lambert_W(x,Ln1) {

    // Error threshold thresh = 10e-10; // Initial guess (use previous value) w = Ln1;

    // Haley's method (Sec. 4.2 of the paper)
    for(i=0; i<1000; i+=1) {
    	
        expw = pow(e,w);
    
        p = w*expw - x;
        r = (w+1)*expw;
        s = (w+2)/(2*(w+1));        err = (p/(r-(p*s)));
    	
    	if (abs(err)<thresh) {
    		break;
    	}
    	
    	w = w - err;
    }
    
    return w;
    

    }

    // Declare previous state History Ln1; History Fn1; History xn1;

    // Constants RL = 7.5e3; R = 15e3;
    VT = 26e-3; Is = 10e-16;

    a = 2RL/R; b = (R+2RL)/(VTR); d = (RLIs)/VT;

    // Antialiasing error threshold thresh = 10e-10;

    // Compute Antiderivative l = sign(in1); u = dpow(e,lbin1); Ln = Lambert_W(u,Ln1); Fn = (0.5VT/b)(Ln(Ln + 2)) - 0.5ain1*in1;

    // Check for ill-conditioning if (abs(in1-xn1)<thresh) {

    // Compute Averaged Wavefolder Output
    xn = 0.5*(in1+xn1);
    u = d*pow(e,l*b*xn);
    Ln = Lambert_W(u,Ln1);
    out1 = l*VT*Ln - a*xn;
    

    } else {

    // Apply AA Form
    out1 = (Fn-Fn1)/(in1-xn1);
    

    }

    // Update States Ln1 = Ln; Fn1 = Fn; xn1 = in1;

    opened by spiricom 1
  • fix oversampler

    fix oversampler

    test if it currently works (I think it lowpasses too far)

    switch to fir implementation

    think about how oversampled processes are handled (does the oversampler take an array of two samples and then you manipulate them?) or do you pass a function pointer?

    Is it possible to have a generic upsample object that can have the ratio passed in?

    make an array of possible fir filter coefficients for different sample ratios?

    maybe use arm functions for interpolating and decimating?? https://www.keil.com/pack/doc/CMSIS/DSP/html/group__FIR__Interpolate.html https://www.keil.com/pack/doc/CMSIS/DSP/html/group__FIR__decimate.html

    opened by spiricom 1
  • tNoise now assumes 0.0-1.0 random number input and outputs -1.0 to 1.0

    tNoise now assumes 0.0-1.0 random number input and outputs -1.0 to 1.0

    In our dev code (Matt and I changed this while working on the oversampler). Let's push that change to Master. One thing that is weird about that, though, is that previous code in the Genera and Vocodec set the random number generation function in Main() to generate a -1.0 to 1.0 number, so that tNoise could use it directly. We need to change this in our user code where it occurs. I think this is the right change for LEAF, though.

    opened by spiricom 0
  • documentation

    documentation

    make examples for each object

    list objects and their inputs/functions (maybe this is part of moving to Doxygen)

    reorder objects within their files to put common ones first (neuron should not be first in oscillators!)

    more credits! need to credit where we took things from (stk, mda, pitchshift)

    opened by spiricom 2
Owner
Jeff Snyder
Jeff Snyder
PortAudio is a portable audio I/O library designed for cross-platform support of audio

PortAudio is a cross-platform, open-source C language library for real-time audio input and output.

PortAudio 750 Nov 19, 2022
JUCE is an open-source cross-platform C++ application framework for desktop and mobile applications, including VST, VST3, AU, AUv3, RTAS and AAX audio plug-ins.

JUCE is an open-source cross-platform C++ application framework for creating high quality desktop and mobile applications, including VST, VST3, AU, AU

JUCE 4.6k Nov 21, 2022
C++ audio plug-in framework for desktop, mobile (iOS) and web

iPlug 2 C++ audio plug-in framework for desktop, mobile (iOS) and web iPlug 2 is a simple-to-use C++ framework for developing cross-platform audio plu

iPlug 2 Framework 1.4k Nov 21, 2022
A simple C++ library for reading and writing audio files.

AudioFile A simple header-only C++ library for reading and writing audio files. Current supported formats: WAV AIFF Author AudioFile is written and ma

Adam Stark 667 Nov 22, 2022
A C library for reading and writing sound files containing sampled audio data.

libsndfile libsndfile is a C library for reading and writing files containing sampled audio data. Authors The libsndfile project was originally develo

null 1.1k Nov 15, 2022
C library for cross-platform real-time audio input and output

libsoundio C library providing cross-platform audio input and output. The API is suitable for real-time software such as digital audio workstations as

Andrew Kelley 1.6k Nov 22, 2022
C++ Audio and Music DSP Library

_____ _____ ___ __ _ _____ __ __ __ ____ ____ / \\_ \\ \/ / |/ \| | | | \_ \/ \ | Y Y \/ /_ \> <| | Y Y \ | |_|

Mick Grierson 1.4k Nov 26, 2022
Single file audio playback and capture library written in C.

A single file library for audio playback and capture. Example - Documentation - Supported Platforms - Backends - Major Features - Building - Unofficia

David Reid 2.5k Nov 25, 2022
SimplE Lossless Audio

SELA SimplE Lossless Audio A lossless audio codec which aims to be as simple as possible while still having good enough compression ratios. Code Quali

Ratul Saha 207 Sep 13, 2022
Easy and efficient audio synthesis in C++

Tonic Fast and easy audio synthesis in C++. Prefer coding to patching? Love clean syntax? Care about performance? That's how we feel too, and why we m

null 479 Nov 22, 2022
An audio mixer that supports various file formats for Simple Directmedia Layer.

An audio mixer that supports various file formats for Simple Directmedia Layer.

Simple Directmedia Layer 188 Nov 22, 2022
Explore fractals in an audio-visual sandbox

Fractal Sound Explorer Explore fractals in an audio-visual sandbox Download executable on my itch.io page: https://codeparade.itch.io/fractal-sound-ex

null 976 Nov 24, 2022
Audio out with an FTDI UART cable

Audio out with an FTDI UART cable This encodes audio as either PDM (using a first order sigma-delta stage), 32-bits PWM or 64-bits PWM and sends it as

Konrad Beckmann 42 Jun 23, 2022
Sexy, audio-responsive effects on LED strips.

Striptease Sexy, audio-responsive effects on LED strips. For Teensy 4 with Audio Adapter Board, by PJRC. Quick demo Shooting video of LEDs is very tri

Luca Paolini 35 Jun 19, 2022
audio monitor filter for OBS Studio

Audio Monitor dock and filter for OBS Studio Plugin for OBS Studio to add Audio Monitor dock and filter. It allows you to put the audio of a OBS sourc

Exeldro 213 Nov 20, 2022
An Audio Steganography Tool, written in C++

HiddenWave Hide your personal Data inside The Audio wav file Written in C++ by Gaurav Raj [TheHackersBrain] Hiddenwave is an audio steganography tool

Gaurav Raj 48 Nov 2, 2022
C++ audio time-stretching implementation

Time Stretcher C++ audio time-stretching implementation, based on the algorithms presented in: Audio Time Stretching with an Adaptive Phase Vocoder, N

null 59 Nov 13, 2022
Block all ads in vídeo, áudio, banner, anti-skip

NoAdSpotify Block spotify ad This is an updated and simplified version of the project: BlockTheSpot Last updated: 6th June 2021 Last tested version: 1

null 14 Nov 5, 2022
Arduino Audio Tools (Music Player, Music Recorder supporting I2S, Microphones, DAC, ADC, A2DP, Url)

Arduino Audio Tools Some basic header-only C++ classes that can be used for Audio Processing provided as Arduino Library: a simple I2S class (to read

Phil Schatzmann 478 Nov 27, 2022