Coqui Inference Engine is a library for efficiently deploying speech models.

Overview

Coqui Inference Engine

Covenant Gitter

Coqui Inference Engine is a library for efficiently deploying speech models.

This project is at an early proof-of-concept stage. Collaboration on design and implementation is very welcome. Join our Gitter channel by clicking the badge above!

This project is the successor to the STT "native client", containing the core inference logic for deploying Coqui STT models (and eventually Coqui TTS and other models too).

Coqui Inference Engine aims to be:

  • Fast: streaming inference with low latency on small devices (phones, IoT)
  • Easy to use: simple, stable, well-documented API
  • Available: easy to expose to different programming languages, available in standard package managers
  • Extensible: able to handle different model types, architectures, and formats

📰 Subscribe to the 🐸 Coqui.ai Newsletter

Build instructions

For the build you'll need to install CMake >= 3.10.

Currently you'll also have to build onnxruntime yourself and place the built files manually before building the inference engine, following the steps below:

$ # Clone the Coqui Inference Engine repo
$ git clone https://github.com/coqui-ai/inference-engine
$ # Clone onnxruntime repo
$ git clone --recursive https://github.com/microsoft/onnxruntime/
$ cd onnxruntime
$ # Build it
$ ./build.sh --config Debug --parallel
$ # Copy built files for inference engine build
$ cp -R build/*/*/libonnxruntime* ../inference-engine/onnxruntime/lib

Now, we're ready to build the inference engine:

$ cd ../inference-engine
$ # Create build dir
$ mkdir build
$ cd build
$ # Prepare build
$ cmake -DCMAKE_BUILD_TYPE=Debug ..
$ # Build
$ make -j

You should now be able to run the test client by running ./main.

$ ./main --model ../output_graph.onnx --audio ../test-audio.wav

Exporting a test model from Coqui STT

You can use the experimental-inference-engine-export branch of Coqui STT to export an STT checkpoint in the format expected by the inference engine.

$ git clone --branch experimental-inference-engine-export https://github.com/coqui-ai/STT
$ cd STT
$ python -m pip install -e .
$ cd native_client/ctcdecode
$ make bindings
$ python -m pip install --force-reinstall dist/*.whl

After the steps above, you can then follow the documentation for exporting a model, and include the --export_onnx true flag. You should then get an output_graph.onnx file exported which can be read by the inference engine.

You might also like...
MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files supporting textures, skeletons, and all provided vertex data.

MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files. A dynamic model is one that is animated or is spawned in during the game.

16 channel R/C system for models of all kinds

A 16 channel Radio Controlled system for R/C models of all kinds. This project was started during the 2020 pandemic lockdown (thus the title), and has since grown out of passion & enthusiasm.

The repository contains some examples of pre-trained SNN (Spiking Neural Network) models.

About the Project The repository contains some examples of pre-trained SNN (Spiking Neural Network) models. The models were trained using the MM-BP tr

ToPS is an objected-oriented framework implemented using C++ that facilitates the integration of probabilistic models for sequences over a user defined alphabet

ToPS is an objected-oriented framework implemented using C++ that facilitates the integration of probabilistic models for sequences over a user defined alphabet

imGuIZMO.quat is a ImGui widget: like a trackball it provides a way to rotate models, lights, or objects with mouse, and graphically visualize their position in space, also around any single axis (Shift/Ctrl/Alt/Super)
imGuIZMO.quat is a ImGui widget: like a trackball it provides a way to rotate models, lights, or objects with mouse, and graphically visualize their position in space, also around any single axis (Shift/Ctrl/Alt/Super)

imGuIZMO.quat v3.0 imGuIZMO.quat is a ImGui widget: like a trackball it provides a way to rotate models, lights, or objects with mouse, and graphicall

6D - Pose Annotation Tool (6D-PAT) - is a tool that allows the user to load a set of images and also a set of 3D models and annotate where in the 2D image the 3D object ist placed.
6D - Pose Annotation Tool (6D-PAT) - is a tool that allows the user to load a set of images and also a set of 3D models and annotate where in the 2D image the 3D object ist placed.

6D - Pose Annotation Tool (6D-PAT) For detiled explanations checkout the WikiPage. What is it? With 6D-PAT you can create 6D annotations on images for

Love 6's Regular Expression Engine. Support Concat/Select/Closure Basic function. Hope u can enjoy this tiny engine :)
Love 6's Regular Expression Engine. Support Concat/Select/Closure Basic function. Hope u can enjoy this tiny engine :)

Regex_Engine Love 6's Blog Website: https://love6.blog.csdn.net/ Love 6's Regular Expression Engine Hope u can love my tiny regex engine :) maybe a fe

Sword Engine is a fork of Psych Engine that plans on adding more features and quality of life improvements.
Sword Engine is a fork of Psych Engine that plans on adding more features and quality of life improvements.

⚠️ WARNING: This README is currently incomplete, This warning will be removed once it's complete. Friday Night Funkin' - Sword Engine Sword Engine is

Two Tribes Engine; the engine which we used to create Toki Tori 2+ and RIVE with

Two Tribes Engine We, Two Tribes, have used this engine for over a decade. It started with early development on the Nintendo DS and ultimately resulte

Comments
Owner
coqui
Coqui, a startup providing open speech tech for everyone 🐸
coqui
ML based Text-to-Speech (TTS) engine for the Armenian language.

ArmTTS {#mainpage} ML based Text-to-Speech(TTS) C++ library for the Armenian language (light version, only CPU and Tensorflow Lite). The full version

null 1 Jul 17, 2022
A simple tool that aims to efficiently and quickly parse the outputs of web scraping tools like gau

massurl is a simple tool that aims to parse the outputs of tools like gau, and extract the parameters for each URL, remove duplicates and do it all very quickly. Because web scraping tools' outputs can get very large very quickly, it is nice to have a tool that parses them and and outputs something clean and easy to read.

Fr1nge 13 Jul 24, 2022
Fast unidirectional synchronization - make or efficiently update a copy of a database, without slow dumping & reloading

Fast unidirectional synchronization - make or efficiently update a copy of a database, without slow dumping & reloading

Will Bryant 267 Dec 23, 2022
OpenMLDB is an open-source database particularly designed to efficiently provide consistent data for machine learning driven applications.

English version | 中文版 1. Introduction OpenMLDB is an open-source database particularly designed to efficiently provide consistent data for machine lea

4Paradigm 1.2k Jan 8, 2023
Extension for PHP to interface efficiently with a Controller Area Network (CAN bus) 2.0A / 2.0B

PHP-CanBus Extension PHP-canbus is THE extension for PHP on Linux that allows PHP code to interface efficiently with a Controller Area Network (CAN bu

Adamczyk Piotr 5 Sep 10, 2022
Separable Subsurface Scattering is a technique that allows to efficiently perform subsurface scattering calculations in screen space in just two passes.

Separable Subsurface Scattering Separable Subsurface Scattering is a technique that allows to efficiently perform subsurface scattering calculations i

Jorge Jimenez 540 Dec 22, 2022
ESP32-CAM with LVGL Speech/Face Recognition IR Control

ESP_MASTER 视频介绍:https://www.bilibili.com/video/BV1SM4y1V7Yb This is a comprehensive project that combines the powerful computing capabilities of ESP32

Kevincoooool 198 Jan 5, 2023
Trio-based Inference of Dominance and Selection

TIDES Trio-based inference of dominance and selection TIDES is a statistical model for inferring dominance and the strength of natural selection from

Gustavo Valadares Barroso 1 Oct 25, 2021
OneFlow Backend For Triton Inference Server

Triton Inference Server OneFlow Backend

ZeKai Zhou 3 Jan 6, 2022
Add Sidecar support to unsupported models

SidecarFixup Lilu Kernel extension for enabling Sidecar support on the following SMBIOS

Acidanthera 262 Jan 2, 2023