Lyra: a generative low bitrate speech codec

Related tags

Algorithms lyra
Overview

Lyra: a generative low bitrate speech codec

What is Lyra?

Lyra is a high-quality, low-bitrate speech codec that makes voice communication available even on the slowest networks. To do this it applies traditional codec techniques while leveraging advances in machine learning (ML) with models trained on thousands of hours of data to create a novel method for compressing and transmitting voice signals.

Overview

The basic architecture of the Lyra codec is quite simple. Features are extracted from speech every 40ms and are then compressed for transmission at a bitrate of 3kbps. The features themselves are log mel spectrograms, a list of numbers representing the speech energy in different frequency bands, which have traditionally been used for their perceptual relevance because they are modeled after human auditory response. On the other end, a generative model uses those features to recreate the speech signal.

Lyra harnesses the power of new natural-sounding generative models to maintain the low bitrate of parametric codecs while achieving high quality, on par with state-of-the-art waveform codecs used in most streaming and communication platforms today.

Computational complexity is reduced by using a cheaper recurrent generative model, a WaveRNN variation, that works at a lower rate, but generates in parallel multiple signals in different frequency ranges that it later combines into a single output signal at the desired sample rate. This trick, plus 64-bit ARM optimizations, enables Lyra to not only run on cloud servers, but also on-device on mid-range phones, such as Pixel phones, in real time (with a processing latency of 90ms). This generative model is then trained on thousands of hours of speech data with speakers in over 70 languages and optimized to accurately recreate the input audio.

Prerequisites

There are a few things you'll need to do to set up your computer to build Lyra.

Common setup

Lyra is built using Google's build system, Bazel. Install it following these instructions.

Lyra can be built from linux using bazel for an arm android target, or a linux target. The android target is optimized for realtime performance. The linux target is typically used for development and debugging.

You will also need to install some tools (which may already be on your system). You can install them with:

sudo apt update
sudo apt install ninja-build git cmake clang python

Linux requirements

The instructions below are for Ubuntu and have been verified on 20.04.

You will need to install a certain version of clang to ensure ABI compatibility.

git clone https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout 96ef4f307df2

mkdir build_clang
cd build_clang
cmake -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_ENABLE_PROJECTS="clang" -DCMAKE_BUILD_TYPE=release ../llvm
ninja
sudo $(which ninja) install

cd ..
mkdir build_libcxx
cd build_libcxx
cmake -G Ninja -DCMAKE_C_COMPILER=/usr/local/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/bin/clang++ -DLLVM_ENABLE_PROJECTS="libcxx;libcxxabi" -DCMAKE_BUILD_TYPE=release ../llvm
ninja
sudo $(which ninja) install

sudo ldconfig

Note: the above will install a particular version of libc++ to /usr/local/lib, and clang to /usr/local/bin, which the toolchain depends on.

Android requirements

Building on android requires downloading a specific version of the android NDK toolchain. If you develop with Android Studio already, you might not need to do these steps if ANDROID_HOME and ANDROID_NDK_HOME are defined and pointing at the right version of the NDK.

  1. Download the sdk manager from https://developer.android.com/studio
  2. Unzip and cd to the directory
  3. Check the available packages to install in case they don't match the following steps.
bin/sdkmanager  --sdk_root=$HOME/android/sdk --list

Some systems will already have the java runtime set up. But if you see an error here like ERROR: JAVA_HOME is not set and no 'java' command could be found on your PATH., this means you need to install the java runtime with sudo apt install default-jdk first. You will also need to add export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 (type ls /usr/lib/jvm to see which path was installed) to your $HOME/.bashrc and reload it with source $HOME/.bashrc.

  1. Install the r21 ndk, android sdk 29, and build tools:
bin/sdkmanager  --sdk_root=$HOME/android/sdk --install  "platforms;android-29" "build-tools;29.0.3" "ndk;21.4.7075529"
  1. Add the following to .bashrc (or export the variables)
export ANDROID_NDK_HOME=$HOME/android/sdk/ndk/21.4.7075529
export ANDROID_HOME=$HOME/android/sdk
  1. Reload .bashrc (with source $HOME/.bashrc)

Building

The building and running process differs slightly depending on the selected platform.

Building for Linux

You can build the cc_binaries with the default config. encoder_main is an example of a file encoder.

bazel build -c opt :encoder_main

You can run encoder_main to encode a test .wav file with some speech in it, specified by --input_path. The --model_path flag contains the model data necessary to encode, and --output_path specifies where to write the encoded (compressed) representation.

bazel-bin/encoder_main --model_path=wavegru --output_dir=$HOME/temp --input_path=testdata/16khz_sample_000001.wav

Similarly, you can build decoder_main and use it on the output of encoder_main to decode the encoded data back into speech.

bazel build -c opt :decoder_main
bazel-bin/decoder_main  --model_path=wavegru --output_dir=$HOME/temp/ --encoded_path=$HOME/temp/16khz_sample_000001.lyra

Building for Android

Android App

There is an example APK target called lyra_android_example that you can build after you have set up the NDK.

This example is an app with a minimal GUI that has buttons for two options. One option is to record from the microphone and encode/decode with Lyra so you can test what Lyra would sound like for your voice. The other option runs a benchmark that encodes and decodes in the background and prints the timings to logcat.

bazel build android_example:lyra_android_example --config=android_arm64 --copt=-DBENCHMARK
adb install bazel-bin/android_example/lyra_android_example.apk

After this you should see an app called "Lyra Example App".

You can open it, and you will see a simple TextView that says the benchmark is running, and when it finishes.

Press "Record from microphone", say a few words (be sure to have your microphone near your mouth), and then press "Encode and decode to speaker". You should hear your voice being played back after being coded with Lyra.

If you press 'Benchmark', you should you should see something like the following in logcat on a Pixel 4 when running the benchmark:

I  Starting benchmarkDecode()
I  I20210401 11:04:06.898649  6870 lyra_wavegru.h:75] lyra_wavegru running fast multiplication kernels for aarch64.
I  I20210401 11:04:06.900411  6870 layer_wrapper.h:162] |lyra_16khz_ar_to_gates_| layer:  Shape: [3072, 4]. Sparsity: 0
I  I20210401 11:04:07.031975  6870 layer_wrapper.h:162] |lyra_16khz_gru_layer_| layer:  Shape: [3072, 1024]. Sparsity: 0.9375
...
I  I20210401 11:04:26.700160  6870 benchmark_decode_lib.cc:167] Using float arithmetic.
I  I20210401 11:04:26.700352  6870 benchmark_decode_lib.cc:85] conditioning_only stats for generating 2000 frames of audio, max: 506 us, min: 368 us, mean: 391 us, stdev: 10.3923.
I  I20210401 11:04:26.725538  6870 benchmark_decode_lib.cc:85] model_only stats for generating 2000 frames of audio, max: 12690 us, min: 9087 us, mean: 9237 us, stdev: 262.416.
I  I20210401 11:04:26.729460  6870 benchmark_decode_lib.cc:85] combined_model_and_conditioning stats for generating 2000 frames of audio, max: 13173 us, min: 9463 us, mean: 9629 us, stdev: 270.788.
I  Finished benchmarkDecode()

This shows that decoding a 25Hz frame (each frame is .04 seconds) takes 9629 microseconds on average (.0096 seconds). So decoding is performed at around 4.15 (.04/.0096) times faster than realtime.

For even faster decoding, you can use a fixed point representation by building with --copt=-DUSE_FIXED16, although there may be some loss of quality.

To build your own android app, you can either use the cc_library target outputs to create a .so that you can use in your own build system. Or you can use it with an android_binary rule within bazel to create an .apk file as in this example.

There is a tutorial on building for android with Bazel in the bazel docs.

Android command-line binaries

There are also the binary targets that you can use to experiment with encoding and decoding .wav files.

You can build the example cc_binary targets with:

bazel build -c opt :encoder_main --config=android_arm64
bazel build -c opt :decoder_main --config=android_arm64

This builds an executable binary that can be run on android 64-bit arm devices (not an android app). You can then push it to your android device and run it as a binary through the shell.

# Push the binary and the data it needs, including the model, .wav, and .so files:
adb push bazel-bin/encoder_main /data/local/tmp/
adb push bazel-bin/decoder_main /data/local/tmp/
adb push wavegru/ /data/local/tmp/
adb push testdata/ /data/local/tmp/
adb shell mkdir -p /data/local/tmp/_U_S_S_Csparse_Uinference_Umatrixvector___Ulib_Sandroid_Uarm64
adb push bazel-bin/_solib_arm64-v8a/_U_S_S_Csparse_Uinference_Umatrixvector___Ulib_Sandroid_Uarm64/libsparse_inference.so /data/local/tmp/_U_S_S_Csparse_Uinference_Umatrixvector___Ulib_Sandroid_Uarm64

adb shell
cd /data/local/tmp
./encoder_main --model_path=/data/local/tmp/wavegru --output_dir=/data/local/tmp --input_path=testdata/16khz_sample_000001.wav
./decoder_main --model_path=/data/local/tmp/wavegru --output_dir=/data/local/tmp --encoded_path=16khz_sample_000001.lyra

The encoder_main/decoder_main as above should also work.

API

For integrating Lyra into any project only two APIs are relevant: LyraEncoder and LyraDecoder.

DISCLAIMER: At this time Lyra's API and bit-stream are not guaranteed to be stable and might change in future versions of the code.

On the sending side, LyraEncoder can be used to encode an audio stream using the following interface:

class LyraEncoder : public LyraEncoderInterface {
 public:
  static std::unique_ptr Create(
      int sample_rate_hz, int num_channels, int bitrate, bool enable_dtx,
      const ghc::filesystem::path& model_path);

  absl::optionaluint8_t>> Encode(
      const absl::Span<const int16_t> audio) override;

  int sample_rate_hz() const override;

  int num_channels() const override;

  int bitrate() const override;

  int frame_rate() const override;
};

The static Create method instantiates a LyraEncoder with the desired sample rate in Hertz, number of channels and bitrate, as long as those parameters are supported. Else it returns a nullptr. The Create method also needs to know if DTX should be enabled and where the model weights are stored. It also checks that these weights exist and are compatible with the current Lyra version.

Given a LyraEncoder, any audio stream can be compressed using the Encode method. The provided span of int16-formatted samples is assumed to contain 40ms of data at the sample rate chosen at Create time. As long as this condition is met the Encode method returns the encoded packet as a vector of bytes that is ready to be stored or transmitted over the network.

The rest of the LyraEncoder methods are just getters for the different predetermined parameters.

On the receiving end, LyraDecoder can be used to decode the encoded packet using the following interface:

class LyraDecoder : public LyraDecoderInterface {
 public:
  static std::unique_ptr Create(
      int sample_rate_hz, int num_channels, int bitrate,
      const ghc::filesystem::path& model_path);

  bool SetEncodedPacket(absl::Span<const uint8_t> encoded) override;

  absl::optionalint16_t>> DecodeSamples(int num_samples) override;

  absl::optionalint16_t>> DecodePacketLoss(
      int num_samples) override;

  int sample_rate_hz() const override;

  int num_channels() const override;

  int bitrate() const override;

  int frame_rate() const override;

  bool is_comfort_noise() const override;
};

Once again, the static Create method instantiates a LyraDecoder with the desired sample rate in Hertz, number of channels and bitrate, as long as those parameters are supported. Else it returns a nullptr. These parameters don't need to be the same as the ones in LyraEncoder. And once again, the Create method also needs to know where the model weights are stored. It also checks that these weights exist and are compatible with the current Lyra version.

Given a LyraDecoder, any packet can be decoded by first feeding it into SetEncodedPacket, which returns true if the provided span of bytes is a valid Lyra-encoded packet.

Then the int16-formatted samples can be obtained by calling DecodeSamples, as long as the total number of samples obtained this way between any two calls to SetEncodedPacket is less than 40ms of data at the sample rate chose at Create time.

If there isn't a packet available, but samples still need to be generated, DecodePacketLoss can be used, which doesn't have a restriction on the number of samples.

In those cases, the decoder might switch to a comfort noise generation mode, which can be checked using is_confort_noise.

The rest of the LyraDecoder methods are just getters for the different predetermined parameters.

For an example on how to use LyraEncoder and LyraDecoder to encode and decode a stream of audio, please refer to the integration test.

License

Use of this source code is governed by a Apache v2.0 license that can be found in the LICENSE file.

Please note that there is a closed-source kernel used for math operations that is linked via a shared object called libsparse_inference.so. We provide the libsparse_inference.so library to be linked, but are unable to provide source for it. This is the reason that a specific toolchain/compiler is required.

Papers

  1. Kleijn, W. B., Lim, F. S., Luebs, A., Skoglund, J., Stimberg, F., Wang, Q., & Walters, T. C. (2018, April). Wavenet based low rate speech coding. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 676-680). IEEE.
  2. Denton, T., Luebs, A., Lim, F. S., Storus, A., Yeh, H., Kleijn, W. B., & Skoglund, J. (2021). Handling Background Noise in Neural Speech Generation. arXiv preprint arXiv:2102.11906.
  3. Kleijn, W. B., Storus, A., Chinen, M., Denton, T., Lim, F. S., Luebs, A., ... & Yeh, H. (2021). Generative Speech Coding with Predictive Variance Regularization. arXiv preprint arXiv:2102.09660.
Comments
  • bazel build -c opt :encoder_main error on 20.04-Ubuntu

    bazel build -c opt :encoder_main error on 20.04-Ubuntu

    when use bazel build -c opt :encoder_main, some error has occur

    ERROR: /home/w/lyra-main/BUILD:860:10: Compiling encoder_main.cc failed: undeclared inclusion(s) in rule '//:encoder_main': this rule is missing dependency declarations for the following files included by 'encoder_main.cc': '/usr/local/lib/clang/14.0.0/include/stddef.h' '/usr/local/lib/clang/14.0.0/include/__stddef_max_align_t.h' '/usr/local/lib/clang/14.0.0/include/stdarg.h' '/usr/local/lib/clang/14.0.0/include/stdint.h' '/usr/local/lib/clang/14.0.0/include/limits.h' Target //:encoder_main failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 6.123s, Critical Path: 4.18s INFO: 3 processes: 3 internal. FAILED: Build did NOT complete successfully

    bug 
    opened by wzhlovepy 11
  • Clang version to build llvm

    Clang version to build llvm

    When i use Clang 3.9.1 to build LLVM 12.0, it failed. llvm-project/llvm/include/llvm/ADT/DenseMap.h:550:37: error: no matching constructor for initialization of 'llvm::ValueEnumerator::MDRange' llvm-project/llvm/include/llvm/ADT/DenseMap.h:201:12: error: no matching constructor for initialization of 'llvm::ValueEnumerator::MDRange' llvm-project/llvm/lib/Bitcode/Writer/ValueEnumerator.cpp:811:11: error: no matching constructor for initialization of 'llvm::ValueEnumerator::MDRange' Should i use which version of Clang to build llvm?

    bug 
    opened by HyacinthJingjing 9
  • Building on Linux (Debian) without Android support

    Building on Linux (Debian) without Android support

    Hello,

    When building on a Linux using the command given in README (bazel build -c opt :encoder_main), I get the following errors:

    ERROR: /usr/local/src/lyra/WORKSPACE:121:1: name 'android_sdk_repository' is not defined
    ERROR: /usr/local/src/lyra/WORKSPACE:128:1: name 'android_ndk_repository' is not defined
    

    If I remove the lines related to android_sdk_repository and android_sdk_repository from the WORKSPACE file, I can build the encoder without issues.

    I would like to find the correct way to deal with this and create a PR, but I don't know bazel, so I didn't find a way to ignore Android targets when building for Linux. Hopefully someone can fix it or point me to the right direction :)

    edit: I'm using Debian in Docker, if that's any help:

    FROM debian:bullseye-slim
    
    WORKDIR /usr/local/src
    
    RUN mkdir -p /usr/share/man/man1 \
        && apt-get update \
        && apt-get install -y \
            ninja-build git cmake clang python bazel-bootstrap
    
    RUN git clone https://github.com/llvm/llvm-project.git \
        && cd llvm-project \
        && git checkout 96ef4f307df2 \
        && mkdir build_clang \
        && cd build_clang \
        && cmake -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_ENABLE_PROJECTS="clang" -DCMAKE_BUILD_TYPE=release ../llvm \
        && ninja \
        && $(which ninja) install \
        && cd .. \
        && mkdir build_libcxx \
        && cd build_libcxx \
        && cmake -G Ninja -DCMAKE_C_COMPILER=/usr/local/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/bin/clang++ -DLLVM_ENABLE_PROJECTS="libcxx;libcxxabi" -DCMAKE_BUILD_TYPE=release ../llvm \
        && ninja \
        && $(which ninja) install \
        && ldconfig
    
    RUN git clone --depth 1 --branch v0.0.1 https://github.com/google/lyra \
        && cd lyra \
        && bazel build -c opt :encoder_main \
        && bazel build -c opt :decoder_main
    
    opened by a-rose 9
  • Build Lyra with Bazel 5.0.0

    Build Lyra with Bazel 5.0.0

    This fixes #76.

    In the ubuntu-latest and macos-latest virtual environments, the default Bazel version is 5.0.0, which requires build-tools 30.0.0 or newer version to work properly. Lyra is currently using build-tools 29.0.3, which will cause Lyra failed to build with the default Bazel in GitHub Actions virtual environments.

    This change will build Lyra with:

    • Bazel 5.0.0
    • build-tools 30.0.3
    • platforms android-30
    • ndk 21.4.7075529 (unchanged)

    Built successfully on hosts:

    • Ubuntu 20.04 (amd64)
      • linux-bin ✅
      • android-bin ✅
      • android-app ✅
    • macOS 11 (Intel)
      • macos-bin ✅
      • android-bin ❌
      • android-app ❌
    • macOS 12.2 (Apple Silicon)
      • macos-bin ✅
      • android-bin ❌
      • android-app ❌

    I also added actions to build on more platforms:

    • android-bin
    • android-app
    • linux-amd64
    • macos-amd64

    GitHub Actions also uploads artifacts for download nightly builds now.

    opened by reekystive 8
  • How to include the encoder + decoder in a simple c program

    How to include the encoder + decoder in a simple c program

    Hello, I am trying to use lyra in a simple c program (hello world) which is built with cmake. As you can see in the screenshot, I can build lyra with bazel (run cmake .. && make && ./main.o from the build dir) but now I am stuck on how I can actually include lyra in the main.c file and how I can link the encoder_main and decoder_main binaies to my main.o file. Can someone help me out? Thanks! screen

    question 
    opened by GatCode 6
  • failure in building Lyra

    failure in building Lyra

    I am building Lyra on Ubuntu 18.04.1 for Linux. The bazel version that I have is 4.0.0 (although I have tried with 4.1.0 as well). The gcc/g++ version that I have is 7.5.0.

    The error appears to be that bazel is invoking gcc with -std=c++0x, yet calling in functionalies that are only supported starting C++17.

    Any help would be appreciated.

    In file included from layer_wrapper.h:29:0, from conv1d_layer_wrapper.h:27, from layer_wrappers_lib.h:21, from causal_convolutional_conditioning.h:28, from wavegru_model_impl.h:28, from wavegru_model_impl.cc:15: layer_wrapper_interface.h: At global scope: layer_wrapper_interface.h:82:8: error: 'variant' in namespace 'std' does not name a template type std::variant<FromDisk, FromConstant> from = FromDisk();

    bug 
    opened by shoham5 5
  • Android no matching toolchains found for types @bazel_tools

    Android no matching toolchains found for types @bazel_tools

    It is ok when I run bazel build -c opt :encoder_main or bazel build -c opt :decoder_main

    however, when it comes to build an Android APP, the error occurs.

    I'm sure that I have add the path of sdk and ndk to ~/.bashrc and reload it, and I can't unserstand what the bazel is showing about.

    ///////////////////////////////////////////// ERROR: While resolving toolchains for target //android_example:lyra_android_example: no matching toolchains found for types @bazel_tools//tools/android:sdk_toolchain_type ERROR: Analysis of target '//android_example:lyra_android_example' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/android:sdk_toolchain_type INFO: Elapsed time: 1.526s INFO: 0 processes. FAILED: Build did NOT complete successfully (14 packages loaded, 24 targets configured) //////////////////////////////////////////////

    bug 
    opened by kbtxwer 5
  • About the wavegru architecture

    About the wavegru architecture

    hi, I am little confused about the wavegru built in the code and I can't get the meaning of the rows and cols of the ar_to_gate layer and gru layer? It seems that the ar_to_gate and conditioning stack both have output with dim=3*1024. but the in_channel of gru layer is 1024... Is the model architecture different from the paper "Generative Speech Coding with Predictive Variance Regularization."?

    opened by loopzy 4
  • The conditioning_only decoder.

    The conditioning_only decoder.

    What's the mean of conditioning_only decoder? I checked the source code but still not understand the different between conditioning_only and model_only . Has any explanation with them.

    question 
    opened by aijianiula0601 4
  • Create CI workflow with GitHub Actions

    Create CI workflow with GitHub Actions

    Create CI workflow with GitHub Actions which:

    • Builds the encoder and decoder for Android
    • Builds the Android APK and uploads it to GitHub as build artifact and on releases

    GitHub Actions needs to be enabled here: https://github.com/google/lyra/actions. In the mean time an example run can be found here.

    Update: Removed the Linux build for now and cleaned up commits. A Linux build will be added in another PR since the specific version requirement makes it a little more complex.

    opened by EwoutH 4
  • Cross compilation

    Cross compilation

    Hi,

    I would like to do cross compilation to bare-metal. I have the tool-chain. Could I use the tool-chain in Bazel? or should I put the Lyra code in the IDE and then compile it?

    Thanks,

    question 
    opened by shoham5 3
  • Realtime Encoding and Decoding on Linux Qstn!

    Realtime Encoding and Decoding on Linux Qstn!

    Kudos, Great Work!

    I wish to capture the microphone stream and feed it to Lyra instead of supplying a wav file,The objective is to create Underwater Communication Equipment. So how should I go on about feeding the binary stream to Lyra for encoding it realtime, The API's mentioned in the docs seem to all accept filename as input.The target is LINUX.

    Regards.

    question 
    opened by thesunRider 1
  • Failed to build for linux

    Failed to build for linux

    I try to build lyra for linux with bazel build -c opt :encoder_main, while I met the following error, which seems that the Android NDK is necessary. 2022-08-09 12-51-24 的屏幕截图

    If I just want to build and run lyra on linux, is it necessary to download Android NDK ?

    bug 
    opened by Ryuk17 3
  • Question about Lyra Python implementation

    Question about Lyra Python implementation

    I was wondering if there's any Python implementation on Lyra and SoundStream. I read an article about SoundStream from Google AI blog and it said they will release it as a separate TensorFlow model for experimentation. So if it's not released yet, can you tell me when the release date is?

    question 
    opened by hyungsupsup 2
  • MacOS `bazel build -c opt :encoder_main` fails to build

    MacOS `bazel build -c opt :encoder_main` fails to build

    Hi, I want to build lyra encoder, decoder for macos target. For now it requires android ndk which I don't want to install, ofc.

    Build error

    I also tried with --config=clang_toolchain, same error.

    $ bazel build -c opt :encoder_main

    DEBUG: Rule 'com_google_protobuf' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "fd8aabf32d1d1ae3319e280fed07aa6eb24d150d", shallow_since = "1614800268 -0800" and dropping ["tag"]
    DEBUG: Repository com_google_protobuf instantiated at:
      /Users/sotomi/projects/lyra-audio-codec/WORKSPACE:26:15: in <toplevel>
    Repository rule git_repository defined at:
      /private/var/tmp/_bazel_sotomi/f57c4149873f470db18811454562e7ef/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
    DEBUG: Rule 'gmaven_rules' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "c166ae9f06069025eab59ad74c102db23badc93abb6582ca69c42a0f441c52b5"
    DEBUG: Repository gmaven_rules instantiated at:
      /Users/sotomi/projects/lyra-audio-codec/WORKSPACE:139:13: in <toplevel>
    Repository rule http_archive defined at:
      /private/var/tmp/_bazel_sotomi/f57c4149873f470db18811454562e7ef/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
    ERROR: /Users/sotomi/projects/lyra-audio-codec/WORKSPACE:124:23: fetching android_ndk_repository rule //external:androidndk: Either the path attribute of android_ndk_repository or the ANDROID_NDK_HOME environment variable must be set.
    ERROR: Analysis of target '//:encoder_main' failed; build aborted: Either the path attribute of android_ndk_repository or the ANDROID_NDK_HOME environment variable must be set.
    INFO: Elapsed time: 0.259s
    INFO: 0 processes.
    FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
        currently loading: @androidsdk//
        Fetching @local_jdk; fetching
        Fetching @local_config_cc_toolchains; Building xcode-locator
    

    Versions

    Apple clang version 13.1.6 (clang-1316.0.21.2.5)
    Target: x86_64-apple-darwin21.4.0
    Thread model: posix
    InstalledDir: /Library/Developer/CommandLineTools/usr/bin
    
    bazel 5.2.0-homebrew
    MacOS 12.3.1 (intel)
    
    bug duplicate 
    opened by SolomidHero 1
  • The same wav file decodes different files in different CPU architectures

    The same wav file decodes different files in different CPU architectures

    I follow this example: image I change parameters:--cpu=armeabi-v7a or --cpu=arm64-v8a, to generate programs with different CPU architectures.

    use --cpu=armeabi-v7a,The MD5 value of decoded file: alioth:/data/local/tmp$ md5sum 16khz_sample_000001_decoded.wav b4b6f9cff4c62615f23b3147133487cc 16khz_sample_000001_decoded.wav

    The MD5 value of decoded file is b4b6f9cff4c62615f23b3147133487cc

    use --cpu=arm64-v8a,The MD5 value of decoded file: alioth:/data/local/tmp$ md5sum 16khz_sample_000001_decoded.wav bf9981f4e88f4b4f1c4d9859a02f0d7f 16khz_sample_000001_decoded.wav

    The MD5 value of decoded file is bf9981f4e88f4b4f1c4d9859a02f0d7f

    the original wav file is same,but decoded file is different. Is this normal ?

    my phone cpu info: alioth:/data/local/tmp $ cat /proc/cpuinfo Processor : AArch64 Processor rev 0 (aarch64) processor : 0 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x51 CPU architecture: 8 CPU variant : 0xd CPU part : 0x805 CPU revision : 14

    processor : 1 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x51 CPU architecture: 8 CPU variant : 0xd CPU part : 0x805 CPU revision : 14

    processor : 2 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x51 CPU architecture: 8 CPU variant : 0xd CPU part : 0x805 CPU revision : 14

    processor : 3 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x51 CPU architecture: 8 CPU variant : 0xd CPU part : 0x805 CPU revision : 14

    processor : 4 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x1 CPU part : 0xd0d CPU revision : 0

    processor : 5 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x1 CPU part : 0xd0d CPU revision : 0

    processor : 6 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x1 CPU part : 0xd0d CPU revision : 0

    processor : 7 BogoMIPS : 38.40 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x1 CPU part : 0xd0d CPU revision : 0

    Hardware : Qualcomm Technologies, Inc SM8250

    alioth:/data/local/tmp $ getprop ro.build.version.release 12 alioth:/data/local/tmp $ getprop ro.build.version.sdk 31 alioth:/data/local/tmp $

    bug 
    opened by liangruxing 3
Releases(v0.0.2)
  • v0.0.2(Jun 28, 2021)

    Lyra version 0.0.2 is now available on GitHub. The main improvement of this version is the open-source release of the sparse_matmul library code, which was co-developed by Google and DeepMind. That means no more pre-compiled “.so” dynamic library binaries and no more restrictions on which toolchain to use, which opens up the door to port Lyra onto different platforms. The full list of features and fixes include:

    • Release sparse_matmul library code and remove pre-compiled dynamic library binaries.
    • Add support for the Bazel default gcc toolchain on linux, and make this the default instead of the clang toolchain.
    • Fix noise bursts at the beginning of output audio files.
    • Abstract out UnitFloatToInt16Scalar, UnitFloatToInt16 and Int16ToUnitFloat functions.
    • Provide operator<< to unique_ptr to be used with CHECK() macros.
    • Fix float distribution compatibility in benchmark_decode_lib.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Apr 6, 2021)

Owner
Google
Google ❤️ Open Source
Google
Speech Algorithms Collections

Speech Algorithms Collections

Ryuk 441 Sep 18, 2022
Through hole PCB version of the HAGIWO 005 Generative Sequencer Eurorack module.

HAGIWO 005 Eurorack Sequencer PCB and Code Through hole PCB version of the HAGIWO 005 Generative Sequencer Eurorack module. The module is a very simpl

null 10 Aug 17, 2022
This speech synthesizer is actually the SAM speech synthesizer in an ESP8266

SSSSAM Serial Speech Synthesizer SAM This speech synthesizer is actually the SAM speech synthesizer in an ESP8266. Where SAM was a software applicatio

Jan 11 Jul 30, 2022
LFAC - Low Fidelity Audio Codec

LFAC - Low-Fidelity Audio Codec Copyright 2021 Jari Komppa, http://iki.fi/sol Licensed under Unlicense. Not to be confused with FLAC. What is this? Do

Jari Komppa 15 Sep 18, 2022
Legion Low Level Rendering Interface provides a graphics API agnostic rendering interface with minimal CPU overhead and low level access to verbose GPU operations.

Legion-LLRI Legion-LLRI, or “Legion Low Level Rendering Interface” is a rendering API that aims to provide a graphics API agnostic approach to graphic

Rythe Interactive 26 Aug 13, 2022
CC2500 Low-Cost Low-Power 2.4 GHz RF Transceiver driver for esp-idf

esp-idf-cc2500 CC2500 Low-Cost Low-Power 2.4 GHz RF Transceiver driver for esp-idf. I ported from this. 2.00mm pitch External Antena 1.27mm pitch PCB

null 3 May 29, 2022
Simple Binary Encoding (SBE) - High Performance Message Codec

Simple Binary Encoding (SBE) SBE is an OSI layer 6 presentation for encoding and decoding binary application messages for low-latency financial applic

Real Logic 2.7k Sep 17, 2022
Open h.265 video codec implementation.

libde265 - open h.265 codec implementation libde265 is an open source implementation of the h.265 video codec. It is written from scratch and has a pl

struktur AG 1.4k Sep 19, 2022
Open Source H.264 Codec

OpenH264 OpenH264 is a codec library which supports H.264 encoding and decoding. It is suitable for use in real time applications such as WebRTC. See

Cisco Systems 4.7k Sep 22, 2022
hessian2-codec it is a complete C++ implementation of hessian2 spec

hessian2-codec is a C++ library from Alibaba for hessian2 codec. It is a complete C++ implementation of hessian2 spec. Because it was originally intended to implement the Dubbo Filter of Envoy, it did not provide good support for serialization of user-defined types (there is only one way to implement user-defined types using ADL, but it is not very complete and does not support nested types well). At the moment it is simply deserializing content into some C++ intermediate types.

Alibaba 16 Sep 11, 2022
Lossless data compression codec with LZMA-like ratios but 1.5x-8x faster decompression speed, C/C++

LZHAM - Lossless Data Compression Codec Public Domain (see LICENSE) LZHAM is a lossless data compression codec written in C/C++ (specifically C++03),

Rich Geldreich 635 Sep 21, 2022
A bespoke sample compression codec for 64k intros

pulsejet A bespoke sample compression codec for 64K intros codec pulsejet lifts a lot of ideas from Opus, and more specifically, its CELT layer, which

logicoma 34 Jul 25, 2022
A free, fast, cross-platform volumetric codec for everyone.

The open source Universal Volumetric (".uvol") compressed interchange format for streaming mesh sequences. This project also includes a cross-platform player implementation using h.264 video for texture.

XR Foundation 73 Sep 5, 2022
Python bindings of silk codec.

Python silk module. --- pysilk --- APIs See test\test.py. import pysilk as m m.silkEncode(buf , 24000) m.silkDecode(buf , 24000) #the first param is b

DCZ_Yewen 15 Jul 2, 2022
Cross-platform silk codec wrap library depends on ploverlake/silk.

libSilkCodec Cross-platform silk codec wrap library depends on ploverlake/silk. Clone & Build Linux/Unix like # clone $ git clone https://github.c

KonataDev 8 Sep 9, 2022
ffmpeg supporting EVC codec and file formats.

ffevc ffmpeg supporting EVC codec and file formats. MPEG-5 Essential Video Coding (EVC) integration with FFmpeg project. It is supported under Linux a

MPEG-5 26 Aug 15, 2022
Nuvoton codec/amp driver with different ASoC version

Nuvoton-ASoC Nuvoton codec and amp drivers with different ASoC versions. The Nuvoton-ASoC repository stores the Linux driver for codec and amplifier m

Nuvoton Technology Corp. 1 Aug 31, 2022
Basis Universal GPU Texture Codec

basis_universal Basis Universal Supercompressed GPU Texture Codec Basis Universal is a "supercompressed" GPU texture data interchange system that supp

null 2.2k Sep 23, 2022
Facebook AI Research's Automatic Speech Recognition Toolkit

wav2letter++ Important Note: wav2letter has been moved and consolidated into Flashlight in the ASR application. Future wav2letter development will occ

Facebook Research 6.1k Sep 19, 2022
🐸 Coqui STT is an open source Speech-to-Text toolkit which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers

Coqui STT ( ?? STT) is an open-source deep-learning toolkit for training and deploying speech-to-text models. ?? STT is battle tested in both producti

Coqui.ai 1.5k Sep 24, 2022