Ported lstm op from TF GitHub
BUG= microspeech model with lstm op on HiFi5 DSP

Signed-off-by: Lukman Rahumathulla <[email protected]>
Signed-off-by: Manjunath CP <[email protected]>
Signed-off-by: Niranjan Yadla <[email protected]>
Signed-off-by: Raj Pawate <[email protected]>

Hey, so I was attempting to build the hello world example for RISC-V target and then use Spike to run the subsequent binary file. These are the commands I used to do this:

1. git clone https://github.com/tensorflow/tflite-micro.git
2. cd tflite-micro
3. make -f tensorflow/lite/micro/tools/make/Makefile third_party_downloads
4. Change the target from riscv32_mcu to mcu_riscv in /tensorflow/lite/micro/tools/make/mcu_riscv_makefile.inc and in /tensorflow/lite/micro/examples/hello_world/riscv32_mcu/Makefile.inc
5. Moved the flags -fnothreadsafe-statics and -fno-use-cxa-atexit from PLATFORM FLAGS to CXX FLAGS in /tensorflow/lite/micro/tools/make/targets/mcu_riscv_makefile.inc
6. Changed -c11 to -gnu11 under CCFLAGS in /tensorflow/lite/micro/tools/make/Makefile
7. Changed the functions fmin and fmax to min and max respectively in /tensorflow/lite/kernels/internal/min.h and /tensorflow/lite/kernels/internal/max.h.
8. Run the command make -f tensorflow/lite/micro/tools/make/Makefile TARGET=mcu_riscv TARGET_ARCH=riscv32_mcu hello_world_bin
9. Run the binary using Spike: spike --isa=rv32imafdc pk tensorflow/lite/micro/tools/make/gen/mcu_riscv_riscv32_mcu_default/bin/hello_world

I've checked that the binary generated is indeed ELF 32-bit RISC-V executable, but still on running the final command, it shows bbl loader couldn't open ELF program: tensorflow/lite/micro/tools/make/gen/mcu_riscv_riscv32_mcu_default/bin/hello_world!

Can anyone let me know what I'm doing wrong in this case??

I'm seeing build errors with the different TFLM build attempts I've been making. I have not tried a Bazel build, if there are build instructions on how to do a Bazel build for this repo on Windows I would try that, currently I am trying to build with a Make project. I am successfully building Tensorflow Lite (from the main Tensorflow repo) with CMake, but with that build I don't have clarity on what is just needed for TFLM.

Build attempt #1:

Following the first Micro Speech example "Deploy to ARC EM SDP" found here: https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/examples/micro_speech

Command line commands from an empty folder: git clone https://github.com/tensorflow/tflite-micro tflite-micro cd tflite-micro make -f tensorflow/lite/micro/tools/make/Makefile TARGET=arc_emsdp ARC_TAGS=reduce_codesize OPTIMIZED_KERNEL_DIR=arc_mli generate_micro_speech_mock_make_project

Result: Several packages are downloaded, but the end result is "make: *** No rule to make target 'generate_micro_speech_mock_make_project'. Stop."

Build attempt #2:

Following the Micro Speech example "Deploy to ESP32" found here: https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/examples/micro_speech

ESP IDF should be properly installed.

Command line commands from an empty folder: git clone https://github.com/tensorflow/tflite-micro tflite-micro cd tflite-micro make -f tensorflow/lite/micro/tools/make/Makefile TARGET=esp generate_micro_speech_esp_project

Result: There are a couple "FIND: Parameter format not correct" messages and "File not found - *.cc", "File not found - *.h". It appears that pigweed is downloaded, along with some other packages. Ultimately, the build fails with "make: *** No rule to make target 'generate_micro_speech_esp_project'. Stop.".

Build attempt #3:

Following the Micro Speech example "Deploy to Sparkfun Edge" found here: https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/examples/micro_speech

Command line commands from an empty folder: git clone https://github.com/tensorflow/tflite-micro tflite-micro cd tflite-micro make -f tensorflow/lite/micro/tools/make/Makefile TARGET=sparkfun_edge OPTIMIZED_KERNEL_DIR=cmsis_nn micro_speech_bin

Results: Essentially the same results as build attempt #2 above.

Build attempt #4:

Following the Micro Speech example "Deploy to STM32F746" found here: https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/examples/micro_speech

Command line commands from an empty folder: git clone https://github.com/tensorflow/tflite-micro tflite-micro cd tflite-micro make -f tensorflow/lite/micro/tools/make/Makefile TARGET=disco_f746ng OPTIMIZED_KERNEL_DIR=cmsis_nn generate_micro_speech_mbed_project

Same results as above.

Build attempt #5:

Following the Hello World example "Deploy to SparkFun Edge" found here: https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/examples/hello_world

Command line commands from an empty folder: git clone https://github.com/tensorflow/tflite-micro tflite-micro cd tflite-micro make -f tensorflow/lite/micro/tools/make/Makefile TARGET=sparkfun_edge hello_world_bin

Same results as above.

Arduino 33 BLE Sense Tensorflow 2.5.0

I have the following model, on Linux this model classifies all 20 images in an Acute Lymphoblastic Leukemia test set correctly:

micro_op_resolver.AddAveragePool2D(); micro_op_resolver.AddConv2D(); micro_op_resolver.AddDepthwiseConv2D(); micro_op_resolver.AddReshape(); micro_op_resolver.AddFullyConnected(); micro_op_resolver.AddSoftmax();

On Arduino 33 BLE Sense I am using JpegDecoder to decode the images from an SD and pass them to input->data.int8, I have also tested with the instructions for converting to c array in the following link: https://github.com/tensorflow/tflite-micro/blob/f583f92992c3c9dfb8e10f36d66b2fe7267cf7bc/tensorflow/lite/micro/examples/person_detection/person_image_data.h#L17-L20 with exact same results.

It is not working well :D On Arduino I am always receiving an integer for positive and the negative of the integer for negative. IE:

Positive image: Positive score: 111 Negative score: -111

Negative image: Positive score: 74 Negative score: -74

Nearly always 74, -74 for negative and nearly always 111, -111 for positive.

At first I was simply reading the image in using SD.open ........ jpegFile.read() and looping through assigning to input->data.int8, but then I made a version that uses decodeSdFile and made a script based on the person detector project that does the grayscaling etc, but either way gets exactly the same results as above.

Do you have any suggestions as to why this is happening, I don't have the experience with Arduino ML to work this out. Thanks in advance.

In the latest source code of tflite-micro, TARGET_ARCH and BUILD_TYPE could not be override by Makefile of selected TARGET, this commit is trying to fix this issue introduced by PR #432

Thanks Huaqi

Oreilly TinyML example build issue: page 84 OS Windows x64 (cygwin)

make -f tensorflow/lite/micro/tools/make/Makefile test_hello_world_test

Log: $ make -f tensorflow/lite/micro/tools/make/Makefile test_hello_world_test FIND: Parameter format not correct FIND: Parameter format not correct File not found - *.cc File not found - *.h tensorflow/lite/micro/tools/make/downloads/flatbuffers already exists, skipping the download. tensorflow/lite/micro/tools/make/downloads/pigweed already exists, skipping the download. tensorflow/lite/micro/tools/make/downloads/person_model_int8 already exists, skipping the download. make: *** No rule to make target 'test_hello_world_test'. Stop.

Some CMSIS-NN kernels are still having uint8 support.

Remove uint8 support for remaining CMSIS-NN kernels.
Remove uint8 support for equivalent reference kernels.
Remove uint8 unit tests for equivalent kernels.
Replace uint8 default model in network tester to a int8 model.

This is a fix for: https://github.com/tensorflow/tflite-micro/issues/108

This was previously merged and reverted in the old repo because of some unknown issue. There was also a revert of the reverted PR in the old repo, which this is based on. Here is that PR for more background: https://github.com/tensorflow/tensorflow/pull/49485

• adding toolchain root so that the generated project can be set to the downloaded toolchain.

make -f tensorflow/lite/micro/tools/make/Makefile generate_hello_world_make_project TARGET=cortex_m_generic TARGET_ARCH=cortex-m4+fp TENSORFLOW_ROOT=/path/to/tflite-micro

the generated makefile will now have the correct path for the compiler

in tools/make/gen/cortexm4_cortex-m4_micro/prj/xxxx/Makefile

  TARGET_TOOLCHAIN_ROOT := <path-to-tflite-micro>/tensorflow/lite/micro/tools/make/downloads/gcc_embedded/bin/
  TARGET_TOOLCHAIN_PREFIX := arm-none-eabi-
  
  # These are microcontroller-specific rules for converting the ELF output
  # of the linker into a binary image that can be loaded directly.
  CXX             := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)g++'
  CC              := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)gcc'
  AS              := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)as'
  AR              := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)ar'
  LD              := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)ld'
  NM              := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)nm'
  OBJDUMP         := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)objdump'
  OBJCOPY         := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)objcopy'
  SIZE            := '$(TARGET_TOOLCHAIN_ROOT)$(TARGET_TOOLCHAIN_PREFIX)size'

BUG=fix makefile project generation.

While compiling the hello_world code for development machine, I keep getting the following error.

tensorflow/lite//micro/sparkfun_edge/system_setup.cc:22:10: fatal error: 'am_bsp.h' file not found

Wondering if it is just me or is it a known bug.

@advaitjain , @njeffrie, @petewarden, @nyadla-sys

Describe the problem

TFL int16 reference code compared to float implementation is off by 3-4 bits (int8 is off by 1 bit). TFLM does not have int16 support yet. Sigmoid shows maxdiff of 6 (3 bit). Tanh shows maxdiff of 12 (4 bit). Cadence’s hardware implementation is 1 bit off and that’s how it was discovered that TFLM implementation was off A bug with test vectors will help the TFLM team figure out what needs to be done here

Source code / logs

input_file_sigmoid_Q12.txt input_file_tanh_Q12.txt Attached couple of files with input data which shows maximum difference. This input list is for Q12 (1 sign bit, 3 integer bits, 12 fractional bits). We also observed 3-4 bit difference for other Q format as well (Q11 and Q15).

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template

System information

• TensorFlow version (you are using): 2.5.0
• Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state. Currently, when using an interpreter in TFLM you need to manually add the desired operators, or use an AllOpsResolver. The first option is tedious, and the other has a significantly larger memory footprint.

We propose an automatic way to generate a MicroMutableOpResolver with only the used operators in a given model.

Will this change the current api? How? It can be a standalone script that takes a tflite file as input, which doesn't affect the existing API. Alternatively, it can be done at the convert stage, however, this limits the usage, as you cannot get the MicroMutableOpResolver from an existing tflite model.

Who will benefit with this feature? Most TFLM users.

Any Other info.

Several essential classes and functions are missing code comments that would be helpful for new users who want to run inference with TFLM.

I've already written several API doc comments for the coralmicro project, so I'll offer these in a pull request.

These code comments cover the essential APIs needed to setup a MicroInterpreter and run inference with TFLM. The comments are written in a style that's intended for parsing into HTML with Doxygen. (RE: issue #1640)

I originally created these code comments for the coralmicro API reference. (See info about building the API docs.)

Basically all of the code comments are still the same as created for coralmicro, except I removed the ErrorReporter parameter from the MicroInterpreter constructor (no longer used in tflite-micro) and made a few new small copyedits.

Given same tflite model, I have attempted to benchmark using tflite and tflite-micro. tflite benchmark https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark tflite micro benchmark https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/benchmarks However, there is a significant difference on its performance. Using same tflite model, tflite benchmark gives about 40ms for one interpreter->invoke process, while tflite micro benchmark gives about 1200ms for one interpreter->invoke process. Test log is attached. benchmark_log_from_tflite.log benchmark_log_from_tflite_micro.log

Machine system info : x86_64 GNU/Linux Ubuntu22.04

I question if this difference is expected. Any comments would be appreciated.

Updated HiFi LSTM optimizations to latest refactored LSTM reference code. All kelnel_lstm_eval_test (10/10) and unidirectional_sequence_lstm_test (8/8) are passing okay.

I am going to the link that was posted in this discussion. However, when I open the link, it can't open the file. I also can't find create_sine_model.ipynb anywhere else.