Openvino tensorflow - OpenVINO™ integration with TensorFlow

Overview

English | 简体中文

OpenVINO™ integration with TensorFlow

This repository contains the source code of OpenVINO™ integration with TensorFlow, designed for TensorFlow* developers who want to get started with OpenVINO™ in their inferencing applications. TensorFlow* developers can now take advantage of OpenVINO™ toolkit optimizations with TensorFlow inference applications across a wide range of Intel® compute devices by adding just two lines of code.

import openvino_tensorflow
openvino_tensorflow.set_backend('<backend_name>')

This product delivers OpenVINO™ inline optimizations which enhance inferencing performance with minimal code modifications. OpenVINO™ integration with TensorFlow accelerates inference across many AI models on a variety of Intel® silicon such as:

  • Intel® CPUs
  • Intel® integrated GPUs
  • Intel® Movidius™ Vision Processing Units - referred to as VPU
  • Intel® Vision Accelerator Design with 8 Intel Movidius™ MyriadX VPUs - referred to as VAD-M or HDDL

[Note: For maximum performance, efficiency, tooling customization, and hardware control, we recommend the developers to adopt native OpenVINO™ APIs and its runtime.]

Installation

Prerequisites

  • Ubuntu 18.04, 20.04, macOS 11.2.3 or Windows1 10 - 64 bit
  • Python* 3.7, 3.8 or 3.9
  • TensorFlow* v2.7.0

1Windows package is released in Beta preview mode and currently supports only Python3.9

Check our Interactive Installation Table for a menu of installation options. The table will help you configure the installation process.

The OpenVINO™ integration with TensorFlow package comes with pre-built libraries of OpenVINO™ version 2021.4.2. The users do not have to install OpenVINO™ separately. This package supports:

  • Intel® CPUs

  • Intel® integrated GPUs

  • Intel® Movidius™ Vision Processing Units (VPUs)

      pip3 install -U pip
      pip3 install tensorflow==2.7.0
      pip3 install -U openvino-tensorflow
    

For installation instructions on Windows please refer to OpenVINO™ integration with TensorFlow for Windows

To use Intel® integrated GPUs for inference, make sure to install the Intel® Graphics Compute Runtime for OpenCL™ drivers

To leverage Intel® Vision Accelerator Design with Movidius™ (VAD-M) for inference, install OpenVINO™ integration with TensorFlow alongside the Intel® Distribution of OpenVINO™ Toolkit.

For more details on installation please refer to INSTALL.md, and for build from source options please refer to BUILD.md

Configuration

Once you've installed OpenVINO™ integration with TensorFlow, you can use TensorFlow* to run inference using a trained model.

For the best results, it is advised to enable oneDNN Deep Neural Network Library (oneDNN) by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.

To see if OpenVINO™ integration with TensorFlow is properly installed, run

python3 -c "import tensorflow as tf; print('TensorFlow version: ',tf.__version__);\
            import openvino_tensorflow; print(openvino_tensorflow.__version__)"

This should produce an output like:

    TensorFlow version:  2.7.0
    OpenVINO integration with TensorFlow version: b'1.1.0'
    OpenVINO version used for this build: b'2021.4.2'
    TensorFlow version used for this build: v2.7.0
    CXX11_ABI flag used for this build: 0

By default, Intel® CPU is used to run inference. However, you can change the default option to either Intel® integrated GPU or Intel® VPU for AI inferencing. Invoke the following function to change the hardware on which inferencing is done.

openvino_tensorflow.set_backend('<backend_name>')

Supported backends include 'CPU', 'GPU', 'GPU_FP16', 'MYRIAD', and 'VAD-M'.

To determine what processing units are available on your system for inference, use the following function:

openvino_tensorflow.list_backends()

For more API calls and environment variables, see USAGE.md.

[Note: If a CUDA capable device is present in the system then set the environment variable CUDA_VISIBLE_DEVICES to -1]

Examples

To see what you can do with OpenVINO™ integration with TensorFlow, explore the demos located in the examples directory.

Try it on Intel® DevCloud

Sample tutorials are also hosted on Intel® DevCloud. The demo applications are implemented using Jupyter Notebooks. You can interactively execute them on Intel® DevCloud nodes, compare the results of OpenVINO™ integration with TensorFlow, native TensorFlow and OpenVINO™.

License

OpenVINO™ integration with TensorFlow is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

Support

Submit your questions, feature requests and bug reports via GitHub issues.

How to Contribute

We welcome community contributions to OpenVINO™ integration with TensorFlow. If you have an idea for improvement:

We will review your contribution as soon as possible. If any additional fixes or modifications are necessary, we will guide you and provide feedback. Before you make your contribution, make sure you can build OpenVINO™ integration with TensorFlow and run all the examples with your fix/patch. If you want to introduce a large feature, create test cases for your feature. Upon our verification of your pull request, we will merge it to the repository provided that the pull request has met the above mentioned requirements and proved acceptable.


* Other names and brands may be claimed as the property of others.

Comments
  • Slower performance than tf

    Slower performance than tf

    Hi, I'm trying to use openvino_tensorflow on my core i7-6700 CPU but if I use openvino tensorflow is around 10x slower than using only tf. OS : ubuntu 20.04 CPU: core i7-6700

    I just installed openvino_tensorflow and if I run my code using only TF I got an inference time of 0.10 s. Then I add the line

    import openvino_tensorflow openvino_tensorflow.set_backend('CPU')

    My inference time is 1.5 s, log says 2022-07-11 09:50:48.690843: OVTF Summary -> 90 out of 624 nodes in the graph (14%) are now running with OpenVINO™ backend

    opened by jams01 6
  • Entry Point not Found

    Entry Point not Found

    After installing openvino_tensorflow, the message "Entry Point not Found" shows up on the command python "import openvino_tensorflow as ovtf" Screenshot 2022-07-04 230310

    Installation: python 3.9,windows10 install with to pip3 install -U pip pip3 install tensorflow==2.8.0 pip3 install openvino-tensorflow==2.0.0

    Test under python: import openvino_tensorflow See image for error message

    Did I missed something simple?

    opened by terryll 6
  • ERROR: tensorflow-2.9.1-cp39-cp39-win_amd64.whl is not a supported wheel on this platform.

    ERROR: tensorflow-2.9.1-cp39-cp39-win_amd64.whl is not a supported wheel on this platform.

    Greetings,

    When I tried to install openvino_tensorlow, the following error came out:

    ERROR: tensorflow-2.9.1-cp39-cp39-win_amd64.whl is not a supported wheel on this platform.

    Any clue? Thanks in advance

    my system:

    windows 10 python 3.8, in the venv Im trying to install pip command: pip install

    I try changing 3.8 with 3.9, NOTHING....

    Had anyone installed openvino_tensorflow in Windows 10 x64?

    Thanks in advance

    opened by davemaster 3
  • Enable OneDNN Optimizations by default

    Enable OneDNN Optimizations by default

    • Remove a few unused imports in init.py
    • Enable OneDNN when OVTF is imported, unless the environmental variable TF_ENABLE_ONEDNN_OPTS is already set
    • Add APIs to enable or disable OneDNN through OVTF
    opened by sspintel 3
  • libtbb.so.2: cannot open shared object file: No such file or directory

    libtbb.so.2: cannot open shared object file: No such file or directory

    when I pip installed the latest version of of tensorflow 2.91 along with openvino-tensorflow 2.1.0 I get this errow that is thrown when importing openvino_tensorflow.
    libtbb.so.2: cannot open shared object file: No such file or directory the libopenvino is looking for libtbb.so.2 whereas the latest libtbb is libtbb.so.12 which is found in the lib folder I am surprised this has not been tested !!!

    opened by ShankarRIntel 2
  • openvino_tensorflow.dll not found

    openvino_tensorflow.dll not found

    while running following command after installing openvino_tensorflow.

    python -c "import openvino_tensorflow; print(openvino_tensorflow.version)"

    tensorflow.python.framework.errors_impl.NotFoundError: XXXXXXXXXXXXXXXXX \lib\site-packages\openvino_tensorflow\openvino_tensorflow.dll not found

    no issues with tensorflow:

    python -c "import tensorflow as tf; print('TensorFlow version: ',tf.version)" 2022-04-07 08:00:06.091197: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found 2022-04-07 08:00:06.091694: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. TensorFlow version: 2.8.0

    opened by dineshnaikc 2
  • [SSD Models] Segmentation Fault

    [SSD Models] Segmentation Fault

    We observed that the program fails whenever the same image has been passed to sess.run() more than once. This error only happens with the SSD object detection models. For example:

    Error occurs here:

    OVTF Summary -> 2093 out of 4245 nodes in the graph (49%) are now running with OpenVINO™ backend
    Predict Image [10]
    Predict Image [89]
    Predict Image [10] --> Segmentation fault
    

    Error doesn't occur if we reinitialise the session:

    OVTF Summary -> 2093 out of 4245 nodes in the graph (49%) are now running with OpenVINO™ backend
    Predict Image [10]
    Predict Image [89]
    OVTF Summary -> 2093 out of 4245 nodes in the graph (49%) are now running with OpenVINO™ backend
    Predict Image [10] 
    ...
    

    Here is a code snippet for where the error was at:

    Model: ssd_inception_v2,ssd_mobilenet_v1,ssd_mobilenet_v1_fpn,ssd_mobilenet_v2,ssd_resnet_50_fpn, ssdlite_mobilenet_v2 TensorFlow version: 2.6.0 openvino-tensorflow version: 1.0.0

    class BackendTensorflowOpenvino(...)
      ...
      def load(self, model_path, inputs=None, outputs=None):
          # there is no input/output meta data i the graph so it need to come from config.
          if not inputs:
              raise ValueError("BackendTensorflow needs inputs")
          if not outputs:
              raise ValueError("BackendTensorflow needs outputs")
          self.outputs = outputs
          self.inputs = inputs
    
          infer_config = tf.compat.v1.ConfigProto()
          infer_config.intra_op_parallelism_threads = int(os.environ['TF_INTRA_OP_PARALLELISM_THREADS']) \
                  if 'TF_INTRA_OP_PARALLELISM_THREADS' in os.environ else os.cpu_count()
          infer_config.inter_op_parallelism_threads = int(os.environ['TF_INTER_OP_PARALLELISM_THREADS']) \
                  if 'TF_INTER_OP_PARALLELISM_THREADS' in os.environ else os.cpu_count()
          infer_config.use_per_session_threads = 1
    
          graph_def = tf.compat.v1.GraphDef()
          with tf.compat.v1.gfile.FastGFile(model_path, "rb") as f:
              graph_def.ParseFromString(f.read())
              g = tf.compat.v1.import_graph_def(graph_def, name='')
              
          self.sess = tf.compat.v1.Session(graph=g, config=infer_config)
          return self
    
      def predict(self, feed):
          ans = self.sess.run(self.outputs, feed_dict=feed)
          return ans
    
    import openvino_tensorflow as ovtf
    ovtf.set_backend('CPU')
    model = BackendTensorflowOpenvino(...)
    model.predict({"image_tensor:0": img1})
    model.predict({"image_tensor:0": img1})
    

    The program fails at the second model.predict() call. Error Message:

    ./tmp-e9_0_ipe.sh: line 42: 65995 Segmentation fault      (core dumped)
    

    Also, the above code works with the original tensorflow backend and other models ( e.g. faster_rcnn_inception_v2_coco, faster_rcnn_resnet50_coco) from TensorFlow Object Detection Model Zoo and yolo-v3.


    To reproduce the error, we can use ck - an automated workflow for designing ML systems.

    pip install ck
    

    Then, pull the relevant program:

    ck pull repo --url=https://github.com/krai/ck-mlperf.git
    ck pull repo --url=https://github.com/krai/ck-object-detection.git
    

    Then, follow the building instruction here. And finally, run the following command

    time docker run -it --rm ${CK_IMAGE} \
    "ck run program:mlperf-inference-vision --cmd_key=direct --skip_print_timers \
      --env.CK_LOADGEN_SCENARIO=SingleStream \
      --env.CK_LOADGEN_MODE='--accuracy' \
      --env.CK_LOADGEN_EXTRA_PARAMS='--count 50' \
      \
      --env.CK_MODEL_PROFILE=default_tf_object_det_zoo \
      --dep_add_tags.weights=ssd_mobilenet_v1_coco \
      \
      --env.CK_INFERENCE_ENGINE=tensorflow \
      --env.CK_INFERENCE_ENGINE_BACKEND=openvino-cpu \
      --env.CUDA_VISIBLE_DEVICES=-1"
    
    opened by elimkwan 2
  • Cannot run classification_sample example

    Cannot run classification_sample example

    On my machine, the classification_sample example returns this error :

    _Create kernel failed: Not found: No registered 'nGraphEncapsulate' OpKernel for 'GPU' devices compatible with node

    I am able to successfully run this command :

    python3 -c "import tensorflow as tf; print('TensorFlow version: ',tf.version);
    import openvino_tensorflow; print(openvino_tensorflow.version)"

    The full error message is :

    $ python3 examples/classification_sample.py 2021-09-27 14:45:03.705017: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 Available Backends: CPU MYRIAD 2021-09-27 14:45:04.998781: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-09-27 14:45:04.999214: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1 2021-09-27 14:45:05.005135: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-27 14:45:05.005512: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: pciBusID: 0000:01:00.0 name: Quadro M2200 computeCapability: 5.2 coreClock: 1.036GHz coreCount: 8 deviceMemorySize: 3.94GiB deviceMemoryBandwidth: 82.08GiB/s 2021-09-27 14:45:05.005547: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 2021-09-27 14:45:05.008327: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11 2021-09-27 14:45:05.008417: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11 2021-09-27 14:45:05.009473: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10 2021-09-27 14:45:05.009687: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10 2021-09-27 14:45:05.010126: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.11 2021-09-27 14:45:05.010675: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11 2021-09-27 14:45:05.010834: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8 2021-09-27 14:45:05.010936: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-27 14:45:05.011675: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-27 14:45:05.012200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0 2021-09-27 14:45:05.012250: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 2021-09-27 14:45:05.344618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-09-27 14:45:05.344663: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0 2021-09-27 14:45:05.344670: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N 2021-09-27 14:45:05.344869: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-27 14:45:05.345291: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-27 14:45:05.345681: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-09-27 14:45:05.346057: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3113 MB memory) -> physical GPU (device: 0, name: Quadro M2200, pci bus id: 0000:01:00.0, compute capability: 5.2) 2021-09-27 14:45:05.380470: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 3000000000 Hz 2021-09-27 14:45:06.246353: OVTF Summary -> 476 out of 480 nodes in the graph (99%) are now running with OpenVINO™ backend 2021-09-27 14:45:06.289576: E tensorflow/core/framework/op_segment.cc:54] Create kernel failed: Not found: No registered '_nGraphEncapsulate' OpKernel for 'GPU' devices compatible with node {{node ovtf_cluster_0}} . Registered: device='CPU'

    opened by benoit-robotics 2
  • Issues with convert_yolov3 in aditya/upgrade_samples branch

    Issues with convert_yolov3 in aditya/upgrade_samples branch

    Connecting to pjreddie.com (pjreddie.com)|128.208.4.108|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 248007048 (237M) [application/octet-stream] Saving to: ‘weights/yolov3.weights’

    weights/yolov3.weights 100%[=================================================>] 236.52M 40.8MB/s in 6.9s

    2021-09-03 10:17:16 (34.3 MB/s) - ‘weights/yolov3.weights’ saved [248007048/248007048]

    ./convert_yolov3.sh: line 17: python: command not found ./convert_yolov3.sh: line 18: python: command not found cp: cannot stat 'weights/yolo_v3_darknet_2.pb': No such file or directory cp: cannot stat 'weights/darknet53.h5': No such file or directory

    opened by vibhubithar 2
  • Cluster splitting added for dynamic to static data flow

    Cluster splitting added for dynamic to static data flow

    DO NOT MERGE YET

    • Cluster splitting if dynamic to static data flow exists
    • Deassigning clusters for dynamic to static data flow removed
    • Deassigning clusters due to Prod op removed
    • StridedSlice translation throws error if it produces zero dimension output.
    opened by cavusmustafa 2
  • transpose_sinking pass

    transpose_sinking pass

    I find it will crash when I use transpose_sinking pass through a ngraph function which only have one Conv op and two inserted transpose op which is used to handle the nhwc input and output. I suppose it should do nothing in this situation instead of crash.

    opened by lisa0314 2
  • Object Detection Example SegFault on 12th Gen Core Processor with GPU Backend

    Object Detection Example SegFault on 12th Gen Core Processor with GPU Backend

    I was able to complete the example on 12th Gen Core Processor using CPU backend. But when I change to GPU backend, the example SegFault.

    I have verified that OpenVINO 2022.1 was correctly installed by running the OpenVINO benchmark_app against the GPU device.

    2022-07-21 15:47:12.926336: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
    2022-07-21 15:47:12.929013: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
    2022-07-21 15:47:12.929026: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
    2022-07-21 15:47:14.114683: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2022.1.0.643/extras/opencv/python/cv2/../../bin:/opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
    2022-07-21 15:47:14.114703: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
    2022-07-21 15:47:14.114715: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (my-adl-s): /proc/driver/nvidia/version does not exist
    2022-07-21 15:47:14.114839: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
    To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
    Available Backends:
    CPU
    GPU
    2022-07-21 15:47:20.447067: OVTF Summary -> 59 out of 1326 nodes in the graph (4%) are now running with OpenVINO™ backend
    2022-07-21 15:47:21.487357: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
    Inference time in ms: 205.22
    person 0.98
    tie 0.81
    Output image is saved in detections.jpg
    
    2022-07-21 15:47:23.855600: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
    2022-07-21 15:47:23.858267: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
    2022-07-21 15:47:23.858281: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
    2022-07-21 15:47:25.036728: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2022.1.0.643/extras/opencv/python/cv2/../../bin:/opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
    2022-07-21 15:47:25.036750: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
    2022-07-21 15:47:25.036761: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (my-adl-s): /proc/driver/nvidia/version does not exist
    2022-07-21 15:47:25.036889: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
    To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
    Available Backends:
    CPU
    GPU
    2022-07-21 15:47:31.407075: OVTF Summary -> 59 out of 1326 nodes in the graph (4%) are now running with OpenVINO™ backend
    Segmentation fault (core dumped)
    
    opened by alexlamfromhome 0
Releases(v2.2.0)
Owner
OpenVINO Toolkit
OpenVINO Toolkit
OpenVINO™ Toolkit repository

This toolkit allows developers to deploy pre-trained deep learning models through a high-level C++ Inference Engine API integrated with application logic.

OpenVINO Toolkit 3.6k Sep 29, 2022
OpenVINO™ optimization for PointPillars*

OpenVINO™ optimization for PointPillars* There are 2 demonstrations in the repo. Demo of PointPillars Optimization - It demonstrates how to implement

Intel Corporation 13 Aug 29, 2022
Number recognition with MNIST on Raspberry Pi Pico + TensorFlow Lite for Microcontrollers

About Number recognition with MNIST on Raspberry Pi Pico + TensorFlow Lite for Microcontrollers Device Raspberry Pi Pico LCDディスプレイ 2.8"240x320 SPI TFT

iwatake 49 Aug 16, 2022
Deep Learning API and Server in C++11 support for Caffe, Caffe2, PyTorch,TensorRT, Dlib, NCNN, Tensorflow, XGBoost and TSNE

Open Source Deep Learning Server & API DeepDetect (https://www.deepdetect.com/) is a machine learning API and server written in C++11. It makes state

JoliBrain 2.4k Oct 5, 2022
Eloquent interface to Tensorflow Lite for Microcontrollers

This Arduino library is here to simplify the deployment of Tensorflow Lite for Microcontrollers models to Arduino boards using the Arduino IDE.

null 177 Sep 27, 2022
TensorFlow Lite, Coral Edge TPU samples (Python/C++, Raspberry Pi/Windows/Linux).

TensorFlow Lite, Coral Edge TPU samples (Python/C++, Raspberry Pi/Windows/Linux).

Nobuo Tsukamoto 85 Sep 26, 2022
Want a faster ML processor? Do it yourself! -- A framework for playing with custom opcodes to accelerate TensorFlow Lite for Microcontrollers (TFLM).

CFU Playground Want a faster ML processor? Do it yourself! This project provides a framework that an engineer, intern, or student can use to design an

Google 302 Sep 25, 2022
TensorFlow Lite for Microcontrollers

TensorFlow Lite for Microcontrollers Build Status Official Builds Community Supported Builds Additional Documentation TensorFlow Lite for Microcontrol

null 892 Sep 25, 2022
Training and Evaluating Facial Classification Keras Models using the Tensorflow C API Implemented into a C++ Codebase.

CFace Training and Evaluating Facial Classification Keras Models using the Tensorflow C API Implemented into a C++ Codebase. Dependancies Tensorflow 2

null 8 Nov 23, 2021
KSAI Lite is a deep learning inference framework of kingsoft, based on tensorflow lite

KSAI Lite English | 简体中文 KSAI Lite是一个轻量级、灵活性强、高性能且易于扩展的深度学习推理框架,底层基于tensorflow lite,定位支持包括移动端、嵌入式以及服务器端在内的多硬件平台。 当前KSAI Lite已经应用在金山office内部业务中,并逐步支持金山

null 77 Sep 30, 2022
OpenEmbedding is an open source framework for Tensorflow distributed training acceleration.

OpenEmbedding English version | 中文版 About OpenEmbedding is an open-source framework for TensorFlow distributed training acceleration. Nowadays, many m

4Paradigm 19 Jul 25, 2022
Swapping face using Face Mesh with TensorFlow Lite

demo.mp4 Aiine Transform (アイン変換) Swapping face using FaceMesh. (could be used to unveil masked faces) Tested Environment Computer Windows 10 (x64) + V

iwatake 17 Apr 26, 2022
TensorFlow implementation of SQN based on RandLA-Net's encoder

SQN_tensorflow This repo is a TensorFlow implementation of Semantic Query Network (SQN). For Pytorch implementation, check our SQN_pytorch repo. Our i

PointCloudYC 8 Sep 5, 2022
PSTensor provides a way to hack the memory management of tensors in TensorFlow and PyTorch by defining your own C++ Tensor Class.

PSTensor : Custimized a Tensor Data Structure Compatible with PyTorch and TensorFlow. You may need this software in the following cases. Manage memory

Jiarui Fang 8 Feb 12, 2022
Movenet cpp deploy; model transformed from tensorflow

MoveNet-PaddleLite Adapted from PaddleDetection; Movenet cpp deploy based on PaddleLite; Movenet model transformed from tensorflow; 简介 Movenet是近年的优秀开源

null 10 Aug 12, 2022
A demo to run tensorflow-lite on Penglai TEE.

Documentation TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, a

IPADS 4 Dec 15, 2021
DeepRec is a recommendation engine based on TensorFlow.

DeepRec Introduction DeepRec is a recommendation engine based on TensorFlow 1.15, Intel-TensorFlow and NVIDIA-TensorFlow. Background Sparse model is a

Alibaba 594 Oct 4, 2022
Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite, TensorRT, OpenCV, ncnn, MNN, SNPE, Arm NN, NNAbla

InferenceHelper This is a helper class for deep learning frameworks especially for inference This class provides an interface to use various deep lear

iwatake 168 Oct 1, 2022
Benchmark framework of 3D integrated CIM accelerators for popular DNN inference, support both monolithic and heterogeneous 3D integration

3D+NeuroSim V1.0 The DNN+NeuroSim framework was developed by Prof. Shimeng Yu's group (Georgia Institute of Technology). The model is made publicly av

NeuroSim 10 Dec 21, 2021