Describe the bug While following the steps in Banana Tutorial under the section "Running the notebook" I am seeing this error appear:

"error": { "code": "ServiceError", "severity": null, "message": "AzureMLCompute job failed.\nServiceError: runTaskLetTask failed because: libffi.so.7: cannot open shared object file: No such file or directory\n\tReason: Job failed with non-zero exit Code", "messageFormat": null, "messageParameters": null, "referenceCode": null, "detailsUri": null, "target": null, "details": [], "innerError": null, "debugInfo": null, "additionalInfo": null }, "correlation": { "operation": "cf742550df05044dbf2b80b3397f31d7", "request": "8ae5bc2bda1f4e59" }, "environment": "australiaeast", "location": "australiaeast", "time": "2021-07-16T05:26:14.7993082+00:00", "componentName": "execution-worker"

} <

The note book I am running is SemanticsSegmentationUNet.ipynb

Screenshots

Hi, is there a way to add an inference script (like when deploying a model in azure ml) to do per/postprocessing? if not is there a way to access the frames of the camera video stream?

what I want to achieve is to use a pre-trained model deployed in a container to detect an object in an image frame and then do some postprocessing and send the result with the image frame to another deployed container

Describe the bug Following the PyTorch from Scratch Tutorial I get the following error, when trying to run detection through the mock-eye-module:

terminate called after throwing an instance of 'std::invalid_argument'  
  what():  stoi

The model used is pulled from openvino workbench zoo. The default ssd model to test the mock-eye-module container. Hence error should not be related to the model ssd_mobilenet_v2_coco.

Mock-eye-module by itself is compiling fine.

Based on simple debugging, error seems to be from the G-API when executed.

To Reproduce Steps to reproduce the behavior: Follow all steps in Prerequisites.

When converted model, and video person-bicycle-car-detection.mp4 downloaded.

I run the following on mac:

./scripts/compile_and_test.sh --video=test-artifacts/person-bicycle-car-detection.mp4 --weights=test-artifacts/ssd_mobilenet_v2_coco.bin --xml=test-artifacts/ssd_mobilenet_v2_coco.xml

I run the following on windows:

./scripts/compile_and_test.ps1 -ipaddr <your IP address> -xml test-artifacts/ssd_mobilenet_v2_coco.xml -video test-artifacts/person-bicycle-car-detection.mp4

Both return:

[setupvars.sh] OpenVINO environment initialized
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /opt/intel/openvino_2021.1.110/opencv (found version "4.5.0") found components:  gapi highgui 
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1") 
-- Checking for module 'gstreamer-1.0>=1.14'
--   Found gstreamer-1.0, version 1.16.2
-- Found InferenceEngine: /opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/libinference_engine.so (Required is at least version "2.0") 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/openvino/tmp/build
Scanning dependencies of target mock_eye_app
[ 14%] Building CXX object CMakeFiles/mock_eye_app.dir/main.cpp.o
[ 28%] Building CXX object CMakeFiles/mock_eye_app.dir/kernels/ssd_kernels.cpp.o
[ 42%] Building CXX object CMakeFiles/mock_eye_app.dir/kernels/utils.cpp.o
[ 57%] Building CXX object CMakeFiles/mock_eye_app.dir/modules/device.cpp.o
[ 71%] Building CXX object CMakeFiles/mock_eye_app.dir/modules/objectdetection/object_detectors.cpp.o
[ 85%] Building CXX object CMakeFiles/mock_eye_app.dir/modules/parser.cpp.o
[100%] Linking CXX executable mock_eye_app
[100%] Built target mock_eye_app
Cannot open labelfile /home/openvino/tmp/labels.txt
Labels will not be available.
terminate called after throwing an instance of 'std::invalid_argument'
  what():  stoi

Expected behavior Based on the documentation, a window should pop out and stream a video with object detection overplayed.

Additional context Additional observation The scripts compile_and_test.sh expects an "example" folder in the mock-eye-module, not included when cloning the repo. The "mock-eye-module-debug" pulls "openvino/ubuntu18_runtime:latest" instead of "openvino/ubuntu18_runtime:2021.1".

Step 1 of the tutorial fails.

It states "create a new folder in your [AML] workspace"

1. there is nothing called "workspace" but I do see a Azure Machine Learning (AML) Studio

2. There is no 'folder' option in the Create New menu

3. There really should be a link in step one to assist new developers to the correct 'workspace'.

Essential the entire tutorial is out of reach because step 1 cannot be completed.

Describe the bug Not sure if its a bug, my environment is not the same. Repeated error message below when doing the myriad_compile step E: [xLinkUsb] [ 367214] [myriad_compile] usb_find_device_with_bcd:266 Library has not been initialized when loaded

To Reproduce Run up to Step 9 of https://github.com/microsoft/azure-percept-advanced-development/blob/main/machine-learning-notebooks/train-from-scratch/SemanticSegmentationUNet.ipynb I'm doing the steps locally (not on a Azure ML remote compute instance). I have the model built locally. Next, I was taking the model thru the conversion steps in the openvino container. Container is openvino/ubuntu18_dev:2021.3 Prior steps, listed below, succeeded:

• pytorch model to onnx - OK
• onnx to IR - OK
• IR to blob - Fail

The exact command is

source /opt/intel/openvino_2021/bin/setupvars.sh 

/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/myriad_compile \                                                                                                                    
8    -m intel/bananas.xml -o intel/bananas.blob -VPU_NUMBER_OF_SHAVES 8 -VPU_NUMBER_OF_CMX_SLICES 8 -ip U8 -op FP32 

Thank you for taking a look.

Update: seems harmless as it generated a blob file.

This tutorial shows you how to:

1. Train a PyTorch model in AML (not really the focus of the tutorial)
2. Convert the model to OpenVINO
3. Double check it still works
4. Create a G-API graph for your custom model and run it on your PC
5. Transition that graph to the Percept DK and run the custom model there

Two things to note:

1. The neural network is not great. I have not tuned it yet, but I can do that in my downtime. I don't want to block getting the tutorial up waiting for me to tune the network.
2. I recommend people read the first tutorial before tackling this one.

Hi,

great news with this development sdk. However, I was wondering where you document the custom possibilities with the audio module. Would be interested to get access to the audio stream. Do you plan to have a similar module as with azureeymodule?

Thanks

While attempting the prerequisites, executing "run_workbench.ps1", I get this error.

`.\run_workbench.ps1 : File C:\percept\azure-percept-advanced-development\scripts\run_workbench.ps1 cannot be loaded. The file C:\percept\azure-percept-advanced-development\scripts\run_workbench.ps1 is not digitally signed. You cannot run this script on the current system. For more information about running scripts and setting execution policy, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170. At line:1 char:1

• .\run_workbench.ps1

•   + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess`
  
  

How do I digitally sign this script for you?

Bumps pillow from 8.1.2 to 8.2.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

If i create a brand new iothub with different name. how to i get azure percept to work again

i tried editing the device_connection_string in /etc/iotedge/config.yaml it connects to the device in the new iothub but only has two modules. edgeAgent and edgeHub

Originally it was sending the content in base64 format to IoT Hub. And the content is non human readable if we routed to Azure storage. (Fig. 1) The pull request is to fix the content encoding from base64 to utf-8 and make it readable. (Fig. 2)

| Fig.1 | | :--- | | |

| Fig.2 | | :--- | | |

Bumps pillow from 8.3.2 to 9.3.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

format that can be converted to OpenVINO IR.

here for a compatiblity matrix.

Only certain ops/layers are supported.

There may be more, but these were ones I needed more info on. Not sure, perhaps the OpenVINO site is down temporarily, and this is only temporary... Worth testing.

I am trying to find a way to combine data from the vision module and data from the Bluetooth module. It is quite hard to determine the right approach for using the Bluetooth module. i was hoping to find any guidance on this here??

Describe the bug Code azure-percept-advanced-development/ssd.cpp at main · Microsoft/azure-percept-advanced-development (github.com) results in compile error when considering camera feed.

To Reproduce recompile the azureeyemodule using the current code

Expected behavior no compile error

Screenshots If applicable, add screenshots to help explain your problem.

Additional context We modified the following to make it work:

// before: auto pipeline = graph.compileStreaming(cv::compile_args(networks, kernels, cv::gapi::mx::mvcmdFile{ this->mvcmd })); 
auto pipeline = graph.compileStreaming(cv::gapi::mx::Camera::params(), cv::compile_args(networks, kernels, cv::gapi::mx::mvcmdFile{ this->mvcmd }));
...
// before: pipeline.setSource<cv::gapi::wip::GCaptureSource>(video_file_path);
pipeline.setSource(cv::gapi::wip::make_src<cv::gapi::mx::Camera>()); 
...
// before: pipeline.setSource<cv::gapi::wip::GCaptureSource>(0); 
pipeline.setSource(cv::gapi::wip::make_src<cv::gapi::mx::Camera>());

When reading the document, we feel the following aspects are not clear enough. Some points mentioned below are not easy to find:

• An overview page that shows people e2e structure of IoT edge modules running on Azure Percept, including relationships between different modules, providing links to different resources.
• A well-structured documentation that gives instructions to developers and machine learning engineers (or data scientists) on how to add custom modules.
• A clear documentation that shows how to compile azureeyemodule end-to-end on Windows and Linux OS. Current document provided in github is not clear in the following aspects:
  • Differences between native compile and non-native compile?
  • When to use a native compile method and when to use non-native compile method?
  • how to revert (or fixed) if any steps go wrong? (reset?)
• The documentation doesn't point out modules to be stopped if re-deploy custom azureeyemodule is needed -- we need to stop both azureeyemodule and IoT agent
• The document doesn't tell the readers that rtsp:3000 is working when you customize it using a native way.
• The document doesn't state that the readers can check 8554 using a VLC player
• The document should clearly state expected drivers for external devices, for example, whether a driver for external monitor is expected.
• The document should be revisited and updated frequently based on the OpenVINO update, and check if the recommended approaches of converting models to blob are still valid.
• The document should state that it is possible to convert model on device using the built-in code provided by azureeyemodule, along with the required model files and format. Currently, when reading the code, we understand it is possible to do so, but no instructions or sample model files are provided.
• It would be nice if the document provided a reference link to OpenCV G-API
• It would be nice if the document shows the structure of the model zip file required by azureeyemodule.
• It would be nice if the hardware spec could also provide the GFLOPs besides TOPs. Or provide latency benchmark on the default models provided in azureeyemodule repo.

Describe the bug I tried to find the OpenVINO version of the default object detection model, ssd_mobilenet_v2_coco.blob, but couldn't find it.

In README.md, it said the default SSD model is from here. However, I checked the labels, the model on Percept has 183 labels, but the model in OpenVINO has only 92 labels. Where is the OpenVINO version of the default ssd_mobilenet_v2_coco.blob?

To Reproduce None

Expected behavior The labels on Azure Percept should be the same as The labels in OpenVINO.

Screenshots None

Logs None

Additional context None