Azure Percept DK Advanced Development
Please note! The experiences in this repository should be considered to be in preview/beta. Significant portions of these experiences are subject to change without warning. No part of this code should be considered stable.
Please consider providing feedback via this questionnaire. Your feedback will help us continue to fine-tune and improve the advanced tools experience.
This repository holds all the code and documentation for advanced development using the Azure Percept DK. In this repository, you will find:
- azureeyemodule: The code for the azureeyemodule, which is the IoT module responsible for running the AI workload on the Percept DK.
- machine-learning-notebooks: Example Python notebooks which show how to train up a few example neural networks from scratch (or using transfer learning) and get them onto your device.
- Model and Data Protection: Azure Percept currently supports AI model and data protection as a preview feature.
One of the main things that this repository can be used for is to bring your own custom computer vision pipeline to your Azure Percept DK. The flow for doing that would be this:
- Use whatever version of whatever DL framework you want (Tensorflow 2.x or 1.x, PyTorch, etc.)
- Develop your custom DL model and save it to a format that can be converted to OpenVINO IR or OpenVINO Myriad X blob. However, make sure your ops/layers are supported by OpenVINO 2021.1. See here for a compatiblity matrix.
- Use OpenVINO to convert it to IR or blob format.
- I recommend using the OpenVINO Workbench to convert your model to OpenVINO IR (or to download a common, pretrained model from their model zoo).
- You can use the scripts/run_workbench.sh script on Unix systems to run the workbench, or just run its single command in Powershell on Windows.
- You can use a Docker container to convert IR to blob for our device. See the scripts/compile_ir.sh script and use it as a reference. Note that you will need to modify it to adjust for if you have multiple output layers in your network.
- Develop a C++ subclass, using the examples we already have. See the azureeyemodule folder for how to do this.
- The azureeyemodule is the IoT module running on the device responsible for doing inference. It will need to grab your model somehow. For development, you could package your model up with your custom azureeyemodule and then have the custom program run it directly. You could also have it pull down a model through the module twin (again, see the azureeyemodule folder for more details).
The Azure Percept DK's azureeeyemodule supports a few AI models out of the box. The default model that runs is Single Shot Detector (SSD), trained for general object detection on the COCO dataset. But there are a few others that can run without any hassle. Here are the links for the models that we officially guarantee (because we host them and test them on every release).
To use these models, you can download them through the Azure Percept Studio, or you can paste the URLs into your Module Twin as the value for "ModelZipUrl".
This repository follows the Microsoft Code of Conduct.
Please see the CONTRIBUTING.md file for instructions on how to contribute to this repository.
Trademarks This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
Reporting Security Vulnerabilities
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include Microsoft, Azure, DotNet, AspNet, Xamarin, and our GitHub organizations.
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets Microsoft's Microsoft's definition of a security vulnerability, please report it to us as described below.
Please do not report security vulnerabilities through public GitHub issues.
Instead, please report them to the Microsoft Security Response Center (MSRC) at https://msrc.microsoft.com/create-report.
If you prefer to submit without logging in, send email to [email protected]. If possible, encrypt your message with our PGP key; please download it from the Microsoft Security Response Center PGP Key page.
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at microsoft.com/msrc.
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
- Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
- Full paths of source file(s) related to the manifestation of the issue
- The location of the affected source code (tag/branch/commit or direct URL)
- Any special configuration required to reproduce the issue
- Step-by-step instructions to reproduce the issue
- Proof-of-concept or exploit code (if possible)
- Impact of the issue, including how an attacker might exploit the issue
This information will help us triage your report more quickly.
If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our Microsoft Bug Bounty Program page for more details about our active programs.
We prefer all communications to be in English.
Microsoft follows the principle of Coordinated Vulnerability Disclosure.