🚀
🚀
🌟
Lite.AI
Introduction.
Lite.AI
Important Notes !!!
-
🔥 (20210728) Added FaceLandmarks1000 for 1000 facial landmarks detection, 2Mb only ! See demo . -
✅ (20210727) Added MobileNetV2SE68、PFLD68 for 68 facial landmarks detection ! See demo . -
✅ (20210726) Added PFLD98 for 98 facial landmarks detection ! See demo . -
✅ (20210722) Update lite.ai.hub.onnx.md ! Lite.AI contains 50+ AI models with 100+ .onnx files now. -
🔥 (20210721) Added YOLOX to Lite.AI ! Use it through lite::cv::detection::YoloX syntax ! See demo .
Contents.
1. Dependencies.
Mac OS.
install OpenCV
and onnxruntime
libraries using Homebrew or you can download the built dependencies from this repo. See third_party and build-docs1 for more details.
brew update
brew install opencv
brew install onnxruntime
Expand for More Details of Dependencies.
Linux.
- todo
⚠️
Windows.
- todo
⚠️
Inference Engine Plans:
- doing:
❇️ onnxruntime
- todo:
⚠️ NCNN
⚠️ MNN
⚠️ OpenMP
2. Build Lite.AI.
Build the shared lib of Lite.AI for MacOS from sources. Note that Lite.AI uses onnxruntime
as default backend, for the reason that onnxruntime supports the most of onnx's operators.
Linux and Windows.
OpenCV
and onnxruntime
firstly and put then into the third_party
directory. Please reference the build-docs1 for third_party
. The documents and docker image for Linux will be coming soon ~
- Clone the Lite.AI from sources:
git clone --depth=1 https://github.com/DefTruth/lite.ai.git # latest
- Build shared lib.
cd lite.ai
sh ./build.sh
cd ./build/lite.ai/lib && otool -L liblite.ai.0.0.1.dylib
liblite.ai.0.0.1.dylib:
@rpath/liblite.ai.0.0.1.dylib (compatibility version 0.0.1, current version 0.0.1)
@rpath/libopencv_highgui.4.5.dylib (compatibility version 4.5.0, current version 4.5.2)
@rpath/libonnxruntime.1.7.0.dylib (compatibility version 0.0.0, current version 1.7.0)
...
Expand for more details of How to link the shared lib of Lite.AI?
cd ../ && tree .
├── bin
├── include
│ ├── lite
│ │ ├── backend.h
│ │ ├── config.h
│ │ └── lite.h
│ └── ort
└── lib
└── liblite.ai.0.0.1.dylib
- Run the built examples:
cd ./build/lite.ai/bin && ls -lh | grep lite
-rwxr-xr-x 1 root staff 301K Jun 26 23:10 liblite.ai.0.0.1.dylib
...
-rwxr-xr-x 1 root staff 196K Jun 26 23:10 lite_yolov4
-rwxr-xr-x 1 root staff 196K Jun 26 23:10 lite_yolov5
...
./lite_yolov5
LITEORT_DEBUG LogId: ../../../hub/onnx/cv/yolov5s.onnx
=============== Input-Dims ==============
...
detected num_anchors: 25200
generate_bboxes num: 66
Default Version Detected Boxes Num: 5
- To link
lite.ai
shared lib. You need to make sure thatOpenCV
andonnxruntime
are linked correctly. Just like:
cmake_minimum_required(VERSION 3.17)
project(testlite.ai)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_BUILD_TYPE debug)
# link opencv.
set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/opencv/lib/cmake/opencv4)
find_package(OpenCV 4 REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
# link onnxruntime.
set(ONNXRUNTIME_DIR ${CMAKE_SOURCE_DIR}/onnxruntime/)
set(ONNXRUNTIME_INCLUDE_DIR ${ONNXRUNTIME_DIR}/include)
set(ONNXRUNTIME_LIBRARY_DIR ${ONNXRUNTIME_DIR}/lib)
include_directories(${ONNXRUNTIME_INCLUDE_DIR})
link_directories(${ONNXRUNTIME_LIBRARY_DIR})
# link lite.ai.
set(LITEHUB_DIR ${CMAKE_SOURCE_DIR}/lite.ai)
set(LITEHUB_INCLUDE_DIR ${LITEHUB_DIR}/include)
set(LITEHUB_LIBRARY_DIR ${LITEHUB_DIR}/lib)
include_directories(${LITEHUB_INCLUDE_DIR})
link_directories(${LITEHUB_LIBRARY_DIR})
# add your executable
add_executable(lite_yolov5 test_lite_yolov5.cpp)
target_link_libraries(lite_yolov5 lite.ai onnxruntime ${OpenCV_LIBS})
A minimum example to show you how to link the shared lib of Lite.AI correctly for your own project can be found at lite.ai-release .
3. Model Zoo.
3.1 Namespace and Lite.AI modules.
Lite.AI contains 50+ AI models with 100+ frozen pretrained .onnx files now. They come from different fields of computer vision. Click the Expand
Expand Details for Namespace and Lite.AI modules.
Namepace | Details |
---|---|
lite::cv::detection | Object Detection. one-stage and anchor-free detectors, YoloV5, YoloV4, SSD, etc. |
lite::cv::classification | Image Classification. DensNet, ShuffleNet, ResNet, IBNNet, GhostNet, etc. |
lite::cv::faceid | Face Recognition. ArcFace, CosFace, CurricularFace, etc. |
lite::cv::face | Face Analysis. detect, align, pose, attr, etc. |
lite::cv::face::detect | Face Detection. UltraFace, RetinaFace, FaceBoxes, PyramidBox, etc. |
lite::cv::face::align | Face Alignment. PFLD(106), FaceLandmark1000(1000 landmarks), PRNet, etc. |
lite::cv::face::pose | Head Pose Estimation. FSANet, etc. |
lite::cv::face::attr | Face Attributes. Emotion, Age, Gender. EmotionFerPlus, VGG16Age, etc. |
lite::cv::segmentation | Object Segmentation. Such as FCN, DeepLabV3, etc. |
lite::cv::style | Style Transfer. Contains neural style transfer now, such as FastStyleTransfer. |
lite::cv::matting | Image Matting. Object and Human matting. |
lite::cv::colorization | Colorization. Make Gray image become RGB. |
lite::cv::resolution | Super Resolution. |
3.2 Lite.AI's Classes and Pretrained Files.
Correspondence between the classes in Lite.AI and pretrained model files can be found at lite.ai.hub.onnx.md. For examples, the pretrained model files for lite::cv::detection::YoloV5 and lite::cv::detection::YoloX are listed as follows.
Expand Examples for Lite.AI's Classes and Pretrained Files.
Class | Pretrained ONNX Files | Rename or Converted From (Repo) | Size |
---|---|---|---|
lite::cv::detection::YoloV5 | yolov5l.onnx | yolov5 ( |
188Mb |
lite::cv::detection::YoloV5 | yolov5m.onnx | yolov5 ( |
85Mb |
lite::cv::detection::YoloV5 | yolov5s.onnx | yolov5 ( |
29Mb |
lite::cv::detection::YoloV5 | yolov5x.onnx | yolov5 ( |
351Mb |
lite::cv::detection::YoloX | yolox_x.onnx | YOLOX ( |
378Mb |
lite::cv::detection::YoloX | yolox_l.onnx | YOLOX ( |
207Mb |
lite::cv::detection::YoloX | yolox_m.onnx | YOLOX ( |
97Mb |
lite::cv::detection::YoloX | yolox_s.onnx | YOLOX ( |
34Mb |
lite::cv::detection::YoloX | yolox_tiny.onnx | YOLOX ( |
19Mb |
lite::cv::detection::YoloX | yolox_nano.onnx | YOLOX ( |
3.5Mb |
It means that you can load the the any one yolov5*.onnx
and yolox_*.onnx
according to your application through the same Lite.AI classes, such as YoloV5, YoloX, etc.
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5x.onnx"); // for server
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5l.onnx");
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5m.onnx");
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5s.onnx"); // for mobile device
auto *yolox = new lite::cv::detection::YoloX("yolox_x.onnx");
auto *yolox = new lite::cv::detection::YoloX("yolox_l.onnx");
auto *yolox = new lite::cv::detection::YoloX("yolox_m.onnx");
auto *yolox = new lite::cv::detection::YoloX("yolox_s.onnx");
auto *yolox = new lite::cv::detection::YoloX("yolox_tiny.onnx");
auto *yolox = new lite::cv::detection::YoloX("yolox_nano.onnx"); // 3.5Mb only !
3.3 Model Zoo for Lite.AI.
Note that the models here are all from third-party projects. Most of the models were converted by Lite.AI. In Lite.AI, different names of the same algorithm mean that the corresponding models come from different repositories, different implementations, or use different training data, etc.
(Baidu Drive code: 8gin)
- Object Detection.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
YoloV5 | 28M | yolov5 |
|
detection |
|
demo | |
YoloV3 | 236M | onnx-models |
|
detection |
|
demo | |
TinyYoloV3 | 33M | onnx-models |
|
detection |
|
demo | |
YoloV4 | 176M | YOLOv4... |
|
detection |
|
demo | |
SSD | 76M | onnx-models |
|
detection |
|
demo | |
SSDMobileNetV1 | 27M | onnx-models |
|
detection |
|
demo | |
YoloX | 3.5M | YOLOX |
|
detection |
|
demo |
⚠️
Expand More Details for Lite.AI's Model Zoo.
- Classification.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
EfficientNetLite4 | 49M | onnx-models |
|
classification |
|
demo | |
ShuffleNetV2 | 8.7M | onnx-models |
|
classification |
|
demo | |
DenseNet121 | 30.7M | torchvision |
|
classification |
|
demo | |
GhostNet | 20M | torchvision |
|
classification |
|
demo | |
HdrDNet | 13M | torchvision |
|
classification |
|
demo | |
IBNNet | 97M | torchvision |
|
classification |
|
demo | |
MobileNetV2 | 13M | torchvision |
|
classification |
|
demo | |
ResNet | 44M | torchvision |
|
classification |
|
demo | |
ResNeXt | 95M | torchvision |
|
classification |
|
demo |
- Face Detection.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
UltraFace | 1.1M | Ultra-Light... |
|
face::detect |
|
demo | |
MobileV1RetinaFace | - | ...Retinaface |
|
face::detect |
|
- | |
ResNetRetinaFace | - | ...Retinaface |
|
face::detect |
|
- | |
FaceBoxes | - | FaceBoxes |
|
face::detect |
|
- |
- Face Alignment.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
PFLD | 1.0M | pfld_106_... |
|
face::align |
|
demo | |
PFLD98 | 4.8M | PFLD... |
|
face::align |
|
demo | |
MobileNetV268 | 9.4M | ...landmark |
|
face::align |
|
demo | |
MobileNetV2SE68 | 11M | ...landmark |
|
face::align |
|
demo | |
PFLD68 | 2.8M | ...landmark |
|
face::align |
|
demo | |
FaceLandmark1000 | 2.0M | FaceLandm... |
|
face::align |
|
demo |
- Face Attributes.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
AgeGoogleNet | 23M | onnx-models |
|
face::attr |
|
demo | |
GenderGoogleNet | 23M | onnx-models |
|
face::attr |
|
demo | |
EmotionFerPlus | 33M | onnx-models |
|
face::attr |
|
demo | |
VGG16Age | 514M | onnx-models |
|
face::attr |
|
demo | |
VGG16Gender | 512M | onnx-models |
|
face::attr |
|
demo | |
SSRNet | 190K | SSR_Net... |
|
face::attr |
|
demo | |
EfficientEmotion7 | 15M | face-emo... |
|
face::attr |
|
demo | |
EfficientEmotion8 | 15M | face-emo... |
|
face::attr |
|
demo | |
MobileEmotion7 | 13M | face-emo... |
|
face::attr |
|
demo | |
ReXNetEmotion7 | 30M | face-emo... |
|
face::attr |
|
demo |
- Face Recognition.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
GlintArcFace | 92M | insightface |
|
faceid |
|
demo | |
GlintCosFace | 92M | insightface |
|
faceid |
|
demo | |
GlintPartialFC | 170M | insightface |
|
faceid |
|
demo | |
FaceNet | 89M | facenet... |
|
faceid |
|
demo | |
FocalArcFace | 166M | face.evoLVe... |
|
faceid |
|
demo | |
FocalAsiaArcFace | 166M | face.evoLVe... |
|
faceid |
|
demo | |
TencentCurricularFace | 249M | TFace |
|
faceid |
|
demo | |
TencentCifpFace | 130M | TFace |
|
faceid |
|
demo | |
CenterLossFace | 280M | center-loss... |
|
faceid |
|
demo | |
SphereFace | 80M | sphere... |
|
faceid |
|
demo | |
PoseRobustFace | 92M | DREAM |
|
faceid |
|
demo | |
NaivePoseRobustFace | 43M | DREAM |
|
faceid |
|
demo | |
MobileFaceNet | 3.8M | MobileFace... |
|
faceid |
|
demo | |
CavaGhostArcFace | 15M | cavaface... |
|
faceid |
|
demo | |
CavaCombinedFace | 250M | cavaface... |
|
faceid |
|
demo | |
MobileSEFocalFace | 4.5M | face_recog... |
|
faceid |
|
demo |
- Head Pose Estimation.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
FSANet | 1.2M | ...fsanet... |
|
face::pose |
|
demo |
- Segmentation.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
DeepLabV3ResNet101 | 232M | torchvision |
|
segmentation |
|
demo | |
FCNResNet101 | 207M | torchvision |
|
segmentation |
|
demo |
- Style Transfer.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
FastStyleTransfer | 6.4M | onnx-models |
|
style |
|
demo |
- Colorization.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
Colorizer | 123M | colorization |
|
colorization |
|
demo |
- Super Resolution.
Class | Size | From | Awesome | File | Type | State | Usage |
---|---|---|---|---|---|---|---|
SubPixelCNN | 234K | ...PIXEL... |
|
resolution |
|
demo |
4. Examples for Lite.AI.
More examples can be found at lite.ai-demos. Note that the default backend for Lite.AI is onnxruntime
, for the reason that onnxruntime supports the most of onnx's operators. Click the Expand
YoloV5. Download model from Model-Zoo2.
Example0: Object Detection using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/yolov5s.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_yolov5_1.jpg";
std::string save_img_path = "../../../logs/test_lite_yolov5_1.jpg";
auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path);
std::vector detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
yolov5->detect(img_bgr, detected_boxes);
lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);
delete yolov5;
}
The output is:
Or you can use Newest
FaceLandmarks1000. Download model from Model-Zoo2.
Example1: 1000 Facial Landmarks Detection using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/FaceLandmark1000.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_face_landmarks_0.png";
std::string save_img_path = "../../../logs/test_lite_face_landmarks_1000.jpg";
auto *face_landmarks_1000 = new lite::cv::face::align::FaceLandmark1000(onnx_path);
lite::cv::types::Landmarks landmarks;
cv::Mat img_bgr = cv::imread(test_img_path);
face_landmarks_1000->detect(img_bgr, landmarks);
lite::cv::utils::draw_landmarks_inplace(img_bgr, landmarks);
cv::imwrite(save_img_path, img_bgr);
delete face_landmarks_1000;
}
The output is:
colorization. Download model from Model-Zoo2.
Example2: Colorization using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/eccv16-colorizer.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_colorizer_1.jpg";
std::string save_img_path = "../../../logs/test_lite_eccv16_colorizer_1.jpg";
auto *colorizer = new lite::cv::colorization::Colorizer(onnx_path);
cv::Mat img_bgr = cv::imread(test_img_path);
lite::cv::types::ColorizeContent colorize_content;
colorizer->detect(img_bgr, colorize_content);
if (colorize_content.flag) cv::imwrite(save_img_path, colorize_content.mat);
delete colorizer;
}
The output is:
⚠️
Expand All Examples for Each Topic in Lite.AI.
4.1 Expand Examples for Object Detection.
YoloV5. Download model from Model-Zoo2.
4.1 Object Detection using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/yolov5s.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_yolov5_1.jpg";
std::string save_img_path = "../../../logs/test_lite_yolov5_1.jpg";
auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path);
std::vector detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
yolov5->detect(img_bgr, detected_boxes);
lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);
delete yolov5;
}
The output is:
Or you can use Newest
#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/yolox_s.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_yolox_1.jpg";
std::string save_img_path = "../../../logs/test_lite_yolox_1.jpg";
auto *yolox = new lite::cv::detection::YoloX(onnx_path);
std::vector detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
yolox->detect(img_bgr, detected_boxes);
lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);
delete yolox;
}
The output is:
More classes for general object detection.
auto *detector = new lite::cv::detection::YoloX(onnx_path); // new !!!
auto *detector = new lite::cv::detection::YoloV4(onnx_path);
auto *detector = new lite::cv::detection::YoloV3(onnx_path);
auto *detector = new lite::cv::detection::TinyYoloV3(onnx_path);
auto *detector = new lite::cv::detection::SSD(onnx_path);
auto *detector = new lite::cv::detection::SSDMobileNetV1(onnx_path);
4.2 Expand Examples for Face Recognition.
ArcFace. Download model from Model-Zoo2.
4.2 Face Recognition using#include "lite/lite.h" static void test_default() { std::string onnx_path = "../../../hub/onnx/cv/ms1mv3_arcface_r100.onnx"; std::string test_img_path0 = "../../../examples/lite/resources/test_lite_faceid_0.png"; std::string test_img_path1 = "../../../examples/lite/resources/test_lite_faceid_1.png"; std::string test_img_path2 = "../../../examples/lite/resources/test_lite_faceid_2.png"; auto *glint_arcface = new lite::cv::faceid::GlintArcFace(onnx_path); lite::cv::types::FaceContent face_content0, face_content1, face_content2; cv::Mat img_bgr0 = cv::imread(test_img_path0); cv::Mat img_bgr1 = cv::imread(test_img_path1); cv::Mat img_bgr2 = cv::imread(test_img_path2); glint_arcface->detect(img_bgr0, face_content0); glint_arcface->detect(img_bgr1, face_content1); glint_arcface->detect(img_bgr2, face_content2); if (face_content0.flag && face_content1.flag && face_content2.flag) { float sim01 = lite::cv::utils::math::cosine_similarity<float>( face_content0.embedding, face_content1.embedding); float sim02 = lite::cv::utils::math::cosine_similarity<float>( face_content0.embedding, face_content2.embedding); std::cout << "Detected Sim01: " << sim << " Sim02: " << sim02 << std::endl; } delete glint_arcface; }
The output is:
Detected Sim01: 0.721159 Sim02: -0.0626267
More classes for face recognition.
auto *recognition = new lite::cv::faceid::GlintCosFace(onnx_path); // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::GlintArcFace(onnx_path); // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::GlintPartialFC(onnx_path); // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::FaceNet(onnx_path);
auto *recognition = new lite::cv::faceid::FocalArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::FocalAsiaArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::TencentCurricularFace(onnx_path); // Tencent(TFace)
auto *recognition = new lite::cv::faceid::TencentCifpFace(onnx_path); // Tencent(TFace)
auto *recognition = new lite::cv::faceid::CenterLossFace(onnx_path);
auto *recognition = new lite::cv::faceid::SphereFace(onnx_path);
auto *recognition = new lite::cv::faceid::PoseRobustFace(onnx_path);
auto *recognition = new lite::cv::faceid::NaivePoseRobustFace(onnx_path);
auto *recognition = new lite::cv::faceid::MobileFaceNet(onnx_path); // 3.8Mb only !
auto *recognition = new lite::cv::faceid::CavaGhostArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::CavaCombinedFace(onnx_path);
auto *recognition = new lite::cv::faceid::MobileSEFocalFace(onnx_path); // 4.5Mb only !
4.3 Expand Examples for Segmentation.
DeepLabV3ResNet101. Download model from Model-Zoo2.
4.3 Segmentation using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/deeplabv3_resnet101_coco.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_deeplabv3_resnet101.png";
std::string save_img_path = "../../../logs/test_lite_deeplabv3_resnet101.jpg";
auto *deeplabv3_resnet101 = new lite::cv::segmentation::DeepLabV3ResNet101(onnx_path, 16); // 16 threads
lite::cv::types::SegmentContent content;
cv::Mat img_bgr = cv::imread(test_img_path);
deeplabv3_resnet101->detect(img_bgr, content);
if (content.flag)
{
cv::Mat out_img;
cv::addWeighted(img_bgr, 0.2, content.color_mat, 0.8, 0., out_img);
cv::imwrite(save_img_path, out_img);
if (!content.names_map.empty())
{
for (auto it = content.names_map.begin(); it != content.names_map.end(); ++it)
{
std::cout << it->first << " Name: " << it->second << std::endl;
}
}
}
delete deeplabv3_resnet101;
}
The output is:
More classes for segmentation.
auto *segment = new lite::cv::segmentation::FCNResNet101(onnx_path);
4.4 Expand Examples for Face Attributes Analysis.
SSRNet . Download model from Model-Zoo2.
4.4 Age Estimation using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/ssrnet.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_ssrnet.jpg";
std::string save_img_path = "../../../logs/test_lite_ssrnet.jpg";
lite::cv::face::attr::SSRNet *ssrnet = new lite::cv::face::attr::SSRNet(onnx_path);
lite::cv::types::Age age;
cv::Mat img_bgr = cv::imread(test_img_path);
ssrnet->detect(img_bgr, age);
lite::cv::utils::draw_age_inplace(img_bgr, age);
cv::imwrite(save_img_path, img_bgr);
std::cout << "Default Version Done! Detected SSRNet Age: " << age.age << std::endl;
delete ssrnet;
}
The output is:
More classes for face attributes analysis.
auto *attribute = new lite::cv::face::attr::AgeGoogleNet(onnx_path);
auto *attribute = new lite::cv::face::attr::GenderGoogleNet(onnx_path);
auto *attribute = new lite::cv::face::attr::EmotionFerPlus(onnx_path);
auto *attribute = new lite::cv::face::attr::VGG16Age(onnx_path);
auto *attribute = new lite::cv::face::attr::VGG16Gender(onnx_path);
auto *attribute = new lite::cv::face::attr::EfficientEmotion7(onnx_path); // 7 emotions, 15Mb only!
auto *attribute = new lite::cv::face::attr::EfficientEmotion8(onnx_path); // 8 emotions, 15Mb only!
auto *attribute = new lite::cv::face::attr::MobileEmotion7(onnx_path); // 7 emotions
auto *attribute = new lite::cv::face::attr::ReXNetEmotion7(onnx_path); // 7 emotions
4.5 Expand Examples for Image Classification.
DenseNet. Download model from Model-Zoo2.
4.5 1000 Classes Classification using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/densenet121.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_densenet.jpg";
auto *densenet = new lite::cv::classification::DenseNet(onnx_path);
lite::cv::types::ImageNetContent content;
cv::Mat img_bgr = cv::imread(test_img_path);
densenet->detect(img_bgr, content);
if (content.flag)
{
const unsigned int top_k = content.scores.size();
if (top_k > 0)
{
for (unsigned int i = 0; i < top_k; ++i)
std::cout << i + 1
<< ": " << content.labels.at(i)
<< ": " << content.texts.at(i)
<< ": " << content.scores.at(i)
<< std::endl;
}
}
delete densenet;
}
The output is:
More classes for image classification.
auto *classifier = new lite::cv::classification::EfficientNetLite4(onnx_path);
auto *classifier = new lite::cv::classification::ShuffleNetV2(onnx_path);
auto *classifier = new lite::cv::classification::GhostNet(onnx_path);
auto *classifier = new lite::cv::classification::HdrDNet(onnx_path);
auto *classifier = new lite::cv::classification::IBNNet(onnx_path);
auto *classifier = new lite::cv::classification::MobileNetV2(onnx_path);
auto *classifier = new lite::cv::classification::ResNet(onnx_path);
auto *classifier = new lite::cv::classification::ResNeXt(onnx_path);
4.6 Expand Examples for Face Detection.
UltraFace. Download model from Model-Zoo2.
4.6 Face Detection using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/ultraface-rfb-640.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_ultraface.jpg";
std::string save_img_path = "../../../logs/test_lite_ultraface.jpg";
auto *ultraface = new lite::cv::face::detect::UltraFace(onnx_path);
std::vector detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
ultraface->detect(img_bgr, detected_boxes);
lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);
delete ultraface;
}
The output is:
4.7 Expand Examples for Colorization.
colorization. Download model from Model-Zoo2.
4.7 Colorization using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/eccv16-colorizer.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_colorizer_1.jpg";
std::string save_img_path = "../../../logs/test_lite_eccv16_colorizer_1.jpg";
auto *colorizer = new lite::cv::colorization::Colorizer(onnx_path);
cv::Mat img_bgr = cv::imread(test_img_path);
lite::cv::types::ColorizeContent colorize_content;
colorizer->detect(img_bgr, colorize_content);
if (colorize_content.flag) cv::imwrite(save_img_path, colorize_content.mat);
delete colorizer;
}
The output is:
4.8 Expand Examples for Head Pose Estimation.
FSANet. Download model from Model-Zoo2.
4.8 Head Pose Estimation using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/fsanet-var.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_fsanet.jpg";
std::string save_img_path = "../../../logs/test_lite_fsanet.jpg";
auto *fsanet = new lite::cv::face::pose::FSANet(onnx_path);
cv::Mat img_bgr = cv::imread(test_img_path);
lite::cv::types::EulerAngles euler_angles;
fsanet->detect(img_bgr, euler_angles);
if (euler_angles.flag)
{
lite::cv::utils::draw_axis_inplace(img_bgr, euler_angles);
cv::imwrite(save_img_path, img_bgr);
std::cout << "yaw:" << euler_angles.yaw << " pitch:" << euler_angles.pitch << " row:" << euler_angles.roll << std::endl;
}
delete fsanet;
}
The output is:
4.9 Expand Examples for Face Alignment.
FaceLandmarks1000. Download model from Model-Zoo2.
4.9 1000 Facial Landmarks Detection using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/FaceLandmark1000.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_face_landmarks_0.png";
std::string save_img_path = "../../../logs/test_lite_face_landmarks_1000.jpg";
auto *face_landmarks_1000 = new lite::cv::face::align::FaceLandmark1000(onnx_path);
lite::cv::types::Landmarks landmarks;
cv::Mat img_bgr = cv::imread(test_img_path);
face_landmarks_1000->detect(img_bgr, landmarks);
lite::cv::utils::draw_landmarks_inplace(img_bgr, landmarks);
cv::imwrite(save_img_path, img_bgr);
delete face_landmarks_1000;
}
The output is:
More classes for face alignment.
auto *align = new lite::cv::face::align::PFLD(onnx_path); // 106 landmarks
auto *align = new lite::cv::face::align::PFLD98(onnx_path); // 98 landmarks
auto *align = new lite::cv::face::align::PFLD68(onnx_path); // 68 landmarks
auto *align = new lite::cv::face::align::MobileNetV268(onnx_path); // 68 landmarks
auto *align = new lite::cv::face::align::MobileNetV2SE68(onnx_path); // 68 landmarks
auto *align = new lite::cv::face::align::FaceLandmark1000(onnx_path); // 1000 landmarks !
4.10 Expand Examples for Style Transfer.
FastStyleTransfer. Download model from Model-Zoo2.
4.10 Style Transfer using#include "lite/lite.h"
static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/style-candy-8.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_fast_style_transfer.jpg";
std::string save_img_path = "../../../logs/test_lite_fast_style_transfer_candy.jpg";
auto *fast_style_transfer = new lite::cv::style::FastStyleTransfer(onnx_path);
lite::cv::types::StyleContent style_content;
cv::Mat img_bgr = cv::imread(test_img_path);
fast_style_transfer->detect(img_bgr, style_content);
if (style_content.flag) cv::imwrite(save_img_path, style_content.mat);
delete fast_style_transfer;
}
The output is:
4.11 Expand Examples for Image Matting.
- todo
⚠️
5. Lite.AI API Docs.
5.1 Default Version APIs.
More details of Default Version APIs can be found at default-version-api-docs . For examples, the interface for YoloV5 is:
lite::cv::detection::YoloV5
void detect(const cv::Mat &mat, std::vector &detected_boxes,
float score_threshold = 0.25f, float iou_threshold = 0.45f,
unsigned int topk = 100, unsigned int nms_type = NMS::OFFSET);
Expand for ONNXRuntime, MNN and NCNN version APIs.
5.2 ONNXRuntime Version APIs.
More details of ONNXRuntime Version APIs can be found at onnxruntime-version-api-docs . For examples, the interface for YoloV5 is:
lite::onnxruntime::cv::detection::YoloV5
void detect(const cv::Mat &mat, std::vector &detected_boxes,
float score_threshold = 0.25f, float iou_threshold = 0.45f,
unsigned int topk = 100, unsigned int nms_type = NMS::OFFSET);
5.3 MNN Version APIs.
(todo
lite::mnn::cv::detection::YoloV5
lite::mnn::cv::detection::YoloV4
lite::mnn::cv::detection::YoloV3
lite::mnn::cv::detection::SSD
...
5.4 NCNN Version APIs.
(todo
lite::ncnn::cv::detection::YoloV5
lite::ncnn::cv::detection::YoloV4
lite::ncnn::cv::detection::YoloV3
lite::ncnn::cv::detection::SSD
...
6. Other Docs.
Expand More Details for Other Docs.
6.1 Docs for ONNXRuntime.
- Rapid implementation of your inference using BasicOrtHandler
- Some very useful onnxruntime c++ interfaces
- How to compile a single model in this library you needed
- How to convert SubPixelCNN to ONNX and implements with onnxruntime c++
- How to convert Colorizer to ONNX and implements with onnxruntime c++
- How to convert SSRNet to ONNX and implements with onnxruntime c++
- How to convert YoloV3 to ONNX and implements with onnxruntime c++
- How to convert YoloV5 to ONNX and implements with onnxruntime c++
third_party.
6.2 Docs forOther build documents for different engines and different targets will be added later.
Library | Target | Docs |
---|---|---|
OpenCV | mac-x86_64 | opencv-mac-x86_64-build.zh.md |
OpenCV | android-arm | opencv-static-android-arm-build.zh.md |
onnxruntime | mac-x86_64 | onnxruntime-mac-x86_64-build.zh.md |
onnxruntime | android-arm | onnxruntime-android-arm-build.zh.md |
NCNN | mac-x86_64 | todo |
MNN | mac-x86_64 | todo |
TNN | mac-x86_64 | todo |
7. References.
Many thanks to the following projects. All the Lite.AI's models are sourced from these repos.
- YOLOX (
🔥 🔥 new!!↑) - insightface (
🔥 🔥 🔥 ↑) - yolov5 (
🔥 🔥 💥 ↑)
Expand More Details for References.
- headpose-fsanet-pytorch (
🔥 ↑) - pfld_106_face_landmarks (
🔥 🔥 ↑) - Ultra-Light-Fast-Generic-Face-Detector-1MB (
🔥 🔥 🔥 ↑) - onnx-models (
🔥 🔥 🔥 ↑) - SSR_Net_Pytorch (
🔥 ↑) - colorization (
🔥 🔥 🔥 ↑) - SUB_PIXEL_CNN (
🔥 ↑) - YOLOv4-pytorch (
🔥 🔥 🔥 ↑) - torchvision (
🔥 🔥 🔥 ↑) - facenet-pytorch (
🔥 ↑) - face.evoLVe.PyTorch (
🔥 🔥 🔥 ↑) - TFace (
🔥 🔥 ↑) - center-loss.pytorch (
🔥 🔥 ↑) - sphereface_pytorch (
🔥 🔥 ↑) - DREAM (
🔥 🔥 ↑) - MobileFaceNet_Pytorch (
🔥 🔥 ↑) - cavaface.pytorch (
🔥 🔥 ↑) - CurricularFace (
🔥 🔥 ↑) - face-emotion-recognition (
🔥 ↑) - face_recognition.pytorch (
🔥 🔥 ↑) - PFLD-pytorch (
🔥 🔥 ↑) - pytorch_face_landmark (
🔥 🔥 ↑) - FaceLandmark1000 (
🔥 🔥 ↑) - Pytorch_Retinaface (
🔥 🔥 🔥 ↑) - FaceBoxes (
🔥 🔥 ↑)
Star
License.
The code of Lite.AI is released under the MIT License.
Citations.
If you use this library in your project, please, cite it as follows.
@code{lite.ai2021,
title={Lite.AI: A simple and user friendly C++ library of awesome AI models.},
url={https://github.com/DefTruth/lite.ai},
note={Open-source software available at https://github.com/DefTruth/lite.ai},
author={YanJun},
year={2021}
}