A library for high performance deep learning inference on NVIDIA GPUs.

Last update: May 31, 2022

Overview

Forward - A library for high performance deep learning inference on NVIDIA GPUs

Forward - A library for high performance deep learning inference on NVIDIA GPUs

Forward

Forward is a library for high performance deep learning inference on NVIDIA GPUs. It provides a well-designed scheme that directly parse Tensorflow/PyTorch/Keras models to high-performance engine based on TensorRT. Compared to TensorRT, it is easy-to-use and easy-to-expand. So far, Forward supports not only mainstream deep learning models in CV, NLP and Recommend fields, but also some advanced models such as BERT, GAN, FaceSwap, StyleTransfer.

Features

Utilize TensorRT API and customized operators for high-performance deep learning inference.
Support not only mainstream deep learning models in CV, NLP and Recommend fields, but also advanced models such as BERT, GAN, FaceSwap, StyleTransfer.
Support FLOAT/HALF/INT8 infer modes.
Easy to use: Load directly Tensorflow(.pb)/PyTorch(.pth)/Keras(.h5) models and then do inference with TensorRT.
Easy to expand: Register customized layers refer to add_support_op.md.
Provide C++ and Python interfaces.

Quick Start

Prerequisites

NVIDIA CUDA >= 10.0, CuDNN >= 7 (Recommended version: CUDA 10.2 )
TensorRT >= 7.0.0.11, (Recommended version: TensorRT-7.2.1.6)
CMake >= 3.10.1
GCC >= 5.4.0, ld >= 2.26.1
(Pytorch) pytorch == 1.3.1
(Tensorflow) TensorFlow == 1.15.0 (download Tensorflow 1.15.0 and unzip it to source/third_party/tensorflow/lib)
(Keras) HDF 5

Build with CMake

Generate Makefiles or VS project (Windows) and build. Forward can be built for different framework, such as Fwd-Torch, Fwd-Python-Torch, Fwd-Tf, Fwd-Python-Tf, Fwd-Keras, Fwd-Python-Keras, which controlled by CMake options. For example, Fwd-Python-Tf is built as below.

mkdir build
cd build

cmake ..  \
-DTensorRT_ROOT=/path/to/TensorRT \ 
-DENABLE_LOGGING=ON \  
-DENABLE_PROFILING=ON \  
-DENABLE_DYNAMIC_BATCH=ON \ 
-DBUILD_PTYHON_LIB=ON \
-DENABLE_TORCH=OFF \  
-DENABLE_TENSORFLOW=ON \ 
-DENABLE_KERAS=OFF \ 

make -j

CMake build arguments

TensorRT_ROOT [Required]: Path to the TensorRT installation directory containing libraries
More CMake options refer to CMake Options

Unit Test

When the project is built, unit_test can be used to verify the project is successfully built.

cd build/bin
./unit_test --gtest_filter=TestTfNodes.*

Use Forward-Cpp

Refer to Demo for using Forward-Cpp in Linux

Use Forward-Python

Refer to Demo for using Forward-Python

More Usages

Notice: The name of INPUT in models can be viewed by model viewers, such as Netron.

FAQ

Models & Operators

Models

Operators

Contribution

CONTRIBUTING

Contributors

Any form of contribution is welcome. The above contributors have been officially released by Tencent.

We very much welcome developers to contribute to Tencent's open source, and we will also give them incentives to acknowledge and thank them. Here we provide an official description of Tencent's open source contribution. Specific contribution rules for each project are formulated by the project team. Developers can choose the appropriate project and participate according to the corresponding rules. The Tencent Project Management Committee will report regularly to qualified contributors and awards will be issued by the official contact.

License

Apache License v2.0

Issues

make报错
Describe the bug A clear and concise description of what the bug is. 执行命令： cmake .. -DTensorRT_ROOT=/home/soft/wp/TensorRT-8.2.0.6 -DENABLE_TORC H=ON -DENABLE_TORCH_PLUGIN=ON -DCMAKE_PREFIX_PATH=/home/soft/wp/libtorch

错误日志： -- Found Threads: TRUE -- Found CUDA: /usr/local/cuda (found version "11.1") -- CUDA_NVCC_FLAGS: -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -- Using the single-header code from /home/soft/wp/Forward/source/third_party/json/single_include/ -- Found TensorRT: /home/soft/wp/TensorRT-8.2.0.6/lib/libnvinfer.so;/home/soft/wp/TensorRT-8.2.0.6/lib/libnvinfer_plugin.so;/home/soft/wp/TensorRT-8.2.0.6/lib/libnvonnxparser.so;/home/soft/wp/TensorRT-8.2.0.6/lib/libnvparsers.so (found version "8.2.0") -- Found CUDA: /usr/local/cuda (found version "11.1") -- Caffe2: CUDA detected: 11.1 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 11.1 -- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so -- Found cuDNN: v? (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so) CMake Error at /home/soft/wp/libtorch/share/cmake/Caffe2/public/cuda.cmake:174 (message): PyTorch requires cuDNN 7 and above. Call Stack (most recent call first): /home/soft/wp/libtorch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include) /home/soft/wp/libtorch/share/cmake/Torch/TorchConfig.cmake:68 (find_package) CMakeLists.txt:248 (find_package)

-- Configuring incomplete, errors occurred!

实际上是已经安装了cudnn

ls /usr/local/cuda/lib64 | grep cudnn

Environment

TensorRT Version: TensorRT-8.2.0.6 NVIDIA GPU: P8 NVIDIA Driver Version: 465.19.01 CUDA Version: 11.1 CUDNN Version: 8.2.1.32 Operating System: Ubuntu 16.04.2 LTS Python Version (if applicable): 3.7 Tensorflow Version (if applicable): 2.6.0 PyTorch Version (if applicable): 1.9.0+cu111

Relevant Files

To Reproduce Steps to reproduce the behavior:

Go to '...'

Click on '....'

Scroll down to '....'

See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.
Stale
opened by Nipi64310 22

编译VC项目时出错

编译报错 先按照指引cmake，有test failed，但是完成了。

Selecting Windows SDK version  to target Windows 10.0.19042.
CMake Deprecation Warning at CMakeLists.txt:28 (cmake_policy):
  The OLD behavior for policy CMP0074 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2 (found version "10.2") 
CUDA_NVCC_FLAGS:  -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75
Found TensorRT: D:/Software/TensorRT-7.2.1.6.Windows10.x86_64.cuda-10.2.cudnn8.0/TensorRT-7.2.1.6/lib/nvinfer.lib;D:/Software/TensorRT-7.2.1.6.Windows10.x86_64.cuda-10.2.cudnn8.0/TensorRT-7.2.1.6/lib/nvinfer_plugin.lib;D:/Software/TensorRT-7.2.1.6.Windows10.x86_64.cuda-10.2.cudnn8.0/TensorRT-7.2.1.6/lib/nvonnxparser.lib;D:/Software/TensorRT-7.2.1.6.Windows10.x86_64.cuda-10.2.cudnn8.0/TensorRT-7.2.1.6/lib/nvparsers.lib (found version "7.2.1") 
Using the single-header code from D:/Projects/tencent_forward/Forward/source/third_party/json/single_include/
Use HDF5 on third_party: D:/Projects/tencent_forward/Forward/source/third_party/hdf5
Warnings Configuration: default:   /DWIN32 /D_WINDOWS /W3 :  /DWIN32 /D_WINDOWS /W3 /GR /EHsc /W3 /WX-
Check for STD namespace
Check for STD namespace - found
Performing CXX Test OLD_HEADER_FILENAME - Failed
Performing CXX Test HDF_NO_NAMESPACE - Failed
Performing CXX Test HDF_NO_STD - Failed
Performing CXX Test BOOL_NOTDEFINED - Failed
Performing CXX Test NO_STATIC_CAST - Failed
Performing CXX Test CXX_HAVE_OFFSETOF - Failed
Configuring done
Generating done

然后在项目生成的时候报错：

严重性	代码	说明	项目	文件	行
错误	C2664	“void std::vector<fwd::NamedTensor,std::allocator<_Ty>>::push_back(const fwd::NamedTensor &)”: 无法将参数 1 从“initializer list”转换为“fwd::NamedTensor &&”	trt_engine	D:\Projects\tencent_forward\Forward\source\trt_engine\trt_engine\trt_buffer_manager.h	116
错误	C1083	无法打开源文件: “D:\Projects\tencent_forward\Forward\build\source\third_party\hdf5\H5Tinit.c”: No such file or directory	hdf5	D:\Projects\tencent_forward\Forward\build\source\third_party\hdf5\src\c1	1
错误	C2664	“nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer> nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::parse(nlohmann::detail::input_adapter &&,const std::function<bool (int,nlohmann::detail::parser<nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>>::parse_event_t,BasicJsonType &)>,const bool)”: 无法将参数 1 从“std::string”转换为“nlohmann::detail::input_adapter &&”	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	72
错误	C2679	二进制“=”: 没有找到接受“std::string”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	147
错误	C2679	二进制“=”: 没有找到接受“bool”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	166
错误	C2679	二进制“=”: 没有找到接受“std::string”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	175
错误	C2679	二进制“=”: 没有找到接受“std::basic_string<char,std::char_traits<char>,std::allocator<char>>”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	205
错误	C2679	二进制“=”: 没有找到接受“const char [11]”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	206
错误	C2679	二进制“=”: 没有找到接受“const std::string”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	209
错误	C2679	二进制“=”: 没有找到接受“std::vector<std::vector<std::vector<std::string,std::allocator<_Ty>>,std::allocator<std::vector<_Ty,std::allocator<_Ty>>>>,std::allocator<std::vector<std::vector<_Ty,std::allocator<_Ty>>,std::allocator<std::vector<_Ty,std::allocator<_Ty>>>>>>”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.cpp	213
错误	C2666	“fwd::operator ==”: 3 个重载有相似的转换	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.h	62
错误	C2666	“fwd::operator ==”: 3 个重载有相似的转换	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.h	62
错误	C2666	“fwd::operator ==”: 3 个重载有相似的转换	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\keras_cpp_api.h	62
错误	C2664	“void std::vector<fwd::TrtLayerOutput,std::allocator<_Ty>>::push_back(const fwd::TrtLayerOutput &)”: 无法将参数 1 从“initializer list”转换为“fwd::TrtLayerOutput &&”	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\trt_keras_parser.cpp	157
错误	C2679	二进制“=”: 没有找到接受“initializer list”类型的右操作数的运算符(或没有可接受的转换)	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\trt_keras_parser.cpp	177
错误	C2664	“void std::vector<fwd::TrtLayerOutput,std::allocator<_Ty>>::push_back(const fwd::TrtLayerOutput &)”: 无法将参数 1 从“initializer list”转换为“fwd::TrtLayerOutput &&”	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\trt_keras_parser.cpp	185
错误	C2664	“void std::vector<fwd::TrtLayerOutput,std::allocator<_Ty>>::push_back(const fwd::TrtLayerOutput &)”: 无法将参数 1 从“initializer list”转换为“fwd::TrtLayerOutput &&”	fwd_keras	D:\Projects\tencent_forward\Forward\source\fwd_keras\keras_cvt\trt_keras_parser.cpp	201
错误	C4579	'nlohmann::detail::static_const<nlohmann::detail::from_json_fn>::value': in-class initialization for type 'const T' is not yet implemented; static member will remain uninitialized at runtime but use in constant-expressions is supported	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C2131	表达式的计算结果不是常数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C4579	'nlohmann::detail::static_const<nlohmann::detail::to_json_fn>::value': in-class initialization for type 'const T' is not yet implemented; static member will remain uninitialized at runtime but use in constant-expressions is supported	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C4579	'nlohmann::detail::static_const<nlohmann::detail::from_json_fn>::value': in-class initialization for type 'const T' is not yet implemented; static member will remain uninitialized at runtime but use in constant-expressions is supported	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C2131	表达式的计算结果不是常数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C4579	'nlohmann::detail::static_const<nlohmann::detail::to_json_fn>::value': in-class initialization for type 'const T' is not yet implemented; static member will remain uninitialized at runtime but use in constant-expressions is supported	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C4579	'nlohmann::detail::static_const<nlohmann::detail::from_json_fn>::value': in-class initialization for type 'const T' is not yet implemented; static member will remain uninitialized at runtime but use in constant-expressions is supported	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C2131	表达式的计算结果不是常数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C4579	'nlohmann::detail::static_const<nlohmann::detail::to_json_fn>::value': in-class initialization for type 'const T' is not yet implemented; static member will remain uninitialized at runtime but use in constant-expressions is supported	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2235
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2780	“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”: 应输入 2 个参数，却提供了 1 个	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2893	未能使函数模板“unknown-type nlohmann::adl_serializer<T,void>::from_json(BasicJsonType &&,ValueType &) noexcept(<expr>)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2513
错误	C2783	“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“ValueType nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept(<expr>) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“BasicJsonType nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer> nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2893	未能使函数模板“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept const”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2893	未能使函数模板“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“ValueType nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept(<expr>) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“BasicJsonType nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer> nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2893	未能使函数模板“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept const”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2893	未能使函数模板“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“ValueType nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept(<expr>) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“BasicJsonType nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2783	“nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer> nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) const”: 未能为“__formal”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2893	未能使函数模板“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept const”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2893	未能使函数模板“unknown-type nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get(void) noexcept”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	2516
错误	C2784	“const _Ty *std::begin(const std::valarray<_Ty> &)”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“const std::valarray<_Ty> &”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“_Ty *std::begin(std::valarray<_Ty> &)”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“std::valarray<_Ty> &”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“_Ty *std::begin(_Ty (&)[_Size]) noexcept”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“_Ty (&)[_Size]”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2893	未能使函数模板“unknown-type std::begin(const _Container &)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2893	未能使函数模板“unknown-type std::begin(_Container &)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“const _Elem *std::begin(std::initializer_list<_Elem>) noexcept”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“std::initializer_list<_Elem>”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2039	“iterator_category”: 不是“nlohmann::detail::iterator_traits<unknown-type,void>”的成员	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2146	语法错误: 缺少“>”(在标识符“iterator_category”的前面)	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“const _Ty *std::begin(const std::valarray<_Ty> &)”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“const std::valarray<_Ty> &”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“_Ty *std::begin(std::valarray<_Ty> &)”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“std::valarray<_Ty> &”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“_Ty *std::begin(_Ty (&)[_Size]) noexcept”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“_Ty (&)[_Size]”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2893	未能使函数模板“unknown-type std::begin(const _Container &)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2893	未能使函数模板“unknown-type std::begin(_Container &)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“const _Elem *std::begin(std::initializer_list<_Elem>) noexcept”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“std::initializer_list<_Elem>”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2039	“iterator_category”: 不是“nlohmann::detail::iterator_traits<unknown-type,void>”的成员	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2146	语法错误: 缺少“>”(在标识符“iterator_category”的前面)	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“const _Ty *std::begin(const std::valarray<_Ty> &)”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“const std::valarray<_Ty> &”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“_Ty *std::begin(std::valarray<_Ty> &)”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“std::valarray<_Ty> &”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“_Ty *std::begin(_Ty (&)[_Size]) noexcept”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“_Ty (&)[_Size]”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2893	未能使函数模板“unknown-type std::begin(const _Container &)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2893	未能使函数模板“unknown-type std::begin(_Container &)”专用化	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2784	“const _Elem *std::begin(std::initializer_list<_Elem>) noexcept”: 未能从“add_rvalue_reference<const ContiguousContainer>::type”为“std::initializer_list<_Elem>”推导 模板 参数	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2039	“iterator_category”: 不是“nlohmann::detail::iterator_traits<unknown-type,void>”的成员	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290
错误	C2146	语法错误: 缺少“>”(在标识符“iterator_category”的前面)	fwd_keras	D:\Projects\tencent_forward\Forward\source\third_party\json\single_include\nlohmann\json.hpp	4290

一共失败了3个

10>------ 已启动全部重新生成: 项目: ALL_BUILD, 配置: Debug x64 ------
10>  Building Custom Rule D:/Projects/tencent_forward/Forward/CMakeLists.txt
========== 全部重新生成: 成功 7 个，失败 3 个，跳过 0 个 ==========

这是什么原因？请问怎么解决？

Environment

TensorRT Version: 7.2.1.6 NVIDIA GPU: RTX 2080 SUPER NVIDIA Driver Version: 441.22 CUDA Version: 10.2 CUDNN Version: 8.2.0.53 Operating System: Windows 10 专业版 Python Version: 3.8.5

opened by xdqf2128 14

Help to support tensorflow slim "Flatten" pattern to Tensorrt.

There is a layer in tensorflow slim named "Flatten", it includes servel tensorflow operations like: "Shape", "StridedSlice" and "Reshape". The test code like this.

import numpy as np
import os


def create_tf_flatten(model_file):
    import tensorflow as tf
    import tf_slim as slim
    with tf.Session() as sess:
        x1 = tf.placeholder(shape=(None,299,299,3),dtype=tf.float32, name='x')

        op = slim.flatten(x1)

        sess.run(tf.global_variables_initializer())
        feed_dict = {x1: np.ones((1,299,299,3))}
        print(sess.run(op, feed_dict))

        graphdef = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['Flatten/flatten/Reshape'])
        tf.train.write_graph(graphdef, './', model_file, as_text=False)
        return feed_dict

def forward_transfer(model_file, dummy_input):
    import forward
    # 1. 构建 Engine
    builder = forward.TfBuilder()
    infer_mode = 'float32'

    builder.set_mode(infer_mode)
    tf_engine = builder.build(model_file, dummy_input)

    # save engine
    engine_path = os.path.splitext(model_file)[0] + '.engine'
    tf_engine.save(engine_path)


def test_forward(model_file, dummy_input):
    import forward
    engine_path = os.path.splitext(model_file)[0] + '.engine'
    # load saved engine
    tf_engine = forward.TfEngine()
    tf_engine.load(engine_path)

    inputs = dummy_input
    outputs = tf_engine.forward(inputs)
    print(outputs)


model_file = 'tf_model.pb'
create_tf_flatten(model_file)

x = {'x':np.ones([1,299,299,3],dtype='float32')}

forward_transfer(model_file, x)
test_forward(model_file, x)

Environment

TensorRT Version: 7.1.3.4 NVIDIA GPU: T4 NVIDIA Driver Version: 410.104 CUDA Version: 10.2 CUDNN Version: 8.0 Operating System: ubuntu 18.04 Python Version (if applicable): 3.6.9 Tensorflow Version (if applicable): 1.15.0 PyTorch Version (if applicable): 1.7.0

@aster2013 @yuanzexi

enhancement Framework: Tensorflow Stale

opened by ys0232 10

Segmentation fault (core dumped)

Describe the bug When I finished compiling cmake，then I copy forward.cpython-36m-aarch64-linux-gnu.so to a new directory, I run 'python test_forward.py' or ‘import forward’，an error appear ' Segmentation fault (core dumped) '

Environment

Device : Jetson Xavier NX System: Jetpack4.4 [L4T 32.4.4] TensorRT Version: 7.1.3 CUDA Version: 10.2.89 CUDNN Version: 8.0.0.180 Python Version (if applicable): 3.6.9 Tensorflow Version (if applicable): 1.15.2 Keras Version (if applicable): 2.1.5

Relevant Files

cmake successful

error information

some gdb information

What causes such error? Looking forward to the answer！！
enhancement Framework: Keras Stale

opened by evil-potato 10
reflectPad存在两个问题
Describe the bug 第一个问题是即使plugin继承了IPluginV2DynamicExt，但是不能用于动态输入当中，因为getOutputDimensions()的写法有误。第二个问题是这个插件的padding过程在某些GPU卡上输出结果是错的，具体原因未知。如在2080ti中可以正常输出结果，但是在A100出错

Environment

TensorRT Version: 7.2.1 NVIDIA GPU: 2080TI & A100 NVIDIA Driver Version: CUDA Version: 11.0 CUDNN Version: Operating System: Python Version (if applicable): Tensorflow Version (if applicable): PyTorch Version (if applicable):

Relevant Files

To Reproduce Steps to reproduce the behavior:

Go to '...'

Click on '....'

Scroll down to '....'

See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.
Stale
opened by Ehrich-wen 9
对‘fwd::TrtForwardEngine::Load(std::__cxx11::basic_string, std::allocator > const&)’未定义的引用
i have build and get the libtrt_engine.so, libfwd_torch.so

when i build the demo/fwd_cpp, i get the error: [ 50%] Linking CXX executable test_fwd_engine CMakeFiles/test_fwd_engine.dir/test_fwd_engine.cpp.o：在函数‘main’中： test_fwd_engine.cpp:(.text+0x1c8)：对‘fwd::TrtForwardEngine::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)’未定义的引用 collect2: error: ld returned 1 exit status CMakeFiles/test_fwd_engine.dir/build.make:110: recipe for target 'test_fwd_engine' failed make[2]: *** [test_fwd_engine] Error 1 CMakeFiles/Makefile2:96: recipe for target 'CMakeFiles/test_fwd_engine.dir/all' failed make[1]: *** [CMakeFiles/test_fwd_engine.dir/all] Error 2 Makefile:102: recipe for target 'all' failed make: *** [all] Error 2
opened by chenjun2hao 9
free(): invalid pointer
Describe the bug An error is reported at the end of the program, free(): invalid pointer.

Environment

TensorRT Version: 7.2.3.4 NVIDIA GPU: TITAN Xp NVIDIA Driver Version: 440.33.01 CUDA Version: 10.2 CUDNN Version: 8.0.4 Operating System: CentOS 8 Python Version (if applicable): 3.6.8 Tensorflow Version (if applicable): -- PyTorch Version (if applicable): 1.3.1 (libtorch cpu)

Relevant Files

To Reproduce Build Forward: cmake -DENABLE_LOGGING=OFF
-DENABLE_PROFILING=OFF
-DENABLE_DYNAMIC_BATCH=OFF
-DENABLE_TORCH=ON
-DBUILD_PYTHON_LIB=OFF
-DPYTHON_EXECUTABLE=/usr/bin/python3
-DENABLE_TENSORFLOW=OFF -DENABLE_KERAS=OFF
-DTORCH_CMAKE_PATH=/usr/local/lib/libtorch/share/cmake/Torch/
.. Steps to reproduce the behavior:

#include "fwd_torch/torch_engine/torch_engine.h" #include "fwd_torch/torch_engine/torch_infer.h"

create main function std::cout << "Hello World" << std::endl; after cmake and make, run this function.

Expected behavior None

Screenshots

Additional context The problem occurs in Forward cmake with libtorch. When we use Forward which cmake with python, the problem do not come up.
question Framework: PyTorch Stale
opened by BoBoer 9
keras.layer里Embedding的trt实现

感谢大佬做的项目！帮助很大，已star 我遇到了一个问题，我在做keras.layer.Embedding层的trt实现，想问下可否使用tensorRT自带的embLayerNormPlugin插件来实现功能？？我的python和C++源码放在下面： python代码： python运行的结果从一个32长度数组变成32x128tensor：
Stale

opened by YuzhouPeng 8
Cannot build the pb model (tensorflow), got "tf_graph_parser.cpp(48): Creating input desc is failed." .
Describe the bug I can convert the pytorch model (my resnet classification model) by the Forward. When I try to convert tensorflow pb model, I cannot build the saved pb model after obtaining the forword.**.so successfully and can import forward. When I tested this code:" engine = builder.build('./test_tfmodel.pb', dummy_inputs)", the problem "tf_graph_parser.cpp(48): Creating input desc is failed." was appeared and "已放弃(吐核)".

Environment

TensorRT Version: 7.2.1.6 NVIDIA GPU: GTX1080TI NVIDIA Driver Version: 450.80.02 CUDA Version: 11.0 CUDNN Version: 8.0.4 Operating System: 7.5 Python Version (if applicable): 3.6.13 Tensorflow Version (if applicable): tensorflow==1.15.0(cpu) PyTorch Version (if applicable): 1.7.1

Relevant Files

just testing the add model.

To Reproduce Steps to reproduce the behavior: 1. cmake .. \ -DTensorRT_ROOT=/data/wind/TensorRT-7.2.1.6/
-DENABLE_LOGGING=OFF
-DENABLE_PROFILING=OFF
-DENABLE_DYNAMIC_BATCH=OFF
-DBUILD_PYTHON_LIB=ON
-DPYTHON_EXECUTABLE=/root/anaconda3/envs/xyang/bin/python
-DENABLE_TORCH=OFF
-DENABLE_TENSORFLOW=ON
-DENABLE_KERAS=OFF

make -j then I import forward doesn't get wrong.

import numpy as np import forward # 1. 构建 Engine builder = forward.TfBuilder() # img = torch.randn(1, 784) img = np.ones([1,784], dtype='float32') dummy_inputs = {'inputs': img} infer_mode = 'float32' # float32 / float16 / int8_calib / int8 builder.set_mode(infer_mode) engine = builder.build('./test_tfmodel.pb', dummy_inputs)

then the last code got wrong!

Screenshots When I ./unit_test --gtest_filter=TestTfNodes.*, the following error(已放弃(吐核)) happened. But I can import forward successfully.
Framework: Keras Stale
opened by favorxin 8
编译出错找不到cublas_device库

你好，编译的时候报cublas_device找不到，具体如下：

Environment

TensorRT Version: 7.2.3.4 CUDA Version: 10.2 CUDNN Version: 7.4 Operating System: ubuntu18.04 Python Version (if applicable): 3.7 PyTorch Version (if applicable): 1.8

错误信息： [ 97%] Linking CXX shared library ../../bin/libfwd_torch.so [ 97%] Built target fwd_torch Scanning dependencies of target forward [ 98%] Building CXX object source/py_fwd/CMakeFiles/forward.dir/py_forward.cpp.o [100%] Linking CXX shared module ../../bin/forward.cpython-37m-x86_64-linux-gnu.so /usr/bin/x86_64-linux-gnu-ld: cannot find -lCUDA_cublas_device_LIBRARY-NOTFOUND collect2: error: ld returned 1 exit status source/py_fwd/CMakeFiles/forward.dir/build.make:117: recipe for target 'bin/forward.cpython-37m-x86_64-linux-gnu.so' failed make[2]: *** [bin/forward.cpython-37m-x86_64-linux-gnu.so] Error 1 CMakeFiles/Makefile2:687: recipe for target 'source/py_fwd/CMakeFiles/forward.dir/all' failed make[1]: *** [source/py_fwd/CMakeFiles/forward.dir/all] Error 2 Makefile:83: recipe for target 'all' failed

cublas_device这个库找不到，cuda10以后这个库就废弃了吧。

opened by tianyu06030020 7
make error

I've done the 'cmake' part successfully. But, when I run 'make', error happens like: [ 5%] Building NVCC (Device) object source/trt_engine/CMakeFiles/trt_engine.dir/trt_network_crt/plugins/emb_layer_norm_plugin/trt_engine_generated_emb_layer_norm_kernel.cu.o In file included from /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/block_discontinuity.cuh:37:0, from /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/block_histogram_sort.cuh:37, from /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/block_histogram.cuh:36, from /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/cub.cuh:38, from /home/agx/SCW/Forward-master/source/trt_engine/trt_network_crt/plugins/common/bert_plugin_util.h:33, from /home/agx/SCW/Forward-master/source/trt_engine/trt_network_crt/plugins/emb_layer_norm_plugin/emb_layer_norm_kernel.cu:36: /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:238:61: warning: missing terminating " character asprmt"bfi.b32 %0, %1, %2, %3;" : "=r"(ret) : ar"(x), "r"(x), b, in) - 1; ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:282:2: error: #else without #if #else ^~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:284:2: error: #endif without #if #endif ^~~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:294:2: error: #else without #if #else ^~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:296:2: error: #endif without #if #endif ^~~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:306:2: error: #else without #if #else ^~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:308:2: error: #endif without #if #endif} ^~~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:319:2: error: #else without #if #else ^~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:321:46: warning: missing terminating " character 3;" : "word(ret) : word("(x), "rc_bit-of("(x), flags() + z; ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:322:2: error: #endif without #if #endif ^~~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:334:2: error: #else without #if #else ^~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:336:46: warning: missing terminating " character 3;" : "word(ret) : word("(x), "rc_bit-of("(x), flags() + z; ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:337:2: error: #endif without #if #endif ^~~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:349:2: error: #else without #if #else ^~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:351:44: warning: missing terminating " character 3;" : "word(ret) : word("(x), "rc_lane("(x), flags() + z; ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:352:2: error: #endif without #if #endif ^~~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:371:61: warning: missing terminating " character asfma.rz.ffi.b32 %0, %1, %2, %3;" f"(d(ret)f: ar"(xf, "r"(xf, cr) - 1; ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:375:2: error: #endif without #if #endif // DOXYGEN_SHOULD_SKIP_THIS ^~~~~ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:383:20: warning: missing terminating " character vo orileasexit;")>()) ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:395:20: warning: missing terminating " character vo orileasllup;")>()x; ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:406:18: warning: missing terminating " character tef adIdx"urn x; ^ /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/block/specializations/../../block/../util_ptx.cuh:475:1: error: unterminated comment /** ^ In file included from /home/agx/SCW/Forward-master/source/trt_engine/trt_network_crt/plugins/common/bert_plugin_util.h:33:0, from /home/agx/SCW/Forward-master/source/trt_engine/trt_network_crt/plugins/emb_layer_norm_plugin/emb_layer_norm_kernel.cu:36: /home/agx/SCW/Forward-master/source/third_party/cub-1.8.0/cub/cub.cuh:54:10: fatal error: device/device_run_length_encode.cuh: No such file or directory #include "device/device_run_length_encode.cuh" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. CMake Error at trt_engine_generated_emb_layer_norm_kernel.cu.o.cmake:220 (message): Error generating /home/agx/SCW/Forward-master/build/source/trt_engine/CMakeFiles/trt_engine.dir/trt_network_crt/plugins/emb_layer_norm_plugin/./trt_engine_generated_emb_layer_norm_kernel.cu.o

source/trt_engine/CMakeFiles/trt_engine.dir/build.make:1591: recipe for target 'source/trt_engine/CMakeFiles/trt_engine.dir/trt_network_crt/plugins/emb_layer_norm_plugin/trt_engine_generated_emb_layer_norm_kernel.cu.o' failed make[2]: *** [source/trt_engine/CMakeFiles/trt_engine.dir/trt_network_crt/plugins/emb_layer_norm_plugin/trt_engine_generated_emb_layer_norm_kernel.cu.o] Error 1 CMakeFiles/Makefile2:556: recipe for target 'source/trt_engine/CMakeFiles/trt_engine.dir/all' failed make[1]: *** [source/trt_engine/CMakeFiles/trt_engine.dir/all] Error 2 Makefile:90: recipe for target 'all' failed make: *** [all] Error 2

Wonder what's wrong...

opened by CB-Jack-S 7
优化模型时报错
Describe the bug

使用for-torch 优化模型时出错，以resnest50d为例：代码如下： ` origion_model = timm.create_model('resnest50d', pretrained=True)

infer_mode = 'float16' # float32 / float16 / int8_calib / int8 jit_model = torch.jit.script(origion_model).cpu().eval()

model_path="resnest50d_bs_{}-half_{}_jit_cpu.pt".format(batch_size,half)

jit_model.save(model_path)

dummy = torch.randn(batch_size, 3, 244, 244)

builder = forward.TorchBuilder()

builder.set_mode(infer_mode)

engine = builder.build(model_path, dummy) print(engine) outputs = engine.forward(dummy) print(outputs) os._exit(-1) `

Environment

TensorRT Version: 7.2.3.4 NVIDIA GPU: T4 NVIDIA Driver Version: 440.33.01 CUDA Version: 10.2 CUDNN Version: 7 Operating System: ubuntu 18.04 Python Version (if applicable): 3.6 Tensorflow Version (if applicable): 2.3.4 PyTorch Version (if applicable): 1.10.1

Relevant Files

To Reproduce Steps to reproduce the behavior:

Go to '...'

Click on '....'

Scroll down to '....'

See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.
opened by kisisjrlly 0

Releases(v2.0.2)

v2.0.2(Nov 30, 2021)
ChangeLog

Added

Support tf.math.top_k, tf.math.greater, tf.math.less, tf.math.rsqrt, tf.math.maximum, tf.math.minimum, tf.fill ops

Update test models and save scripts

Update docs

Fixed

Fix std::round for float and half data type

Fix EmbeddingBag layer to avoid extending to undesired dimensions for tf.gather

Changed

Explicitly call .contiguous() on tensor in convolution operation

Allow Select layer to handle const inputs

Remove macro control in FullyConnected layer

Remove outdated plugin files

Source code(tar.gz)
Source code(zip)
v2.0.1(Nov 8, 2021)
ChangeLog

Fixed

fix torch_resize_creator

fix upsampling_trilinear_3d

Added

support torch::squeeze.

add utests of upsampling_trilinear_3d

Changed

cancel broadcastdim in trt_element_wise_creator.h, and add CHECK_EQ instead.

Source code(tar.gz)
Source code(zip)
v2.0.0(Nov 4, 2021)
ChangeLog

Added

Support TensorRT 8 (compatible with TensorRT 7) [TRT]

Support c10::prim::NumToTensor op in TrtElementWiseDesc [TORCH]

Support c10::aten::upsample_trilinear3d and c10::aten::upsample_nearest3d ops in TrtResizeDesc [TORCH]

Support Select op type in TrtSelectDesc [TF]

Add torch.matmul op unit test [TORCH]

Update docs

Changed

Integrate skipLayerNormPlugin into ForwardLayerNormPluginDynamic [TRT]

Switch Bert-related plugins to official TensorRT NvInfer Plugins [TRT]

Replace addScale with addScaleNd layer in TrtScaleDesc [TRT]

Source code(tar.gz)
Source code(zip)
v1.3.3(Oct 11, 2021)
ChangeLog

Changed

Clean up dependencies in demo

Source code(tar.gz)
Source code(zip)
v1.3.2(Sep 2, 2021)
ChangeLog

Added

Update ONNX-related demo dependencies

Update Forward documents

Fixed

Fix core dump issues when OnnxEngine is built in Python

Source code(tar.gz)
Source code(zip)
v1.3.1(Aug 26, 2021)
ChangeLog

Fixed

Fix demo dependencies

Source code(tar.gz)
Source code(zip)
v1.3.0(Aug 24, 2021)
ChangeLog

Added

Support ONNX framework [Usage]

Update Forward documents

Fixed

Fix SkipLayerNorm abnormal behavior when enabling ENABLE_DYNAMIC_BATCH

Source code(tar.gz)
Source code(zip)
v1.2.5(Aug 17, 2021)
ChangeLog

Fixed

Update SkipLayerNorm's usages in the trt_bert_creator.h.

Source code(tar.gz)
Source code(zip)
v1.2.4(Jul 30, 2021)
ChangeLog

Added

Add FuseTorchModule such that continuous TorchModulePlugin will be fused as single TorchModulePlugin.

Source code(tar.gz)
Source code(zip)
v1.2.3(Jul 19, 2021)
ChangeLog

Added

Documents related to Forward

Support to c10::prim::GetAttr and torch::ones

Source code(tar.gz)
Source code(zip)
v1.2.2(Jul 2, 2021)
ChangeLog

Added

Support Keras::Flatten

Support torch.pixel_shuffle and tf.depth_to_space

Source code(tar.gz)
Source code(zip)
v1.2.1(Jul 2, 2021)
ChangeLog

Added

Update fwd::DataType to support more DataType and its related usages

Changed

TorchModulePlugin is controlled by ENABLE_TORCH_PLUGIN, ENABLE_TORCH_PLUGIN=OFF as default.

Fixed

Fix torch_matmul not support ConstantInput

Fix torch_cast not support constantInput

Fix split not support minus dim

Refact tf_matmul_creator.h and torch_matrix_multiply_creator.h

Fix skip_layer_norm'no_skip not work well on DYNAMIC_BATCH

Fix: For TorchModulePlugin, only clone required attributes instead of all attributes.

Fix: Shuffle Layer is append to the Slice Layer of Torch-Select

Source code(tar.gz)
Source code(zip)
v1.2.0(Jun 15, 2021)
ChangeLog

Added

Support TorchModulePlugin: When some torch nodes are not supported, they will be encapsulated as a SubModule to reconstruct a TorchModulePlugin Layer for a TensorRT engine.

Fixed

Fixed namespace conflicts of BERT plugins: BERT plugins, inherited from TensorRT's BERT plugins, have the same global static variables with those of TensorRT, which leading to conflicts and even free-pointer core problem.

Changed

Update Keras forward interface to fwd::Tensor

Update unit_tests.

Update minimum Torch Version required to 1.7.0.

Source code(tar.gz)
Source code(zip)
v1.1.1(May 26, 2021)
ChangeLog

Fixed

Hot fix: Constant input checks in torch_element_wise_creator.h are fixed.

Changed

Symbol link README to README_CN

Source code(tar.gz)
Source code(zip)
v1.1.0(May 21, 2021)
ChangeLog

Added

Support Tensorflow [Reshape]

Fwd-Tf links to _pywarp_tensorflow_internal.so in Python-Tensorflow.

Changed

Logging usage: Using forward_log.conf to control the behavior of Logging. (See logging usage in ReadMe)

Source code(tar.gz)
Source code(zip)
v1.0.2(May 17, 2021)
ChangeLog

Added

Nodes supported for Torch: aten::unsqeeze_

Nodes supported for Keras: Dropout, ReduceSum

CUDA_VERSION and CUDA_ARCHs Check in CMake

Changed

Fwd-Tf depends on PyWrap_Tensorflow.so instead of Tensorflow_C_Lib

BUILD_PYTHON_LIB=ON is only valid on Linux Platform

include files related to libtrt_fwd_engine.so

elementwise and reduce implementations

Source code(tar.gz)
Source code(zip)
v1.0.1(Apr 8, 2021)
Changelog

Added

Demos for tutorial.

Support Torch >= 1.7.0

Changed

Requirements for TensorRT, PyTorch, and Keras.

Use cudaMemcpyAsync in PrepareInputBuffer.

Change GLIBCXX_USE_CXX11_ABI usage in CMakeLists.txt.

Source code(tar.gz)
Source code(zip)
v1.0.0(Mar 15, 2021)

Forward First Release.
Source code(tar.gz)
Source code(zip)

A library for high performance deep learning inference on NVIDIA GPUs.

Related tags

Overview

Forward - A library for high performance deep learning inference on NVIDIA GPUs

Forward

Features

Quick Start

Prerequisites

Build with CMake

CMake build arguments

Unit Test

Use Forward-Cpp

Use Forward-Python

More Usages

FAQ

Models & Operators

Models

Operators

Contribution

Contributors

License

Issues

Environment

Relevant Files

Environment

Environment

Environment

Relevant Files

Environment

Relevant Files

Environment

Relevant Files

Environment

Relevant Files

Environment

Environment

Relevant Files

Releases(v2.0.2)

v2.0.2(Nov 30, 2021)

ChangeLog

Added

Fixed

Changed

v2.0.1(Nov 8, 2021)

ChangeLog

Fixed

Added

Changed

v2.0.0(Nov 4, 2021)

ChangeLog

Added

Changed

v1.3.3(Oct 11, 2021)

ChangeLog

Changed

v1.3.2(Sep 2, 2021)

ChangeLog

Added

Fixed

v1.3.1(Aug 26, 2021)

ChangeLog

Fixed

v1.3.0(Aug 24, 2021)

ChangeLog

Added

Fixed

v1.2.5(Aug 17, 2021)

ChangeLog

Fixed

v1.2.4(Jul 30, 2021)

ChangeLog

Added

v1.2.3(Jul 19, 2021)

ChangeLog

Added

v1.2.2(Jul 2, 2021)

ChangeLog

Added

v1.2.1(Jul 2, 2021)

ChangeLog