MozoLM: A language model (LM) serving library

Overview

GitHub license C++ version Build Tests (Linux) Build Tests (macOS) Build Tests (Windows) Build Tests (Android) Build Tests (iOS)

MozoLM

A language model serving library, with middleware functionality including mixing of probabilities from disparate base language model types and tokenizations along with RPC client/server interactions.

License

MozoLM is licensed under the terms of the Apache license. See LICENSE for more information.

Mandatory Disclaimer

This is not an officially supported Google product.

Issues
Owner
Google Research
Google Research
A flexible, high-performance serving system for machine learning models

XGBoost Serving This is a fork of TensorFlow Serving, extended with the support for XGBoost, alphaFM and alphaFM_softmax frameworks. For more informat

iQIYI 120 Aug 1, 2022
faiss serving :)

faiss-server faiss-server provides gRPC services to for similarity search using faiss. It is written in C++ and now supports only CPU environments. In

null 104 Jul 14, 2022
Deploying Deep Learning Models in C++: BERT Language Model

This repository show the code to deploy a deep learning model serialized and running in C++ backend.

null 42 Mar 24, 2022
Deploy ultralytics Yolov5 pretained model with C++ language

Introdution Deploy ultralytics Yolov5 pretained model with C++ language ; Env GCC 7.5 Opencv 4.5.4 Get ONNX Model go to yolov5 release page download y

Xee 36 Jul 27, 2022
HackySAC is a C++ header only library for model estimation using RANSAC.

HackySAC HackySAC is a C++ header only library for model estimation using RANSAC. Available under the MIT license. Examples Minimal working example fo

Jonathan Broere 1 Oct 10, 2021
power-grid-model is a Python library for steady-state distribution power system analysis

Power Grid Model power-grid-model is a Python library for steady-state distribution power system analysis. The core of the library is written in C++.

Alliander Open Source 41 Aug 5, 2022
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Aug 1, 2022
Object Based Generic Perception Object Model

This model is a highly parameterizable generic perception sensor and tracking model. It can be parameterized as a Lidar or a Radar. The model is based on object lists and all modeling is performed on object level.

TU Darmstadt - FZD 5 Jun 11, 2022
This is a small example project, that showcases the possibility of using a surrogate model to estimate the drag coefficient of arbitrary triangles.

flowAroundTriangles This is a small example project, that showcases the possibility of using a surrogate model to estimate the drag coefficient of arb

null 5 Jun 23, 2022
Operating system model using an assembler RISC-V RV32I instruction set.(development)

General Information Operating system model using an assembler RISC-V RV32I instruction set.(development) С++ Standard - c++17 gcc 9.3.0(Linux,unicode)

Alex Green 1 Dec 21, 2021
Hardware-accelerated DNN model inference ROS2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU.

Isaac ROS DNN Inference Overview This repository provides two NVIDIA GPU-accelerated ROS2 nodes that perform deep learning inference using custom mode

NVIDIA Isaac ROS 42 Jul 18, 2022
model infer framework with multithreads based on PaddleX

model_infer_multiThreads (最近更新:2021-10-28 增加了原生的所有api接口,支持clas/det/seg/mask) 该repo基于PaddleX模型推理动态链接库的接口代码进行修改,支持多线程并行访问。大部分代码均来自paddleX的model_infer.cp

Liang Su 14 Apr 27, 2022
It is a simple AI model running at Art-Pi.

Garbage Classification Device at Embedded AI Summer School 2021 of Nanjing IC Training Base Based on garbage datasets online, apply TensorFlow to get

MJ.XU 6 Jul 17, 2022
A lightweight 2D Pose model can be deployed on Linux/Window/Android, supports CPU/GPU inference acceleration, and can be detected in real time on ordinary mobile phones.

A lightweight 2D Pose model can be deployed on Linux/Window/Android, supports CPU/GPU inference acceleration, and can be detected in real time on ordinary mobile phones.

JinquanPan 49 Aug 1, 2022
A GKR-based zero-knowledge proof protocol for CNN model inference.

zkCNN Introduction This is the implementation of this paper, which is a GKR-based zero-knowledge proof for CNN reference, containing some common CNN m

null 26 Jun 21, 2022
Movenet cpp deploy; model transformed from tensorflow

MoveNet-PaddleLite Adapted from PaddleDetection; Movenet cpp deploy based on PaddleLite; Movenet model transformed from tensorflow; 简介 Movenet是近年的优秀开源

null 9 May 28, 2022
Pedestrian simulator powered by the social force model

Pedestrian simulator powered by the social force model

Social Robotics Lab, University of Freiburg 285 Aug 5, 2022
A Tool for Verifying Neural Networks using SMT-Based Model Checking

Project Title QNNVerifier Description A Tool for Verifying Neural Networks using SMT-Based Model Checking. Using Frama-C and ESBMC as the backends. Yo

null 2 Dec 11, 2021
Mmdeploy - OpenMMLab Model Deployment Framework

Introduction English | 简体中文 MMDeploy is an open-source deep learning model deployment toolset. It is a part of the OpenMMLab project. Major features F

OpenMMLab 1.2k Aug 9, 2022