Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

Related tags

Artificial Intelligence SCOPE
Overview

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

This is the implementation of the approach described in the paper:

Taosha Fan, Kalyan Vasudev Alwala, Donglai Xiang, Weipeng Xu, Todd Murphey, Mustafa Mukadam. Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021.

More demos are available at https://sites.google.com/view/scope-human

Install

git clone https://github.com/fantaosha/SCOPE.git
cd SCOPE
export SCOPE_ROOT=$(pwd)
mkdir build
cd build
cmake ../C++
make -j4

Usage

SMPL Model

Download the SMPL model and extract these.pkl files to $SCOPE_ROOT/model.

Preprocess

cd $SCOPE_ROOT/model
python3 preprocess.py smpl_male.npz YOUR_SMPL_MALE.pkl

Run

cd $SCOPE_ROOT
./build/bin/run --model ./model/smpl_male.npz --prior ./model/joint_prior.json --keypoint ./examples/keypoints.json --result ./examples/results.json

Dataset

2D and 3D Keypoints

The 2D and 3D keypoints estimates from AlphaPose and VideoPose3D can be downloaded from Google Drive.

2D Keypoint Index

0: nose
1: left eye
2: right eye
3: left ear
4: right ear
5: left upper arm
6: right upper arm
7: left elbow
8: right elow
9: left wrist
10: right wrist
11: left hip
12: right hip
13: left knee
14: right knee
15: left ankle
16: right ankle
17: head top
18: thorax
19: middle hip
20: left big toe
21: right big toe
22: left small toe
23: right small toe
24: left heel
25: right heel
26: chest
27: neck

3D Keypoint Index

0: middile hip
1: left hip
2: left knee
3: left ankle
4: right hip
5: right knee
6: right ankle
7: chest
8: thorax
9: neck
10: head top
11: left upper arm
12: left elbow
13: left wrist
14: right upper arm
15: right elbow
16: right wrist

Citation

@article{fan2021revitalizing,
title={Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation},
author={Fan, Taosha and Alwala, Kalyan Vasudev and Xiang, Donglai and Xu, Weipeng and Murphey, Todd and Mukadam, Mustafa},
journal={Proceedings of the IEEE/CVF International Conference on Computer Vision},
year={2021}
}
You might also like...
Real-Time Neural 3D Hand Pose Estimation from an Event Stream [ICCV 2021]

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream Project Page Index TRAIN.md -- how to train the model from scratch EVAL_REAL

null 23 Nov 7, 2022
HybridPose: 6D Object Pose Estimation under Hybrid Representation (CVPR 2020)
HybridPose: 6D Object Pose Estimation under Hybrid Representation (CVPR 2020)

HybridPose: 6D Object Pose Estimation under Hybrid Representations This repository contains authors' implementation of HybridPose: 6D Object Pose Esti

SONG, Chen 358 Nov 22, 2022
the code of paper
the code of paper "Optimizing Global Injectivity for Constrained Parameterization" (SIGGRAPH Asia 2021)

Optimizing Global Injectivity for Constrained Parameterization Xingyi Du, Danny M. Kaufman, Qingnan Zhou, Shahar Kovalsky, Yajie Yan, Noam Aigerman, T

Charles Du 4 Nov 10, 2022
heuristically and dynamically sample (more) uniformly from large decision trees of unknown shape

PROBLEM STATEMENT When writing a randomized generator for some file format in a general-purpose programming language, we can view the resulting progra

John Regehr 4 Feb 15, 2022
[3DV 2021] DSP-SLAM: Object Oriented SLAM with Deep Shape Priors
[3DV 2021] DSP-SLAM: Object Oriented SLAM with Deep Shape Priors

DSP-SLAM Project Page | Video | Paper This repository contains code for DSP-SLAM, an object-oriented SLAM system that builds a rich and accurate joint

Jingwen Wang 368 Dec 29, 2022
BladeDISC - BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.
BladeDISC - BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

BladeDISC Introduction Overview Features and Roadmap Frontend Framework Support Matrix Backend Support Matrix Deployment Solutions Numbers of Typical

Alibaba 517 Jan 4, 2023
Android hair/human segmentation demo by ncnn
Android hair/human segmentation demo by ncnn

ncnn_Android_human Android hair/human segmentation demo by ncnn PS:performance maybe poor.it's just a demo:) Reference: 1.https://github.com/Tencent/n

null 23 May 23, 2022
This code accompanies the paper
This code accompanies the paper "Human-Level Performance in No-Press Diplomacy via Equilibrium Search".

Diplomacy SearchBot This code accompanies the paper "Human-Level Performance in No-Press Diplomacy via Equilibrium Search". A very brief orientation:

Facebook Research 34 Dec 20, 2022
MasterAI decisively defeated 14 top human Texas hold'em poker professsionals in September 2020.

MasterAI-1.0-1vs1-Limit Introduction MasterAI is an AI poker dedicated to suport n-play (single- or multi-agent) Texas Hold'em imperfect-informatin ga

MasterAI 5 Mar 27, 2022
Comments
  • Replace Eigen::Matrix3X<double>

    Replace Eigen::Matrix3X

    Hi Taosha,

    In "C++/scope/utils/Setup.cpp", there are two places with "Eigen::Matrix3X", which seem to cause compilation error: "error: ‘Matrix3X’ is not a member of ‘Eigen’;". I fixed that by replacing them with "Eigen::Matrix<double, 3, Eigen::Dynamic>". Would you consider updating them in the repo?

    Thank you! Muchen

    opened by MuchenSun 2
  • error: ‘Matrix3X’ is not a member of ‘Eigen’

    error: ‘Matrix3X’ is not a member of ‘Eigen’

    This is a wonderful job! The following error occurred during compilation: SCOPE/C++/scope/utils/Setup.cpp:195:36: error: ‘Matrix3X’ is not a member of ‘Eigen’; did you mean ‘Matrix3Xd’? 195 | Measurements = Eigen::Map<Eigen::Matrix3X>( | ^~~~~~~~ | Matrix3Xd

    Excuse me, is there something wrong with the Eigen version I installed? What is the correct Eigen version?

    opened by Allen-lz 1
  • How to do inference using SMPL-H model?

    How to do inference using SMPL-H model?

    Thank you for your great work! I've seen that you have showed the results of applying your method on smpl-h. Could you please tell me how to use smpl-h for inference rather than smpl. THANK YOU VERY MUCH !!!!!

    opened by luyr 1
  • Could you please release the code of training the joint prior from a scratch?

    Could you please release the code of training the joint prior from a scratch?

    Thank you for sharing this awesome work! I was trying to reproduce the normalizing flow used in the joint prior parts, and I noticed that the network structure (e.g. the PReLU implementation) might be different from the RealNVP, or Low-capacity version of normalizing flows mentioned in https://arxiv.org/abs/2003.10350 Could you please release the method or the code for training this network?

    opened by Phonicavi 3
Owner
Taosha Fan
Taosha Fan
GitHub
NeeDrop: Self-supervised Shape Representation from Sparse Point Clouds using Needle Dropping

NeeDrop: Self-supervised Shape Representation from Sparse Point Clouds using Needle Dropping by: Alexandre Boulch, Pierre-Alain Langlois, Gilles Puy a

valeo.ai 26 Sep 6, 2022
The dgSPARSE Library (Deep Graph Sparse Library) is a high performance library for sparse kernel acceleration on GPUs based on CUDA.

dgSPARSE Library Introdution The dgSPARSE Library (Deep Graph Sparse Library) is a high performance library for sparse kernel acceleration on GPUs bas

dgSPARSE 59 Dec 5, 2022
A project to control Petoi Bittle using human pose

Petoi Bittle Controlled by Pose A project to control Petoi Bittle by human pose Human pose is estimated using MoveNet and TensorFlow Lite YouTube Syst

iwatake 11 Dec 26, 2021
Distributed Pose Graph Optimization

Distributed Pose Graph Optimization

MIT Aerospace Controls Laboratory 99 Dec 28, 2022
nanoPGO: A header-only library for Pose-Graph-Optimization in SE(2).

nanoPGO nanoPGO: A header-only library for Pose-Graph-Optimization in SE(2). 1. Description This repo is an implementation of 2D Pose Graph Optimizati

道锋 3 Jul 7, 2022
Fast and robust certifiable relative pose estimation

Fast and Robust Relative Pose Estimation for Calibrated Cameras This repository contains the code for the relative pose estimation between two central

null 42 Dec 6, 2022
Android hand detect and pose estimation by ncnn

ncnn_nanodet_hand 1.hand detect:用nanode-m训练了个hand detect模型, 2.hand pose:用CMU的数据集训练了个ghostnet作为backbone模仿pfld的handpose模型 3.推理:handpose.cpp单独检测pose，nano

null 80 Jan 3, 2023
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV Workshop @ CVPR 2021.

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 47 Nov 18, 2022
CUDA-accelerated Apriltag detection and pose estimation.

Isaac ROS Apriltag Overview This ROS2 node uses the NVIDIA GPU-accelerated AprilTags library to detect AprilTags in images and publishes their poses,

NVIDIA Isaac ROS 46 Dec 26, 2022
Android MoveNet pose estimation by ncnn

ncnn_Android_MoveNet Android MoveNet single human pose estimation by ncnn this project is a ncnn Android demo for MoveNet, it depends on ncnn library

FeiGeChuanShu 93 Dec 31, 2022