This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicity.

Overview

Fast Face Classification (F²C)

This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicity.

Training on ultra-large-scale datasets is time-consuming and takes up a lot of hardware resource. Therefore we design a dul-data loaders and dynamic class pool to deal with large-scale face classification.

Pipeline

Arch

Preparation

As FFC contains LRU module, so you may use lru_python_impl.py or instead compile the code under lru_c directory.

If you choose lru_python_impl.py, you should rename lru_python_impl.py to lru_utils.py. As lru is not the bottleneck of the training procedure, so feel free to use python implementation, though the C++ implementation is 5~10 times faster than python version.

Compile LRU (optional)

Command to build LRU

cd lru_c
mkdir build
cd build
cmake ..
make
cd ../../ && ln -s lru_c/build/lru_utils.so .

You can compare this two implementation using lru_c/python/compare_time.py

Database

Training

In main.py, you should provide the path to your training db at line 152-153.

args.source_lmdb = ['/path to msceleb.lmdb']
args.source_file = ['/path to kv file']

We choose lmdb as the format of our training db. Each element in source_file is the path to a text file, each line of which represents lmdb_key label pairs. You may refer to LFS for more details.

Now you can modify train_ffc.sh. Before running the training, you should set the port number and queue_size. queue_size is a trade-off term that controls the performance and the speed. Larger queue_size means higher performance at the cost of time and GPU resource. It can be any positive integer. The common setting is 1%, 0.1%, 0.001 % of the total identities.

Notice

The difference between r50 and ir50 is that r50 requires 224 × 224 images as input while ir50 requires 112 × 112 as what does by ArcFace. The network ir50 comes from ArcFace.

Evaluation

We provide the whole test script under evaluation_code directory. Each script requires the directory to the images and test pair files.

Tips

Code in evaluation_code/test_megaface.py is much faster than official version. It's also applicable to extremely large-scale testing.

You might also like...
An Open-Source Analytical Placer for Large Scale Heterogeneous FPGAs using Deep-Learning Toolkit
An Open-Source Analytical Placer for Large Scale Heterogeneous FPGAs using Deep-Learning Toolkit

DREAMPlaceFPGA An Open-Source Analytical Placer for Large Scale Heterogeneous FPGAs using Deep-Learning Toolkit. This work leverages the open-source A

Very portable voice recorder with speech recognition.

DictoFun Small wearable voice recorder. NRF52832-based. Concept Device was initiated after my frustration while using voice recorder for storing ideas

NLP-based perching trajectory generation presented in our paper
NLP-based perching trajectory generation presented in our paper "Perception-Aware Perching on Powerlines with Multirotors".

Perception-Aware Perching on Powerlines with Multirotors This repo contains the code for the NLP-based perching trajectory generation presented in our

The repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching.
The repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching.

Geometric Structure Preserving Warp for Natural Image Stitching This repository contains our dataset and C++ implementation of the CVPR 2022 paper, Ge

Efficient training of deep recommenders on cloud.
Efficient training of deep recommenders on cloud.

HybridBackend Introduction HybridBackend is a training framework for deep recommenders which bridges the gap between evolving cloud infrastructure and

A system to flag anomalous source code expressions by learning typical expressions from training data
A system to flag anomalous source code expressions by learning typical expressions from training data

A friendly request: Thanks for visiting control-flag GitHub repository! If you find control-flag useful, we would appreciate a note from you (to niran

《Graph Optimization Approach to Range-based Localization》; UWB localization

This is modified from localization . Thanks for his work for uwb localizaiton. 代码1:https://github.com/qxiaofan/awesome-uwb-localization 代码2:https://gi

OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation
OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation

Build Type Linux MacOS Windows Build Status OpenPose has represented the first real-time multi-person system to jointly detect human body, hand, facia

The SCRFD face detection, depends on ncnn library and opencv
The SCRFD face detection, depends on ncnn library and opencv

The SCRFD face detection, depends on ncnn library and opencv

Comments
  • lmdb dataSets

    lmdb dataSets

    您好,我按照推荐的方式,构建了一个比较小的lmdb 数据测试流程,但是出现了错误。能帮忙分析下:(1)数据集生成方式是否正确,能否提供一个你们生成的数据集信息;(2)这个错误如何解决 ①数据集信息如下:source_file.txt 的格式为: img_name.jpg label。生成lmdb的参数为 --shuffle=true --backend=lmdb --resize_width=128 --resize_height=128
    --encoded=true
    --encode_type=jpg ② 错误如下: File "main.py", line 51, in train_one_epoch images1, images2, id_indexes = next(id_iter) File "FFC_py36/lib/python3.6/site-ackages/torch/utils/data/dataloader.py", line 363, in next data = self._next_data() File "FFC_py36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 964, in _next_data raise StopIteration StopIteration

    opened by MSZLean 2
  • 'FFC' object has no attribute '_momentum_update_gallery'

    'FFC' object has no attribute '_momentum_update_gallery'

    请问下作者,我加载数据运行FFC项目时,提示如下错误,您碰到类似情况吗 File "FFC-master/ffc_ddp.py", line 182, in forward_impl_rollback self._momentum_update_gallery() # update the gallery net File "FFC-py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 772, in __getattr__type(self).name, name)) torch.nn.modules.module.ModuleAttributeError: 'FFC' object has no attribute '_momentum_update_gallery'

    opened by MSZLean 2
Owner
null
The official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach

Graph Optimizer This repo contains the official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averagin

Chenyu 107 Nov 13, 2022
Swapping face using Face Mesh with TensorFlow Lite

demo.mp4 Aiine Transform (アイン変換) Swapping face using FaceMesh. (could be used to unveil masked faces) Tested Environment Computer Windows 10 (x64) + V

iwatake 17 Apr 26, 2022
Gesture Recognition Toolkit (GRT) is a cross-platform, open-source, C++ machine learning library designed for real-time gesture recognition.

Gesture Recognition Toolkit (GRT) The Gesture Recognition Toolkit (GRT) is a cross-platform, open-source, C++ machine learning library designed for re

Nicholas Gillian 788 Nov 16, 2022
Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation"

SSM-VLN Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation". Environment Installation Download Room-to-Room

hanqing 34 Aug 24, 2022
Fairring (FAIR + Herring) is a plug-in for PyTorch that provides a process group for distributed training that outperforms NCCL at large scales

Fairring (FAIR + Herring): a faster all-reduce TL;DR: Using a variation on Amazon’s "Herring" technique, which leverages reduction servers, we can per

Meta Research 45 Oct 6, 2022
FG-Net: Fast Large-Scale LiDAR Point Clouds Understanding Network Leveraging Correlated Feature Mining and Geometric-Aware Modelling

FG-Net: Fast Large-Scale LiDAR Point Clouds Understanding Network Leveraging Correlated Feature Mining and Geometric-Aware Modelling Comparisons of Running Time of Our Method with SOTA methods RandLA and KPConv:

Kangcheng LIU 76 Nov 8, 2022
Square Root Bundle Adjustment for Large-Scale Reconstruction

Square Root Bundle Adjustment for Large-Scale Reconstruction

Nikolaus Demmel 193 Nov 4, 2022
[NeurIPS 2021 Spotlight] Learning to Delegate for Large-scale Vehicle Routing

Learning to Delegate for Large-scale Vehicle Routing This directory contains the code, data, and model for our NeurIPS 2021 Spotlight paper Learning t

null 42 Nov 14, 2022
Codebase for "SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems"

Codebase for "SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems"

Beidi Chen 1k Oct 24, 2022