Motion Cam is a camera application for Android that replaces the entire camera pipeline.

Overview

About

Motion Cam is a camera application for Android that replaces the entire camera pipeline. It consumes RAW images and uses computational photography to combine multiple images to reduce noise.

You can install it from the Play Store

Dual exposure

Dual exposure is similar to the feature found in the Google Camera. The two sliders control the exposure compensation and tonemapping.

GIF

Zero shutter lag burst capture

GIF

Overview

Camera Preview

Motion Cam uses the GPU to generate a real time preview of the camera from its RAW data. It uses a simplified pipeline to produce an accurate representation of what the final image will look like. This means it is possible to adjust the tonemapping, contrast and colour settings in real time.

Noise Reduction

The denoising algorithm uses bayer RAW images as input. Motion Cam treats the RAW data as four colour channels (red, blue and two green channels). It starts by creating an optical flow map between a set of images and the reference image utilising Fast Optical Flow using Dense Inverse Search. Then, each colour channel is transformed into the wavelet domain using a dual tree wavelet transform. The wavelet coefficients are fused with the low pass subband acting as a guide to minimize artifacts due to errors in the optical flow map from occlusion or alignment failure.

The amount of noise present in the image is used to determine how many RAW images are merged together. It is estimated from the high frequency subband of the wavelet transform. A well-lit scene may not need much noise reduction whereas a low light scene will have greater noise and require more images.

Demosaicing

Most modern cameras use a bayer filter. This means the RAW image is subsampled and consists of 25% red, 25% blue and 50% green pixels. There are more green pixels because human vision is most sensitive to green light. The output from the denoising algorithm is demosaiced and colour corrected into an sRGB image. Motion Cam uses the algorithm Color filter array demosaicking: New method and performance measures by Lu and Tan.

Tonemapping

Motion Cam uses the algorithm exposure fusion for tonemapping. The algorithm blends multiple different exposures to produce an HDR image. Instead of capturing multiple exposures, it artificially generates the overexposed image and uses the original exposure as inputs to the algorithm. The shadows slider in the app controls the overexposed image.

Sharpening and Detail Enhancement

The details of the image are enhanced and sharpened with the Guided filter. Motion Cam uses two levels to enhance finer and courser details of the output.

Getting started

MacOS

Install the following dependencies:

brew install cmake llvm python

Set the environment variables:

export ANDROID_NDK=[Path to Android NDK]
export LLVM_DIR=/usr/local/Cellar/llvm/[Installed LLVM version]

Run the ./setupenv script to compile the dependencies needed by the project.

Ubuntu

Install the following dependencies:

apt install git build-essential llvm-dev cmake clang libclang-dev

Set the environment variables:

export ANDROID_NDK=[Path to Android NDK]
export LLVM_DIR=/usr

Run the ./setupenv script to compile the dependencies needed by the project.

Running the application

After setting up the environment, open the project MotionCam-Android with Android Studio. It should compile and run.

Generating code

MotionCam uses Halide to generate the code for most of its algorithms. The generators can be found in libMotionCam/generators. If you make any changes to the generator sources, use the script generate.sh to regenerate them.

Issues
  • S21+ 8.1.0-b0

    S21+ 8.1.0-b0

    Just updated to 8.1.0-b0. MotionCam 0.15 Tools PC can't make preview nor to extract dngs - directly blows up. Previous version 013 - the same

    Will have to go back to 8.0.8-b3, sadly

    opened by nubbie61 38
  • frame rate & audio synchronization issues (Raw Video)

    frame rate & audio synchronization issues (Raw Video)

    Not sure which of the following issues are specific to my phone hardware (Google Pixel 5) and which ones might be fixable in Motion Cam's code:

    • Not all frame rates offered in the Raw Video module do actually work. So far, I've had success with 25p and 30p; in those settings, the camera records the specified frame rate with synchronous audio. At 24p, however, the camera actually records 30p and tags it as 24p so that it plays back as 80% slo-mo with asynchronous sound. [If that footage is conformed back to 30p, video and sound will be synchronous.] I wouldn't be surprised, though, if those frame rate limitations were a limitation of my phone hardware (since 25p and 30p are the standard PAL/European and NTSC/American tv and video frame rates).
    • Even when choosing fully working frame rates, audio recording starts several frames (ca. 6.5 frames at 25p) later than CinemaDNG video recording. As a result, Davinci Resolve doesn't recognize Motion Cam's raw video recordings as proper CinemaDNG [i.e. raw video with sychronous audio], but imports the DNG image sequence and the .wav audio file as two separate files/assets. In the timeline, one then needs to manually align each audio track with the end of its corresponding video clip to have video with sound. Maybe this could be avoided by slightly delaying the beginning of a new video recording so that audio can catch up with video?
    • There can be framedrops in raw video/CinemaDNG material recorded with Motion Cam, which become visible as jumps during video playback. This also happens when recording in lower resolutions with a cropped sensor. It would be very nice if Motion Cam [in case it has ways of detecting framedrops during recording at all] had a configurable option in its setup menu to automatically stop the recording of a raw video clip whenever a frame has been dropped.

    That said, Motion Cam is truly great if not groundbreaking. I am extremely grateful for all your fantastic work!

    opened by nocantsin 31
  • RAW Video - added 60 and 120 FPS options, increased RAM limit

    RAW Video - added 60 and 120 FPS options, increased RAM limit

    There are asterisks because you will need at least 3GB of RAM to run 4K 21:9 RAW Video at 60 FPS, and it will only be enough for about 4 seconds. If your device has a lot of RAM (8 or more GB), you can try increasing the limit to 4GB to improve the performance. 120 fps may not work without binning because as far as I understand MotionCam always requests full resolution RAW from the API, and some devices are capable of 120 only at 16:9 crop (you can try yourself with custom profile in FreeDCam)

    And happy holidays everyone!

    opened by ghost 27
  • Problem when recording raw video in version 8.0.6 (any frame rate)

    Problem when recording raw video in version 8.0.6 (any frame rate)

    Hello. Before addressing the issue, I wanted to thank you for creating this wonderful app. Issue: After updating to the latest version of the app (8.0.6) , my device (galaxy note 10 plus) can't record raw video at any frame rate (even supported frame-rates such as 24fps....). I will happily give or send any information that can help in troubleshooting this problem. Thanks again for putting the time and effort into developing this app.

    opened by He5am2099 23
  • 0.container format question

    0.container format question

    Hello mirsadm, greatly appreciate the work on this app, it is honestly a work of art..

    My question is from the original raw video io stream/container that is a custom format, can we convert that straight to video without the conversion to dng? Tried importing the dngs to resolve but the sequence was giving me problems but I didn't mess with it too long yet. Also read that there was a converter on Linux maybe and was wondering where we could get that, thank you very much and i really enjoy this app, thank you for your hard work.

    opened by jdorman12 19
  • Option (or default behavior) to remember last used mode (video vs. photo)

    Option (or default behavior) to remember last used mode (video vs. photo)

    It would be more natural and of great convenience if the app remembers previously used mode (video vs. photo).

    Right now by default it is in Photo mode (or Night photo mode) when opening the app. And to shoot only videos during video sessions one must always select "Raw video" every time the app is opened. One could miss great moments when shooting videos while forgetting about active default Photo mode.

    opened by darkblaze69 14
  • Too much lens

    Too much lens

    Smartphones don't have that many lenses. There is no connection to some new lenses or the picture slows down. In addition, the telephoto lens completely stopped working. On previous versions it works. 8.0.8-beta1

    Screenshot_2022-01-29-23-58-37-826_Motion Cam

    opened by gzewani 13
  • Failed DNG conversion of RAW video in 8.0.0-beta0

    Failed DNG conversion of RAW video in 8.0.0-beta0

    When I try to convert raw video into DNG sequence in "manage video" menu it changes the button to "queued" but progress bar immediately disappears and nothing else happens. It creates empty folder with proper name and attempt counter for each try, and if I re-enter into this conversion menu it adds "This video has been exported" sign to the clip. The clip looks OK - proper capture time, fps, frame count and preview strip. Poco X3 6G/128 NFC

    opened by slonick81 13
  • Can your processing work on bayer data from non-android system?

    Can your processing work on bayer data from non-android system?

    Hi, I have been doing some work on raw CFA data coming from raspberry pi having Sony camera sensors. I have used libraw/dcraw in the past to get previewable image from such data. I came across your repository and it seems that you are using Halide for real time demosaicing and image stacking for noise reduction which i find interesting.

    I tried creating a metadata file containing information of the RGGB bayer along with raw bayer files and feed it to your algorithm but the output is messed up. It seems a channel/color issue. So my question is that whether your implementation is only specific to Android bayer data?

    Here is my metadata file:

    {
        "colorIlluminant2": "standarda",
        "whiteLevel": 1095,
        "colorMatrix1": [1.506935715675354, -0.8880840539932251, -0.13350626826286316, -0.53555202484130859, 1.5421112775802612, 0.1474035233259201, -0.019240813329815865, 0.095311708748340607, 0.76038378477096558],
        "apertures": [1.6959937810599961],
        "colorMatrix2": [0.82994842529296875, -0.27388653159141541, -0.097103022038936615, -0.57290846109390259, 1.3635042905807495, 0.17090092599391937, -0.086338728666305542, 0.13636419177055359, 0.53642231225967407],
        "sensorArrangment": "rggb",
        "forwardMatrix2": [],
        "blackLevel": [64, 64, 64, 64],
        "focalLengths": [4],
        "colorIlluminant1": "d65",
        "writeDNG": true,
        "frames": [{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame0.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        },{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame1.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        },{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame2.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        },{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame3.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        },{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame4.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        },{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame5.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        },{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame6.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        },{
            "exposureTime": 333333333.33333331,
            "orientation": 0,
            "timestamp": "1598542181000000",
            "filename": "frame7.raw",
            "asShotNeutral": [0.49100935459136963, 1, 0.4330725371837616],
            "width": 4032,
            "rowStride": 4032,
            "height": 3024,
            "colorCorrectionGains": [0, 0, 0, 0],
            "lensShadingMapHeight": 0,
            "lensShadingMapWidth": 0,
            "iso": 800
        }],
        "referenceTimestamp": "1598542181000000",
        "pixelFormat": "raw10",
        "postProcessingSettings": {
            
        },
        "forwardMatrix1": []
    }
    

    Note that i did not have information about the rowStride so i set it same as the width. Since i don't have the lensShadingMap data, i set lensShadingMapHeight and lensShadingMapWidth to 0. This is what the final output image looks like: Screenshot 2021-04-14 at 15 14 01

    Is the above json enough for the algorithm to work with or does it need additional metadata?

    opened by f0enix 11
  • 8.0.7 beta 4 crashes on Google Pixel

    8.0.7 beta 4 crashes on Google Pixel

    8.0.7 beta 4 crashes on a Google Pixel (6 Pro) with the same symptoms and error message as in the past: After starting the app, the camera display remains black except for the UI elements, and after a few seconds, the app crashes with the pop-up message "Error: The camera encountered an error".

    A log file is attached here.

    motion_cam-8.0.7-beta4-crash_log.txt.gz

    opened by nocantsin 10
  • The camera can't see anything since 8.0.0-beta0

    The camera can't see anything since 8.0.0-beta0

    I use Sony Xperia 5 II. This is how the app looked like since 7.0 to 7.0.8-beta5: 7 0

    This is how it looks like since 8.0.0-beta0: 8 0 3

    It's just not that the preview doesn't work, the camera can't capture anything at all, be it photos or videos.

    opened by mido830 10
  • Feature Request: custom FPS for RAW video

    Feature Request: custom FPS for RAW video

    Please add a custom FPS option for the RAW video. This is what cinema cameras like RED can do. It's better to be able to set a float value (at least 2-3 decimal places), but an integer would also be good. Even if there would be problems and crashes, just make a disclaimer message about possible instability. But users would be able to set any possible FPS that works.

    I personally need 12 FPS (can't do 24 because my phone is stable only up to 15). If you would make custom FPS as a paid feature, please add 12 FPS to the existing list. I can neither afford this app nor a better phone.

    Thanks.

    opened by yaroslav-semeniuk 0
  • feature request: save all burst shots button

    feature request: save all burst shots button

    Usually when I do a burst shot I want more than just one photo, most of the time I would like the whole sequence.

    But I don't seem to be able to select more than one photo or store them all. Or ist it possible to just copy the dngs out of a temp folder?

    A "select all" button an the ability to tap to select more than one to be saved would be essential.

    opened by nurps 2
  • Unusable on ZTE Axon 30 Ultra in Version 9.0.5

    Unusable on ZTE Axon 30 Ultra in Version 9.0.5

    opened by LuckyBastard73 14
  • Add RAW DNG stack option from gallery

    Add RAW DNG stack option from gallery

    Hello Mirsad! Thank you for making this great tool for photography. Please add an option in the app to choose dngs from gallery that are taken with different camera to stack. It would be really nice if you could add that. Thank you again!

    opened by KirChi123 0
Releases(v0.31)
Owner
Mirsad Makalic
Mirsad Makalic
Growtopia android modmenu with ImGui

Growtopia Android ImGUI Growtopia android modmenu with ImGUI. Features Built with ImGUI. Coming Soon None Requirements The following dependencies are

ZTz 54 Jul 31, 2022
A ImGui Application with Multi Viewports and Docking using D3D11

ImGui-Application Informations A ImGui Application with Multi Viewports and Docking using D3D11 Build You need the DirectX SDK, here the Download link

Argon Projects 9 May 3, 2022
✔️The smallest header-only GUI library(4 KLOC) for all platforms

Welcome to GUI-lite The smallest header-only GUI library (4 KLOC) for all platforms. 中文 Lightweight ✂️ Small: 4,000+ lines of C++ code, zero dependenc

null 6.3k Jul 31, 2022
EdgeKiller is a simple application that fully replaces Microsoft Edge with the Browser of choice.

EdgeKiller EdgeKiller is a simple application that fully replaces Microsoft Edge with the Browser of choice, while also intercepting all the microsoft

Jan Ochwat 2 Nov 30, 2021
Malware that replaces copied crypto addresses with yours.

Crypto-Clipper Malware which replaces copied crypto addresses with yours. ?? Features Multiple cryptos supported (BTC, LTC, ETH, DOGE, DASH, NEO, BCH,

Voltic 7 Aug 7, 2022
A motion-activated LED light strip controller. Supports up to two motion detectors.

A simple standalone motion activated controller for WS2812b LED lights Version 0.30 adds the ability to change settings from a web interface without r

null 4 Mar 12, 2022
Convert ASF/AMC motion capture files to BVH motion capture files.

About amc2bvh is a utility that converts a pair of files, one in the Acclaim Skeleton Format (ASF) and the other in the Acclaim Motion Capture (AMC) f

Tom Copeland 6 Jun 24, 2022
Sentry-Picam is a simple wildlife / security camera solution for the Raspberry Pi Zero W, providing 1080p/30fps motion activated H.264 video capture.

Sentry-Picam is a simple wildlife / security camera solution for the Raspberry Pi Zero W, providing 1080p/30fps motion activated H.264 video capture.

null 121 Jul 29, 2022
RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.

RealSR-NCNN-Android Real-ESRGAN is a Practical Algorithms for General Image Restoration. RealSR-NCNN-Android is a simple Android application that base

null 151 Aug 8, 2022
libcu++: The C++ Standard Library for Your Entire System

libcu++, the NVIDIA C++ Standard Library, is the C++ Standard Library for your entire system. It provides a heterogeneous implementation of the C++ Standard Library that can be used in and between CPU and GPU code.

NVIDIA Corporation 2k Aug 8, 2022
This repository is to share the EdgeAI Lab with Microcontrollers Series material to the entire community

This repository is to share the EdgeAI Lab with Microcontrollers Series material to the entire community. We will share documents, presentations and source code of two demo applications.

Machine Learning Tokyo 15 Oct 23, 2021
TCP port scanner, spews SYN packets asynchronously, scanning entire Internet in under 5 minutes.

TCP port scanner, spews SYN packets asynchronously, scanning entire Internet in under 5 minutes.

Robert David Graham 19.4k Aug 5, 2022
Text - A spicy text library for C++ that has the explicit goal of enabling the entire ecosystem to share in proper forward progress towards a bright Unicode future.

ztd.text Because if text works well in two of the most popular systems programming languages, the entire world over can start to benefit properly. Thi

Shepherd's Oasis 204 Jul 25, 2022
TCP port scanner, spews SYN packets asynchronously, scanning entire Internet in under 5 minutes.

MASSCAN-NG: Mass IP port scanner This is an Internet-scale port scanner. It can scan the entire Internet in under 5 minutes, transmitting 10 million p

BI.ZONE 47 Jul 15, 2022
DirectX12 Render Pipeline

DirectX12 Render Pipeline WindowSDK Version : 10.0.18362.0 Visual Studion Version : 2017 Rendering pipeline demo based on directx12. font. Text render

null 16 Jun 6, 2022
A software pipeline to decode the Falcon 9 telemetry from the 6MS/s baseband file.

falcon9_pipeline A software pipeline to decode the Falcon 9 telemetry from the 6MS/s baseband file. This is a work in progress, and you need to source

Mike Field 12 May 13, 2021
Gstreamer plugin that allows use of NVIDIA Maxine SDK in a generic pipeline.

GST-NVMAXINE Gstreamer plugin that allows use of NVIDIA MaxineTM sdk in a generic pipeline. This plugin is intended for use with NVIDIA hardware. Visi

Alex Pitrolo 14 May 11, 2022
LineFS: Efficient SmartNIC Offload of a Distributed File System with Pipeline Parallelism

LineFS repository for the Artifact Evaluation of SOSP 2021 1. System requirements (Tested environment) 1.1. Hardware requirements 1.1.1. Host machine

null 48 Jul 8, 2022
A Pipeline for LC-MS/MS Metabolomics Data Process and Analysis

NP³ MS Workflow A Pipeline for LC-MS/MS Metabolomics Data Process and Analysis Overview The NP³ MS workflow is a software system with a collection of

null 3 Feb 15, 2022
Minimalistic pipeline buffer

Synopsis pipebuf is as a low‐overhead buffer for command pipelines. Usage pipebuf reads from standard input and writes to standard output, buffering t

Mikael 1 Feb 26, 2022
Pipet - c++ library for building lightweight processing pipeline at compile-time for string obfuscation, aes ciphering or whatever you want

Pipet Pipet is a lightweight c++17 headers-only library than can be used to build simple processing pipelines at compile time. Features Compile-time p

C. G. 59 Jul 30, 2022
A 3D DNN-based Metric Semantic Dense Mapping pipeline and a Visual Inertial SLAM system

MSDM-SLAM This repository represnets a 3D DNN-based Metric Semantic Dense Mapping pipeline and a Visual Inertial SLAM system that can be run on a grou

ITMO Biomechatronics and Energy Efficient Robotics Laboratory 11 Jul 23, 2022
This Project Implement an interactive camera for 3D model using Quaternion. It have some advantages over eulerian camera like no gimbal lock and faster to compute.

Quaternion-Camera This Project Implement an interactive camera for 3D model using Quaternion. It have some advantages over eulerian camera like no gim

Phan Sang 6 Jul 30, 2022
3D-Lidar Camera Calibration using planar Point to to camera Plane Constraint

3D-Lidar Camera Calibration using planar Point to to camera Plane Constraint

Subodh Mishra 61 Jul 8, 2022
esp32 cam digital low latency fpv

esp32-cam-fpv esp32 cam digital, low latency FPV This project uses a modified esp-camera component running on an AI Thinker board to send low-latency

null 102 Jul 27, 2022
Builds and runs an exported image classification impulse on ESP32 Cam

ESP32 Cam and Edge Impulse How to run custom inference on a ESP32 cam using Edge Impulse. Material This code has been tested the AI Thinker ESP32 Cam

Edge Impulse 51 Jul 16, 2022
My humble attempt at getting tensorflow and the ESP32-CAM to cooperate

ESP32-Object-Recognition My humble attempt at getting tensorflow and the ESP32-CAM to cooperate (among other things) Notes: "Webcam_detection.py" isn'

null 13 Jun 21, 2022