The official implementation of the research paper "DAG Amendment for Inverse Control of Parametric Shapes"

Overview

DAG Amendment for Inverse Control of Parametric Shapes

Teaser

This repository is the official Blender implementation of the paper "DAG Amendment for Inverse Control of Parametric Shapes" by Élie Michel and Tamy Boubekeur, published in Transactions on Graphics (Siggraph 2021).

@article{MB:2021:DAGA, 
  title = "DAG Amendment for Inverse Control of Parametric Shapes", 
  author = "Elie Michel and Tamy Boubekeur", 
  year = "2021", 
  journal = "ACM Transactions on Graphics",
  number = "4",
  volume = "40",
  pages  = "173:1--173:14",
}

Video

Usage

A. Download the latest release zip file and install it like any other Blender add-on (in Edit > Preferences > Add-ons > Install)

B. You should see two new panels in scene properties. The "Parametric Shape - Rig" panel (1) is where one defines the list of public hyper-parameters of the scene and the "Parametric Shape - Anim" panel (2) that exposes all hyper-parameters once they are defined.

Parametric Shape panels in scene properties

C. We first use the Rig panel. Let's add a few hyper-parameters using the "+" button. Select in the list (1) an hyper-parameter to access to its settings bellow. An hyper-parameter drives a given component of an object's property, so for each of them one must define the object (2), the property (3) -- that is typically location, rotation or scale but can be any other property accessible from the Python API -- and the component (4), either 0, 1, 2 or 3 for X, Y, Z or W respectively.

Definition of a few hyper-parameters

These essential settings can be set directly from the hyper-parameter list for quicker access, but there are also extra settings, namely the boundaries, the default value, and the display name (automatically filled in using the object's name and target property/index by default).

D. Make sure your scene uses only supported modifiers, then click on Dag Amendment. This will insert modifiers in the scene, with name starting by _AMENDMENT_.

Dag Amendment button

E. In Object mode (1), enable the "SmartGrab" tool (2). This is the brush that enables one to manipulate the scene in a way that will only affect hyper-parameters. Setting are displayed in (3) upon activation of the tool.

SmartGrab tool

F. (optional) Tune the settings. The brush radius can be modified using Ctrl + Mouse Left, or the F key. Other settings are accessible in the upper bar. The two drop down menus are used to chose the solver (1) and the jacobian filtering algorithm (2), and depending on your choice they might have dedicated settings in (1b) et (2b) respectively. Then come settings related to the sampling and finite differences (3) and to display options (4). Default values are the one used in the paper.

SmartGrab settings overview

Advanced Usage

Drivers Hyper-parameter targets are simple object properties, like an object's position, but if you want to drive properties from other entities, like a parameter of a modifier for instance, you can use Blender's drivers to link the driven object (typically an empty) to the actual target.

Setup From Collection If you organized your scene such that a collection groups all the objects driven by hyper-parameters and that you locked all transform channels that must not be changed, then instead of manually adding the hyper-parameters one by one you may simply use the Setup From Collection button:

Setup from collection scene tree

Setup from collection button

View Layer The SmartGrab brush can use a different view layer than the one you are visualizing. Points at which jacobians are measured are sampled using the view layer stated at the beginning of the Rig panel. Typically, use this if you want to see in wireframe the operands of some boolean operation but want the brush to ignore these.

Supported modifiers

As a general rule of thumb, we support the modifiers that are able to transmit UV coordinates from input to output without introducing overlap, or if they introduce overlap with the possibility to disambiguate it like for instance the array modifier enables one to offset UVs for each instance.

A note about Geometry Nodes Geometry Nodes would be a very natural target for our method, but they did not exist in the main branch of Blender until a few months ago, so our prototype was developed without them. As a consequence, they are not supported yet by this add-on; we stuck to modifiers.

Building

NB If you just want to use the add-on, you do not need to build it yourself, you may simply download the latest release.

Steps 1 and 2 can be performed by simply running:

# on linux: apt install build-essential cmake python3.9-dev
python make_releases.py
  1. Build Accel
cd Accel
mkdir build
cd build
cmake .. -DPYTHON_EXECUTABLE="C:\Python39\python.exe"
cmake --build . --config Release
cmake --install .

The install command simply copies the pyd file that results from building (e.g. Accel.cp39-win_amd64.pyd) to DagAmendment.

NB: For Blender <= 2.92, the python version must be 3.7 (the filename must contain cp37). From 2.93 on, it must be 3.9 (cp39). If it is not the case, you must either set your PATH so that where python or which python points to the relevant version, or use -DPYTHON_EXECUTABLE as in the example above. This requires to have the relevant version of Python installed independently of Blender's embedded version (because the latter does not include libraries to link against).

  1. Create zip from DagAmendment/ and copy it to the releases directory (gitignore'd). This zip file is an add-on that can be installed in Blender.

  2. In Blender, go to Edit > Preferences, Add-ons tab, "Install...", browse to the release directory created by the script above and install DiffParam and DepsgraphNodes.

Walkthrough

Main entry points

The four main sections of the code that relates to the paper are:

  • DAG Amendment: Located in dag_amendment_operators.py, it inserts new nodes (in practice new modifiers) in the scene based on the analysis of the dependency graph generated using the DepsgraphNodes/ submodule.

  • Jacobian Filtering: Located in JFilters/NegativeJFilter.py for the one described in the paper, but other alternative jacobian filtering algorithm are available (at least the naive AverageJFilter.py, for comparison). The JFilter takes the list of sample points and their gradients (through an instance of the SamplePoints class) and builds a single jacobian, for the solver to use.

  • Solver: Located in Solvers/StandardSolver.py, with an option to use active sets or not, the solver uses the jacobian returned by the JFilter and the user stroke to decide on an update of the hyper-parameters.

  • The main interaction loop: Located in smartgrab_operators.py, this operator is called when the "SmartGrab tool" is enabled to handle key/mouse events and orchestrate the calls to the jacobian filtering and solver.

File layout

The root of the add-on is the DagAmendment directory, which contains only Python code. A small module is accelerated using C++ and put aside in the Accel directory. Once built, this module is a .pyd file that must be copied inside of DagAmendment.

As most Blender add-ons do, DagAmendment defines:

  • properties: extra fields attached to scenes and objects, and save into .blend files. These are defined by all files ending with ...properties.py. Hyper-parameters for instance are properties we attach to a scene.
  • operators: actions that can be called from the UI or from other scripts, that can change properties and call other operators. They are defined in all files ending with ...operators.py.
  • panels: the extra sections that appear in the scene properties (see usage instructions above)
  • overlays: callbacks drawing on top of the 3D viewport:
  • tools: a mapping from key/mouse event bindings to operators that is used only when the tool is activated (in the left-hand side of the 3D viewport). For instance, the "SmartGrab" tool that we define tells that when it is activated, a mouse click event must trigger the operator defined in smartgrab_operators.py.
  • preferences: global settings exposed in the add-ons section of Edit > User preferences.
  • handlers: callbacks executed by Blender at events such as file loaded, scene changed etc. Used here mostly to ensure that hyper-parameter boundaries are respected.

Then, files starting with a capital letter only define a class whose name is the same as the file. Most of them are abstraction meant to make our core implementation less tied to Blender. ParametricShape wraps around a scene to only expose methods we need for the interaction loop and Brush, Projector, Ray, ViewportState and Stroke are used by solvers instead of calling Blender specific equivalent. SamplePoints holds the list of points that are sampled within the brush, and measures the jacobians at each of these points.

uv_coparam holds the function used to convert co-parameters, which identify points independently of the value of the hyper-parameters, into Tuv parameters which quickly identify a point using a triangle index and barycentric coordinates but is valid for the current value of the hyper-parameters only. This is the part that uses Accel to quickly iterate the geometry.

blender_imgui is used only if you manually install the imgui module, to draw colored sliders in the viewport.

Other files are either self explanatory or of minor importance. Files ending with ...utils.py are unorganized lists of utility functions.

The directory DepsgraphNodes/ is an embedded add-on that introduces a new node panel where the relations from the scene (parenting, drivers, pointers to other objects in modifiers, etc.) are represented as nodes. The node tree is built by analyzing the scene. This is used because Blender's API does not natively provides a consistent graph-like interface to access this, while our DAG Amendment needs so for a clear implementation.

The directories JFilters/ and Solvers/ contain class files defining variants of jacobian filter and solvers respectively. When using the SmartGrab tool, the user can chose in drop down menus which of these to use. To create new alternatives, simply copy one of the existing files and change its content, then the files jfilter_registry.py and solver_registry.py will automatically find them and show them in the UI.

Troubleshooting

If the estimation of jacobians seems messy:

  • Check that you are using only supported modifiers
  • Try adding a triangulate modifier on the object that causes trouble
  • Tune the finite difference step

Before reporting an issue Please have a look at the system console (Window > Toggle System Console) for error messages. When reporting an issue, you must attach the full content of this console.

License

Being a Blender add-on, this code is distributed as a whole under the terms of the GPLv3 license. In details, files that use the bpy module (Blender Python) must be GPL, other files use the more permissive MIT license.

Copyright (c) 2020-2021 - Télécom Paris (Élie Michel)

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.
You might also like...
the code of paper
the code of paper "Optimizing Global Injectivity for Constrained Parameterization" (SIGGRAPH Asia 2021)

Optimizing Global Injectivity for Constrained Parameterization Xingyi Du, Danny M. Kaufman, Qingnan Zhou, Shahar Kovalsky, Yajie Yan, Noam Aigerman, T

NLP-based perching trajectory generation presented in our paper
NLP-based perching trajectory generation presented in our paper "Perception-Aware Perching on Powerlines with Multirotors".

Perception-Aware Perching on Powerlines with Multirotors This repo contains the code for the NLP-based perching trajectory generation presented in our

Code for Paper A Systematic Framework to Identify Violations of Scenario-dependent Driving Rules in Autonomous Vehicle Software

Code for Paper A Systematic Framework to Identify Violations of Scenario-dependent Driving Rules in Autonomous Vehicle Software

1day research effort

CVE-2021-3156-Baron-Samedit This repo contains my work on clumsily implementing a public 1day exploit for the sudo bug. Wish me luck. If you would lik

WeNet is to close the gap between research and production end-to-end (E2E) speech recognition models,
WeNet is to close the gap between research and production end-to-end (E2E) speech recognition models,

WeNet is to close the gap between research and production end-to-end (E2E) speech recognition models, to reduce the effort of productionizing E2E models, and to explore better E2E models for production.

DeepRTS is a high-performance Real-TIme strategy game for Reinforcement Learning research written in C++
DeepRTS is a high-performance Real-TIme strategy game for Reinforcement Learning research written in C++

DeepRTS is a high-performance Real-TIme strategy game for Reinforcement Learning research. It is written in C++ for performance, but provides an python interface to better interface with machine-learning toolkits. Deep RTS can process the game with over 6 000 000 steps per second and 2 000 000 steps when rendering graphics. In comparison to other solutions, such as StarCraft, this is over 15 000% faster simulation time running on Intel i7-8700k with Nvidia RTX 2080 TI.

Research on Event Accumulator Settings for Event-Based SLAM
Research on Event Accumulator Settings for Event-Based SLAM

Research on Event Accumulator Settings for Event-Based SLAM This is the source code for paper "Research on Event Accumulator Settings for Event-Based

kaldi-asr/kaldi is the official location of the Kaldi project.

Kaldi Speech Recognition Toolkit To build the toolkit: see ./INSTALL. These instructions are valid for UNIX systems including various flavors of Linux

Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection (CVPR 2021)
Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection (CVPR 2021)

GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Mo

Comments
  • Can't enable in Blender because of error in shader

    Can't enable in Blender because of error in shader

    ERROR (gpu.shader): pyGPUShader VertShader: | 32 | vec4 color = texelFetch(jbuffermaps[param], coord, 0); | ^ | Error: sampler arrays indexed with non-constant expressions are forbidden in GLSL 1.30 and later

    Traceback (most recent call last): File "/usr/share/blender/2.93/scripts/modules/addon_utils.py", line 351, in enable mod = import(module_name) File "/bunny/.config/blender/2.93/scripts/addons/DagAmendment/init.py", line 84, in from . import overlays File "/bunny/.config/blender/2.93/scripts/addons/DagAmendment/overlays.py", line 29, in from .shaders import line_shader, line_shader_2d, point_shader File "/bunny/.config/blender/2.93/scripts/addons/DagAmendment/shaders.py", line 252, in jbuffer_reduce_shader = gpu.types.GPUShader(''' Exception: Shader Compile Error, see console for more details

    opened by Unesty 9
  • Blender 3.3 does not load dag amendment

    Blender 3.3 does not load dag amendment

    Issue description

    Blender 3.3 does not load dag amendment

    Steps to reproduce

    1. Download dag admendment
    2. Install into blender 3.3 addons
    3. Can't dag amend

    Minimal reproduction project

    No response

    Using a released version?

    Yes. https://github.com/eliemichel/DagAmendment/releases/tag/v1.1.0

    What OS? Any other relevant information?

    Windows 11, Nvidia

    opened by fire 5
  • Trouble using in Blender 3.1 / Blender 3.2

    Trouble using in Blender 3.1 / Blender 3.2

    Version: https://github.com/eliemichel/DagAmendment/releases/tag/v1.0.1

    System: Windows 11, RTX 3000 series Nvidia

    I can't seem to compile DagAmendment for Blender 3.1/3.2.

    Reproduction steps:

    1. git clone https://github.com/eliemichel/DagAmendment
    2. cd Accel
    3. cmake .. -DPYTHON_EXECUTABLE="C:\Users\USER\scoop\apps\python\current\python.EXE" -GNinja
    4. ninja
    5. ninja install

    Notes:

    Seems to be using python 3.10. Your previous releases use python 3.9.

    opened by fire 4
Releases(v1.1.0)
  • v1.1.0(May 19, 2022)

    This is the official Blender implementation of the paper "DAG Amendment for Inverse Control of Parametric Shapes" by Élie Michel and Tamy Boubekeur, published in ACM Transactions on Graphics (Proc. Siggraph 2021).

    @article{MB:2021:DAGA, 
      title = "DAG Amendment for Inverse Control of Parametric Shapes", 
      author = "Elie Michel and Tamy Boubekeur", 
      year = "2021", 
      journal = "ACM Transactions on Graphics",
      number = "4",
      volume = "40",
      pages  = "173:1--173:14",
    }
    

    This is a Blender add-on, do not unzip it, rather install it as is from Blender's user preferences. For more information see the README.

    Once installed, check out the examples files.

    NB This release only contains a build for Windows and Linux, but running make_release.py should also work on OSX (not tested).

    Changelog

    • Port to Blender 3.1

    Contributors

    @eliemichel @superboubek

    Source code(tar.gz)
    Source code(zip)
    DagAmendment-v1.1.1.zip(295.98 KB)
  • v1.0.1(Oct 7, 2021)

    This is the first release of the official Blender implementation of the paper "DAG Amendment for Inverse Control of Parametric Shapes" by Élie Michel and Tamy Boubekeur, published in Transactions on Graphics (Siggraph 2021).

    @article{MB:2021:DAGA, 
      title = "DAG Amendment for Inverse Control of Parametric Shapes", 
      author = "Elie Michel and Tamy Boubekeur", 
      year = "2021", 
      journal = "ACM Transactions on Graphics",
      number = "4",
      volume = "40",
      pages  = "173:1--173:14",
    }
    

    If you are attending Siggraph, our live Q&A session is at 11am PDT (20h CET) on August 13, we'll be happy to see you there!

    This is a Blender add-on, do not unzip it, rather install it as is from Blender's user preferences. For more information see the README.

    Once installed, check out the examples files.

    NB This release only contains a build for Windows and Linux, but running make_release.py should also work on OSX (not tested).

    Changelog

    • Remove unused buggy shaders
    • Start working on a more intuitive JFilter
    • An an example scene for 2 min papers
    • Handle better meshes with no faces

    Contributors

    @eliemichel @superboubek

    Thanks @mudkip-hacker for bugfixing

    Source code(tar.gz)
    Source code(zip)
    DagAmendment-v1.0.1.zip(274.40 KB)
  • v1.0.0(Jul 30, 2021)

    This is the first release of the official Blender implementation of the paper "DAG Amendment for Inverse Control of Parametric Shapes" by Élie Michel and Tamy Boubekeur, published in Transactions on Graphics (Siggraph 2021).

    @article{MB:2021:DAGA, 
      title = "DAG Amendment for Inverse Control of Parametric Shapes", 
      author = "Elie Michel and Tamy Boubekeur", 
      year = "2021", 
      journal = "ACM Transactions on Graphics",
      number = "4",
      volume = "40",
      pages  = "173:1--173:14",
    }
    

    If you are attending Siggraph, our live Q&A session is at 11am PDT (20h CET) on August 13, we'll be happy to see you there!

    This is a Blender add-on, do not unzip it, rather install it as is from Blender's user preferences. For more information see the README.

    Once installed, check out the examples files.

    NB This release only contains a build for Windows and Linux, but running make_release.py should also work on OSX (not tested).

    Contributors

    @eliemichel @superboubek

    Source code(tar.gz)
    Source code(zip)
    DagAmendment-v1.0.0.zip(268.49 KB)
Owner
Elie Michel
Research in Computer Graphics.
Elie Michel
Implementation for the "Surface Reconstruction from 3D Line Segments" paper.

Surface Reconstruction from 3D Line Segments Surface reconstruction from 3d line segments. [Paper] [Supplementary Material] Langlois, P. A., Boulch, A

null 82 Nov 26, 2022
The repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching.

Geometric Structure Preserving Warp for Natural Image Stitching This repository contains our dataset and C++ implementation of the CVPR 2022 paper, Ge

null 19 Nov 26, 2022
Official Pytorch implementation of RePOSE (ICCV2021)

RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering (ICCV2021) [Link] Abstract We present RePOSE, a fast iterative refinement method for

Shun Iwase 68 Nov 15, 2022
Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation"

SSM-VLN Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation". Environment Installation Download Room-to-Room

hanqing 34 Aug 24, 2022
This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicity.

Fast Face Classification (F²C) This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicit

null 33 Jun 27, 2021
This code accompanies the paper "Human-Level Performance in No-Press Diplomacy via Equilibrium Search".

Diplomacy SearchBot This code accompanies the paper "Human-Level Performance in No-Press Diplomacy via Equilibrium Search". A very brief orientation:

Facebook Research 33 Oct 28, 2022
Source Code for SIGGRAPH Asia 2021 Paper "Ensemble Denoising for Monte Carlo Renderings"

Ensemble Denoising Source Code for SIGGRAPH Asia 2021 Paper Ensemble Denoising for Monte Carlo Renderings. Paper | Code | Talk Slides Dependencies A C

郑少锟 52 Nov 25, 2022
Source Code for SIGGRAPH Asia 2021 Paper "Ensemble Denoising for Monte Carlo Renderings"

Ensemble Denoising Source Code for SIGGRAPH Asia 2021 Paper Ensemble Denoising for Monte Carlo Renderings. Paper | Code | Talk Slides Dependencies A C

郑少锟 52 Nov 25, 2022
Codes for the paper "A Correlation Attack on Full SNOW-V and SNOW-Vi"

Codes for the paper "A Correlation Attack on Full SNOW-V and SNOW-Vi" #Contents Automatic search of linear approximation trails of SNOW-V Evaluating t

null 2 Oct 19, 2022