Maya Python API 3.0

This PR enables generating typing stubs for cmdc automatically.

This is still WIP and I'm not really happy with the organization of everything yet, I'm open to any ideas there.

To generate the stubs:

• run mayapy scripts/generate_typing_stubs.py
• cmdc needs to be available in the PYTHONPATH
• pybind11-stubgen needs to be installed in mayapy

Some notes:

• I've had to declare all the classes before they're used anywhere to Avoiding having C++ types in the docstrings, this caused syntax errors and messed up the stubs generations. I've put that the forward declarations in ForwardDeclarations.inl. I don't know if there's a better way to do that.
• I haven't automated the generation when building yet because I only have a Windows machine and don't know PowerShell at all.
• The stubs are saved in the build directory, not sure if that's where they should be saved.
• Some of the docstrings have typing information, I'm not sure if they're still relevant but I've left them for now in case they were. pybind11-stubgen does take those into account and some weren't valid ((string, string, ...) instead of List[str] ) so I've updated those.

Added

• Add MDGModifier bindings

Changed

• Add support for OPENMAYA_PRIVATE namespace in parse_headers.py
  • This always appears at the end of the header, so it stops the iterator early.

Please squash the branch when it is merged.

I have added a cmake build system that will make setting up the dependencies and work across Windows, Mac, and Linux (untested). Mac was a difficult problem as Maya's devkit does not ship with the Python headers. So to not require the developer to have all the correct Python versions installed or have Maya installed, I copied the correct headers from the Maya installation including the shared libraries.

I have also added Pybind11 as a submodule so it will be easier to manage it rather than just a copy in the repository. It has to be version 2.9.0 as that is the last version to support Python 2.

I have successfully build Maya 2020 and 2022 on both Windows and Mac. I haven't tested it in linux yet. Some tweaks can still be made, but it works fine for now.

This is not ready to be merged yet. I'm making the PR for visibility, and because I had a few questions.

My questions:

1. Are we aiming to be more consistent with the C++ API or OpenMaya2? DagPath.extendToShape() returns a reference to self in OM2 but not in the C++ API. I feel like not returning anything would make more sense since self is already modified by the method
2. Do we have a standardized format for the error messages? I've just copied OM2's messages so far.
3. I was expecting exclusiveMatrix and exclusiveMatrixInverse to fail when called from an invalid DagPath but they return an identity matrix instead. This seems consistent with OM2 but I'm not sure why they're not raising a RuntimeError where something like childCount does with a similar implementation

Added

• Add full stub for MSelectionList
  • Implement __len__, .isEmpty()
  • Implement .getDependNode
  • Implement .getDagPath
  • Implement .getPlug
  • Implement .getSelectionStrings
• Add full stub for MPlug

Fixed

• Fix bugs in parse_header script
  • Support multi-line statements
  • Ignore trailing comments
  • Skip "No Script Support" blocks

wat

• MSelectionList::getPlug returns an OK status for a node. You have to check if the attribute is null.

Notes

• We should implement a C++ style guide.
• We should implement MString <-> std::string helper functions
• We should implement M[T]Array <-> std::vector helper functions

Follow up PR with #16 with the goal of improving the quality of the stubs

This mainly focuses on ensuring all the arguments are named properly instead of simply being named arg0, arg1, etc.

for that I'm simply using py::arg("arg_name") in order to let pybind know about the argument names

I'm also extracting the docstrings out of the function definition like you've started doing @mottosso to try and have some consistency. I've noticed two mild issues with this so far

1. On windows I get warnings like this cmdc\src\MFnDagNode.inl(21): warning C4005: '_doc_child': macro redefinition. I'm thinking of naming the variables like this instead to avoid redefinitions _doc_FnDagNode_child.
2. On multiline docstrings, the stubs end up with the \ I haven't found a way in C++ to not use those characters. Worst case scenario I can scrape them from the generated stubs before writing the file An example of the stubs with the \ left in

class DagPath():
    def extendToShapeDirectlyBelow(self, index: int) -> None: 
        """
        This method is used if the end of the path is a transform and there are shapes directly below the transform.\
        The shape to extend to is set by passing in an appropriate index parameter.\
        Use the `numberOfShapesDirectlyBelow()` method to determine how many shapes are below
        """

Other than that, the stubs are looking pretty good, I just need to do that on the remaining files before merging

So, when trying to record the gif for the readme, I realized the completion wasn't working on some classes like SelectionList. Turns out pybind11-stubgen gets a bit confused when a class has an inner class defined and tries to import the parent class as a module. This is tracked by this issue: https://github.com/sizmailov/pybind11-stubgen/issues/36

I've opted to just remove the unwanted imports once the stubs are generated, might be a bit brute force but that felt like the only option that didn't cause other issues somewhere else in the stubs.

I'll also update the readme and contributing.md as part of this PR. @mottosso is there a good place where I can store the gif?

I slightly improved the logging of generate_stubs.py to include the line of the invalid signatures as well as added a --no-raise argument to write the stubs file even if there are invalid signatures.

I started using type annotations quite a lot these past months and I've grown to love them.

Since none of the Maya APIs are type annotated, every function takes and returns Any. This is really frustrating to work with as you need to explicitly specify the type of a lot of variables when that should be inferred.

I think it would be really nice if cmdc was fully type annotated and made that one of its primary goals. I haven't looked into how to set that up much so I'm not sure if pybind supports this or if we'd need to also write .pyi stubs for that.

Added

• Implement MPlug::as* methods that take POD or MObject
• Implement MPlug::set* methods that take POD or MObject

wat

• MPlug::asChar and MPlug::setChar are picky when used via Python - you must set using an integer, but get back a string.

Thank gods for parametrized tests :|

I have very little experience with C++ or writing bindings so I figured I would start small. If the PR suits you, I'll start working on other classes.

I did the following:

• Implemented MObject.__eq__.
• Changed MObject.__repr__ to include the apiTypeStr (wasn't 100% sure about the best way to format the string there).
• Added tests for all the MObject methods.

I was getting increasing build times as I was adding more classes. I read the pybind11 faq about breaking up the bindings into multiple translation units. So we now still get the same result but now we can use more processors to compile and speeds up rebuilding while developing. Also we can make slightly more organized folders as the number of classes increases.

Welcome to other ideas, but I get the same results as before but build times are much faster.

This PR implements the Bindings for MFnBase and makes MFnDependencyNode a subclass of it, as well as MFnDagNode a subclass of MFnDependencyNode

I also noticed that the MDGModifier and MDagModifier classes were defined exactly like the other classes in ForwardDeclarations.inl so I've updated that.

MFnBase has two methods that have been added in Maya 2020, I think I managed to get them to compile only for the right releases of cmdc but we'll see. I do feel like this means we should release a version of the stubs per maya version though? Otherwise when working with 2019 we'll get completion on methods that don't exist.

I'm expecting this to fail at the moment because MFnBase::className() returns MFnBase and not FnBase like I think we would expect. I'm not actually sure how to dynamically get the name of the class in C++? I'm also not really sure that this even matters as we can just call MFnBase.__name__ in python.

So, I went down a rabbit hole...

parse_header.py was failing on some header files required for #25. I tried upgrading it but didn't feel like handling a billion edge cases like manually parsing C++ seems to be like

I ended up relying on Universal Ctags which outputs a list of all the classes, methods, etc. that are in a given file.

It's not perfect but it works on all the files I've tested so far

Some things that are not supported yet:

1. Functions declared out of the scope of a class
2. Properties
3. Nested classes
4. Enums

I'll probably leave it at that for this PR though and we can improve on that as we need.

I've made a separate parse_header_ctags.py script in case parse_header.py still produces better results on some files

And of course, ctags needs to be installed and available in your path

Time to get real. This is one of the examples out of the Maya devkit, from API 1.0.

• animCubeNode.py

There are quite a few things missing for this to work.

• MFnPlugin
• MPxNode
• MDataBlock
• MDataHandle
• MFnAttribute (including Unit and Typed)
• MFnMesh
• MFnMeshData
• MTime
• MFloatPoint
• Int and float arrays

So it's gonna be a big one. xD

Hello, welcome to cmdc where we attempt to reproduce Maya Python API 2.0 (i.e. maya.api.OpenMaya) in open source form, fix all bugs and add all of the things missing from it. Autodesk has shown little to no interest in continuing with API 2.0, and I was surprised to find nobody had yet taken matters into their own hands.

So let's do that!

I've put together a few source files demonstrating the fundamentals of how to bind the API, along with commonly used types like MVector and MMatrix and a function set, MFnDependencyNode for manipulating the scene graph.

Here's the simplest example.

• https://github.com/mottosso/cmdc/blob/master/src/Types.inl

The syntax is a little cryptic, because it's doing a lot of automatic detection of argument types and return values. Things we'd otherwise have to do ourselves if manually binding via e.g. Python.h. But it's fairly consistent and only consists of a few key functions, most if not all of them are already present in this repository.

• Syntax reference

Help Wanted

But I need help. There is too much of an API to wrap for one person, and the task of actually binding a new C++ member is of intern-level complexity. That is, anyone can do it. The hard part is patience and diligence. We need to match function signatures, convert all MStatuses to exceptions and write tests for each newly added member.

As soon as I spot some interest in the project, and interest in helping out, I'll expand upon the test suite and CI mechanism to automatically produce compiled versions for all platforms and versions of Maya, along with automatic upload to PyPI. From there, anyone can add or improve upon any member and we'll never have to struggle with maya.api.OpenMaya again.

How can I help?

Glad you asked!

1. Get the project built on your machine
2. Pick your favourite member from the C++ documentation
3. Make a new MYourMember.inl file
4. Add it to main.cpp

Remember to keep the name of arguments as documented, so users of cmdc can refer to this documentation for help on how to use cmdc. And ditch the M prefix that all classes have, Python don't need that.

MVector -> Vector

Any questions or concerns, post here or in a new issue and let's fix this thing once and for all!