3D ray-tracing and animation engine for pixel matrices.

Overview

ProtoTracer: Protogen Ray-Tracing and Animation Engine

This project is a 3D ray-tracing and animation engine for pixel matrices, designed to be used for drawing live animations on Protogen style characters from 3D object files (.OBJ).

Demonstration:

As a quick way to showcase the capabilities of this software, here is a recording of a demo uploaded to YouTube: Protogen Ray-Tracing Engine

Recommended platform requirements:

  • 32-bit FPU
  • 200MHz+ processor
  • At least 80KB dynamic memory

This codebase has been tested on an Arduino Mega (smaller .OBJ file with a 89 pixel matrix) and on a Teensy 4.0 (600 triangle scene with a 2,804 pixel matrix).

Usage:

To use the Protogen Ray-Tracing Engine you must first follow a few requirements:

Have your pixel list ready:

A pixel list is a list of the XY coordinates (in millimeters, used for all dimensions) of the pixels on a camera. The easiest way to generate a pixel list is to export a Pick and Place file from a PCB design tool. The formatting utilizes a CSV file, each line is a pixel containing the object name, X coordinate, and Y coordinate. Here is an example:

U1,18,4
U2,25,4
U3,32,4

This list will be used to create your camera object for rendering the scene, with your list ready, you can pass the string to be parsed into the camera object to intialize it:

//Utilizing relative XYZ Euler Angles for camera orientation:
Camera camFrontTop = Camera(Vector3D(-45, 0, 180), Vector3D(90, -220, -500), 306, &pixelString, true, false);

//Utilizing a quaternion object as an input:
Camera camFrontTop = Camera(Rotation(EulerAngles(Vector3D(-45, 0, 180), EulerConstants::EulerOrderXYZR)).GetQuaternion(), Vector3D(90, -220, -500), 306, &pixelString, true, false);

Okay, so this may look busy so let's break it down. You must first set the orientation of your camera object which will be drawing the pixel data, this can be done as an Vector3D object representing the Euler Angles in an XYZ inertial frame of reference, as a quaternion object, rotation matrix, axis angle, or direction angle. The position also needs to be set which is the second parameter given as a Vector3D object in millimeters. The string reference to the Pixel list is then passed into the object as well as if you would like to invert the X-axis values or Y-axis values.

Have your 3D object (.OBJ) files ready:

Object files can be manually created but must be a triangulated mesh. Triangles that share vertices should be properly linked within the object file to save on memory space as the memory is shared between triangles. Here is an example of a single right-triangle:

v -50.000000 -50.000000 0.000000
v -50.000000 50.000000 0.000000
v 50.000000 -50.000000 0.000000
f 3 2 1

This .OBJ file will be used to create an Object3D instance.

Object3D* objects[2];

Object3D dvdObj = Object3D(dvd, 100, 100);
Object3D dvdObj2 = Object3D(dvd, 100, 100);

objects[0] = &dvdObj;
objects[1] = &dvdObj2;

The Object3D file is initialized very similar to the camera object, you pass the reference to the String object containing the list of vertices and triangles as well as the maximum number of triangles and maximum number of vertices in the object. Once converted this object can be scaled, moved, rotated, enabled/disabled, and reset.

//Example of DVD 3d object scaling, moving, and rotating
//Objects visibility can be enabled and disabled at any point before rasterizing to change its visibility
objects[0]->Enable();

//Resets the object back to the original state before any translates/modifications, must be ran once per loop in most cases
objects[0]->ResetVertices();
    
//Objects can be scaled by origin, center, and at a point
objects[0]->Scale(Vector3D(1.3f + sin(i * 3.14159f / 180.0f * 3.0f) * 0.3f, 1.3f + sin(i * 3.14159f / 180.0f * 3.0f) * 0.3f, 1.0f), Vector3D(0, 0, 0));
    
//Objects can be moved to a coordinate or translated by a vector
objects[0]->Move(Vector3D(-100 + sin(i * 3.14159f / 180.0f * 3.0f) * 100.0f, 60 + cos(i * 3.14159f / 180.0f * 1.5f) * 100.0f, 0.0f));
    
//Objects can be rotated with by any rotation object (quaternion is preferred) and about any coordinate or center
objects[0]->Rotate(Vector3D(7 + sinf(i * 3.14159f / 180.0f * 4.0f) * 1.0f, sinf(i * 3.14159f / 180.0f * 2.0f) * 1.0f, sinf(i * 3.14159f / 180.0f * 2.0f) * 1.0f), Vector3D(0, 100, 0));

Create your light objects:

To create a light object you must specify the lights position, intensity, falloff distance, and falloff curvature:

Light lights[6];

//Set lights position, color intensity, falloff distance, and falloff curvature
lights[0].Set(Vector3D(1000, 0, 0), Vector3D(255, 0, 0), 1000.0f, 0.5f, 0.5f);

//Lights can be moved to any vector coordinate
lights[0].MoveTo(Vector3D(sinf(i * 3.14159f / 180.0f * 2.0f) * 1000.0f, 0, -cosf(i * 3.14159f / 180.0f * 2.0f) * 1000.0f));

Lights can be manipulated similar to an object at any point after creation.

Creating and linking your scene:

To create a scene, you need to pass the reference to your objects and lights as well as the amount of objects and lights into the constructor:

  Scene* scene = new Scene(objects, lights, 1, 6);

Rendering the scene:

Now that you have everything to render the objects, you must call the rasterize function within the Camera object. This function warps the triangles in the scene to the cameras perspective allowing the triangles to be forced to a 2D perspective to optimize the intersection verification of each triangle, then calculates the light at each point of the triangle based on the original 3D triangles normal. To rasterize, you must set your pass your scene, set a scaling factor, and set the maximum brightness output at each pixel:

camFrontTop.Rasterize(scene, 1.0f, 20);

Writing to your LED matrix:

You must implement the firmware for writing to your specific LED matrices. The included example utilizes the OctoWS2811 by Paul Stoffregen for writing to 4 WS2812B LED panels using a Teensy 4.0 or 4.1 microcontroller. To use your own LED code you will need to remove the lines containing the Teensy specific LED code and replace it with your own.

In this example codebase utilizing the OctoWS2811 library, all that must be done to set the LED outputs is as follows:

//Set the library specific parameters
const int ledsPerStrip = 306;//Set LEDs per board
DMAMEM int displayMemory[ledsPerStrip * 6];//Set up the display memory
int drawingMemory[ledsPerStrip * 6];//Set up the drawing memory
const int config = WS2811_GRB | WS2811_800kHz;//Set the config for color and LED type
OctoWS2811 leds(ledsPerStrip, displayMemory, drawingMemory, config);//Initialize the library constructor

//Read each pixel from the ray-tracing engine and write it to the output LED DMA memory
for (int i = 0; i < 306; i++) {
  leds.setPixel(i,       (byte)camFrontTop.GetPixels()[i].RGB.X, (byte)camFrontTop.GetPixels()[i].RGB.Y, (byte)camFrontTop.GetPixels()[i].RGB.Z);
  leds.setPixel(i + 306,   (byte)camRearTop.GetPixels()[i].RGB.X, (byte)camRearTop.GetPixels()[i].RGB.Y, (byte)camRearTop.GetPixels()[i].RGB.Z);
  leds.setPixel(i + 306 * 2, (byte)camFrontBottom.GetPixels()[i].RGB.X, (byte)camFrontBottom.GetPixels()[i].RGB.Y, (byte)camFrontBottom.GetPixels()[i].RGB.Z);
  leds.setPixel(i + 306 * 3, (byte)camRearBottom.GetPixels()[i].RGB.X, (byte)camRearBottom.GetPixels()[i].RGB.Y, (byte)camRearBottom.GetPixels()[i].RGB.Z);
}

//Update the LED outputs
leds.show();

This method is not as efficient as it could be implemented as it could be writing directly to the DMA memory, however, this method allows for easier cross compatibility between platforms.

Creating animations:

With your set up complete, now animations can be made. With all individual object, light, and camera manipulations described above, they simply need to be implemented within your main loop to change over time. This customization is entirely up to you, but as an example, here is an animation of a DVD logo bouncing around the screen in a figure 8 pattern:

for (int i = 0; i < 360; i++) {
    //Example of DVD 3D object scaling, moving, and rotating
    //Objects visibility can be enabled and disabled at any point before rasterizing to change its visibility
    objects[0]->Enable();

    //Resets the object back to the original state before any translates/modifications, must be ran once per loop in most cases
    objects[0]->ResetVertices();

    //Objects can be scaled by origin, center, and at a point
    objects[0]->Scale(Vector3D(1.3f + sin(i * 3.14159f / 180.0f * 3.0f) * 0.3f, 1.3f + sin(i * 3.14159f / 180.0f * 3.0f) * 0.3f, 1.0f), Vector3D(0, 0, 0));
    
    //Objects can be moved to a coordinate or translated by a vector
    objects[0]->Move(Vector3D(-100 + sin(i * 3.14159f / 180.0f * 3.0f) * 100.0f, 60 + cos(i * 3.14159f / 180.0f * 1.5f) * 100.0f, 0.0f));
    
    //Objects can be rotated with by any rotation object (quaternion is preferred) and about any coordinate or center
    objects[0]->Rotate(Vector3D(7 + sinf(i * 3.14159f / 180.0f * 4.0f) * 1.0f, sinf(i * 3.14159f / 180.0f * 2.0f) * 1.0f, sinf(i * 3.14159f / 180.0f * 2.0f) * 1.0f), Vector3D(0, 100, 0));
  }

Here is a video example of the DVD logo rendering across multiple LED matrices: ProtoTracer: DVD Logo Test

If you have any questions on usage or implementation, please ask in the discussion or ask me on my Twitter at Coela Can't!.

Contributing:

Pull requests are very welcome. Any issues found should be reported first to open a discussion. Any recommendations can be added in the discussions tab and if you have any questions contact Coela Can't! on Twitter.

License Agreement:

For this project, AGPL-3.0 is used for licensing as a means to make sure any contributions or usage of the software can benefit the community. If you use and modify this software for a product, you must make the modified code readily available as per the license agreement.

Comments
  • Oprimizations

    Oprimizations

    Oh boy.

    That is the first thing that needs to be done with this monstrosity of changes I present.

    The simple changes

    The only parts that I changed in this was more of the backbone math stuff, it says that 1k files had been changed, but that is because I hit the "clear all" button. The other changes had to do with:

    • Changing the code style in places to match the rest of the project
    • Heavily optimizing, and not testing any of it (I have no way to test it)
    • Patched some logical bugs, and probably created a boat load more in the process.

    Known issues

    • I was lazy and never finished the memory "scaling"/changing of the filters, so it is just sort of there in a half working state.

    There are plenty more issues with it all since I somewhat rushed this, so just tell me what they are and I'll push some fixes if needed.

    It is ready enough... not really...

    opened by MrLava13 6
  • Quadtree Implementation

    Quadtree Implementation

    This PR provides a preview of a proposed screenspace quadtree implementation for accelerating ray-triangle intersection testing. The rendering loop is modified to include the following steps:

    • determine the screenspace bounds
    • initialize a quadtree to the screen bounds
    • insert each projected triangle into the tree
    • for each pixel, determine the intersecting leaf node and test against the contained triangles

    initial benchmarks show a significant performance increase for scenes with moderate to large triangle counts, some initial results can be found below.

    | Scene | Platform | Render Time (original) | Render Time (quadtree) | Speedup| |--------|-----------|-------------------------|----------------------------|----------| |SpyroAnimation|x86|3.82ms|0.45ms| 8.49x|

    Note that this implementation is not yet in a final stage, however the current state can already be used for testing and performance evaluation. For a final release the following TODOs will be addressed:

    • reduce memory usage by avoiding unnecessary duplicate references to triangles
    • adapt root node bounding box to intersection of screen bounds and triangle bounds
    • evaluate performance on more platforms and scenes
    opened by moepforfreedom 2
  • ESP32 Multicore

    ESP32 Multicore

    I think this is correct—let's see if it is working properly now lol.

    I added support for the ESP32, and also implemented a crude dual core setup for improved render times. I'm also using the experimental hardware SPI mode in FastLED, so that may break people's setups. I should probably put this into the readme.md.

    opened by MyloWhylo 0
  • V1.1 releases don’t display properly on the LEDs

    V1.1 releases don’t display properly on the LEDs

    To get the actual faces to show up on the leds, these releases need the face object(s) to be transformed so that anything will actually display on the matrixes. Transform meaning translate, rotate, and reflect. This hasn’t been a problem with other older releases ie 1.0.3.

    opened by MeteorTheProto 0
  • Invalid use of delete when destroying TriangleGroup

    Invalid use of delete when destroying TriangleGroup

    The current implementation of the predefined objects passes pointers to local member variables to the Object3D constructor, which in turn creates a copy of the TriangleGroup using modifiedTriangles = new TriangleGroup(originalTriangles);. This copy constructor copies the vertices and triangles, but stores the pointer to the IndexGroup. The TriangleGroup destructor then attempts to call delete[] indexGroup;. This is only safely possible when the IndexGroup was dynamically allocated and is not referenced anywhere else. In the case of the predefined Objects the IndexGroup is defined as a local member variable, therefore the use of delete[] is invalid and leads to undefined behavior. A similar issue exists for the vertices array of a TriangleGroup since the array pointer is copied in the constructor as well.

    Other similar issues:

    • PixelGroup() allocates the pixel array using new[] but ~PixelGroup() attempts to delete it using delete instead of delete[]

    The practical impact of these issues is likely small though since Objectd3D instances are typically only destroyed at program exit.

    opened by moepforfreedom 0
  • Inconsistent behavior of Quaternion rotation for 2D Vectors

    Inconsistent behavior of Quaternion rotation for 2D Vectors

    The current Quaternion implementation shows different behavior when applied to 2D vectors compared to the 3D counterpart, this breaks some of the normal properties of Quaternions. Specifically, rotating a vector by a Quaternion using Quaternion::RotateVector() and rotating the result back using Quaternion::UnrotateVector() should return the original vector. This is the case for 3D vectors but doesnt apply in the 2D case. The following code snippet demonstrates the difference:

    Quaternion testQuat = Rotation(EulerAngles({ 20.0f, 0, 0 }, EulerConstants::EulerOrderXZYS)).GetQuaternion();
    Vector3D testVec(1, 2, 0);
    Vector3D testRot = testQuat.RotateVector(testVec);
    Vector3D testRotInv = testQuat.UnrotateVector(testRot);
    printf("rotated: %.2f, %.2f, %.2f\n", testRot.X, testRot.Y, testRot.Z);
    printf("unrotated: %.2f, %.2f, %.2f\n", testRotInv.X, testRotInv.Y, testRotInv.Z);
    
    Vector2D testVec2D(1, 2);
    Vector2D testRot2D = testQuat.RotateVector(testVec2D);
    Vector2D testRotInv2D = testQuat.UnrotateVector(testRot2D);
    printf("rotated 2D: %.2f, %.2f\n", testRot2D.X, testRot2D.Y);
    printf("unrotated 2D: %.2f, %.2f\n", testRotInv2D.X, testRotInv2D.Y);
    

    This difference is caused by the implicit use of a 3D Vector with 0 as a Z coordinate in Quaternion::RotateVector(Vector2D v) since a normal 3D rotation can result in a nonzero Z coordinate, eg in the given test case rotating (1, 2, 0) results in (1.00, 1.88, 0.68), discarding the Z coordinate breaks the inverse transform.

    A possible workaround could for example be extracting the corresponding 2D transformation in the XY plane from the Quaternion (eg using the corresponding euler angles) and using that in the 2D rotation functions.

    opened by moepforfreedom 0
Releases(V1.1.0)
  • V1.1.0(Aug 16, 2022)

    Added menu for KB WS35 V1.1 kits

    • allows multiple color selection
    • size selection
    • enable/disable boop sensor
    • enable/disable microphone
    • brightness settings
    • mirroring the spectrum analyzer
    Source code(tar.gz)
    Source code(zip)
  • V1.1(Aug 16, 2022)

    Added menu for KB WS35 V1.1 kits

    • allows multiple color selection
    • size selection
    • enable/disable boop sensor
    • enable/disable microphone
    • brightness settings
    • mirroring the spectrum analyzer
    Source code(tar.gz)
    Source code(zip)
  • V1.0.3(Jun 11, 2022)

  • V1.0.2(Jun 11, 2022)

  • V1.0.1(Jun 10, 2022)

  • v1.0(Aug 20, 2021)

    This is the first release of the ProtoTracer. This project includes the following features:

    • OBJ rendering with UV maps
    • FBX rendering with UV maps and blend shapes
    • Depth, Light, Noise, UV, and Gradient shaders
    • Controller support for the Kaiborg V1 and V1.1 with KB WS35 V1 boards
    • Animations with keyframing
    • Object3D distortion, modification, transformations
    • Triangle view culling for camera rendering
    • Image importing for 2D picture display
    • GIF importing for 2D animations
    Source code(tar.gz)
    Source code(zip)
Owner
Open Protogen
An organization dedicated to creating open source Protogen designs for software and hardware!
Open Protogen
Improved version of the X-Ray Engine, the game engine used in the world-famous S.T.A.L.K.E.R. game series by GSC Game World.

OpenXRay OpenXRay is an improved version of the X-Ray Engine, the game engine used in the world-famous S.T.A.L.K.E.R. game series by GSC Game World. S

null 2.2k Jan 1, 2023
An OpenGL 4.3 / C++ 11 rendering engine oriented towards animation

aer-engine About An OpenGL 4.3 / C++ 11 rendering engine oriented towards animation. Features: Custom animation model format, SKMA, with a Blender exp

Thibault Coppex 29 Nov 22, 2022
Ray-casting game for wasting productive time.

Cub3D The goal of Cub3D is to make something using raycasting from scratch with low level C ✨ (like Wolfenstein 3D). Making this game gave me a basic

Ashad Mohamed 10 Sep 28, 2021
A decompilation of Guxt (2007 game by Studio Pixel)

Guxt Decompilation Project WIP. Mostly based on work of @tilderain. Reverse engineering of remaining parts and various fixups was done by me (Alula).

null 2 May 25, 2022
Cytopia is a free, open source retro pixel-art city building game with a big focus on mods.

Cytopia is a free, open source retro pixel-art city building game with a big focus on mods. It utilizes a custom isometric rendering engine based on SDL2.

CytopiaTeam 1.6k Dec 30, 2022
A procedural sprite animation tool made with the nCine

SpookyGhost A procedural sprite animation tool made with the nCine. You can read the manual online or you can access it by pressing F1 in the program.

SpookyGhost 219 Dec 11, 2022
Animation compression is a fundamental aspect of modern video game engines

Animation compression is a fundamental aspect of modern video game engines. Not only is it important to keep the memory footprint down but it is also critical to keep the animation clip sampling performance fast.

Nicholas Frechette 1k Jan 2, 2023
The Atomic Game Engine is a multi-platform 2D and 3D engine with a consistent API in C++, C#, JavaScript, and TypeScript

The Atomic Game Engine is a multi-platform 2D and 3D engine with a consistent API in C++, C#, JavaScript, and TypeScript

null 2.8k Dec 29, 2022
Godot Engine – Multi-platform 2D and 3D game engine

Godot Engine 2D and 3D cross-platform game engine Godot Engine is a feature-packed, cross-platform game engine to create 2D and 3D games from a unifie

Godot Engine 56.7k Jan 9, 2023
CLUSEK-RT is a complex game engine written in C++ and the successor of the CLUSEK game engine

CLUSEK-RT is a complex game engine written in C++ and the successor of the CLUSEK game engine. This engine has been designed with a cross-platform design in mind. Thanks to Vulkan API it delivers a next-gen experience with ray tracing to both Linux and Windows platforms

Jakub Biliński 48 Dec 29, 2022
Flax Engine – multi-platform 3D game engine

Flax Engine – multi-platform 3D game engine

Flax Engine 3.7k Jan 7, 2023
MAZE (My AmaZing Engine) - 🎮 Personal open-source cross-platform game engine

MAZE (My AmaZing Engine) is the self-written open-source cross-platform game engine in the active development stage. At the moment it is my main pet project, developed for the purpose of learning and preserving different game dev technologies.

Dmitriy Nosov 13 Dec 14, 2022
Ground Engine is an easy to use Game Engine for 3D Game Development written in C++

Ground Engine is an easy to use Game Engine Framework for 3D Game Development written in C++. It's currently under development and its creation will b

 PardCode 61 Dec 14, 2022
Rogy-Engine- - My 3D game engine source code.

Rogy-Engine Development My 3D game engine. (NOT THE FINAL VERSION- Windows only) Features: PBR shading and reflection probes with parallax correction.

AlaX 97 Dec 28, 2022
Hyperion Engine is a 3D game engine written in C++

Hyperion Engine About Hyperion Engine is a 3D game engine written in C++. We aim to make Hyperion be easy to understand and use, while still enabling

null 293 Jan 1, 2023
Powerful, mature open-source cross-platform game engine for Python and C++, developed by Disney and CMU

Panda3D Panda3D is a game engine, a framework for 3D rendering and game development for Python and C++ programs. Panda3D is open-source and free for a

Panda3D 3.6k Dec 31, 2022
OGRE is a scene-oriented, flexible 3D engine written in C++ designed to make it easier and more intuitive for developers to produce games and demos utilising 3D hardware.

OGRE (Object-Oriented Graphics Rendering Engine) is a scene-oriented, flexible 3D engine written in C++ designed to make it easier and more intuitive for developers to produce games and demos utilising 3D hardware. The class library abstracts all the details of using the underlying system libraries like Direct3D and OpenGL and provides an interface based on world objects and other intuitive classes.

null 3.1k Jan 3, 2023
Source and data to build Sonic 3 A.I.R. (Angel Island Revisited) and the Oxygen Engine

Sonic 3 A.I.R. Source code incl. dependencies for "Sonic 3 - Angel Island Revisited", a fan-made remaster of Sonic 3 & Knuckles. Project homepage: htt

Eukaryot 179 Dec 24, 2022
Game engine behind Sea Dogs, Pirates of the Caribbean and Age of Pirates games.

Game engine behind Sea Dogs, Pirates of the Caribbean and Age of Pirates games.

Storm Devs 693 Dec 29, 2022