Exposes Azure Kinect Support for integration into Unreal Engine Applications.

Overview

Azure Kinect for Unreal Engine

Exposes Azure Kinect Support for integration into Unreal Engine Applications. Mainly for depth and color textures creation from Kinect's raw feed.

Prerequisites

  • Platform: Win64
  • Dependencies:
    • Azure Kinect SDK v1.4.1 is installed
      • Download from here
      • An env variable AZUREKINECT_SDK that points to the Azure Kinect SDK root path should be registered.
    • Azure Kinect Body Tracking SDK v1.1.0 is installed
      • Download from here
      • An env variable AZUREKINECT_BODY_SDK that points to the Azure Kinect Body Tracking SDK root path should be registered.
    • This plugin cannot be neither built nor open without the SDKs and paths above.
  • Unreal Engine 4.27~
    • Only tested with 4.27. May work with lower.

Features

In-Editor activation

  • Write Depth / Color buffer into RenderTarget2Ds.

Blueprint activation

Niagara Particle

  • You can modify base a niagara system NS_KinectParticle.

Skeleton tracking

  • Bone mapping node in Anim Graph

Notice

Depthe data are stored RenderTarget2D into standard 8bit RGBA texture.
R: first 8bit as uint8 of original uint16 sample
G: last 8bit as uint8 of original uint16 sample
B: 0x00 or 0xFF (if depth sample is invalid)
A: 0xFF (Constant value)

Thus we need conversion to acquire orignal depth samples.

// In MaterialEditor or Niagara, sample values in Depth texture are normalized to 0-1.
float DepthSample = (G * 256.0 + R) * 256.0; // millimetor
// In C++
uint8 R = Sample.R, G = Sample.G;
uint16 DepthSample = G << 8 | R;  // millimetor

Depth pixel from Azure Kinect SDK is originally a single uint16 in millimetor. But RenderTarget2D can't store uint16 as texture (EPixelFormat::PF_R16_UINT doesn't work for RenderTarget).

Reference

Existing plugin for Azure Kinect

License

MIT License

Copyright 2021 Ayumu Nagamtsu

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Comments
  • Blueprint in BP_AzureKinect Actor does not compile.

    Blueprint in BP_AzureKinect Actor does not compile.

    I get the following errors when trying to use the BP_AzureKinect Actor:

    • In use pin Anim BP no longer exists on node Get . Please refresh node or break links to remove pin.
    • In use pin Anim BP no longer exists on node Set . Please refresh node or break links to remove pin.
    • Can't connect pins Object and Return Value : This cast has an invalid target type (was the class deleted without a redirect?).
    • In use pin As ABP Aure Kinect Skeleton To Humanoid no longer exists on node Bad cast node . Please refresh node or break links to remove pin.
    • Can't connect pins Return Value and Object : This cast has an invalid target type (was the class deleted without a redirect?).
    • The property associated with Skeleton could not be found in '/Game/Kinect/BP_AzureKinectActor.BP_AzureKinectActor_C'

    kind regards Thomas

    bug 
    opened by 120decibel 12
  • StopDevice() Add RemapImage.reset()

    StopDevice() Add RemapImage.reset()

    Hi, @nama-gatsuo

    When I use this plugin I think that StopDevice() should contain the following contents.

    image

    This is because if you change the Remap Mode after executing it once and exiting, the previous RemapImage data remains and is not applied.

    bug 
    opened by YongsunLee 1
  • The button

    The button"LoadDevice"and"Startdevice" does not respond

    Hello,I encountered some problems after importing the plug-in file into the project,these uasset files can be displayed normally.But the"AzureKinectDevice"does not work properly-When I click on it,the button"LoadDevice"and"Startdevice" does not respond,I don't know what went wrong. Does anyone have a similar problem?And how to solve it, I would appreciate it.

    KP4`){H5@EXQ1%TNA76 ZJI

    opened by 990408 0
  • Plugin 'AzureKinect' failed to load because module 'AzureKinect' could not be loaded.

    Plugin 'AzureKinect' failed to load because module 'AzureKinect' could not be loaded.

    Hello,

    After building and running, I get the following error:

    1

    I've made sure to include all .dll and .lib files from Azure Kinect SDK and Body Tracking SDK in the .\Plugins\AzureKinectForUE-master\Binaries\Win64 folder. I've tried rebuilding, cleaning, and running again. I've tried modifying the AzureKinect.Build.cs file as in this issue. The log says it fails to preload k4abt.dll, k4a.dll, and UE4Editor-AzureKinect.dll even though it is looking in the correct path (.\Plugins\AzureKinectForUE-master\Binaries\Win64) and the files are present.

    Log file: Log.log

    Any help would be greatly appreciated. Thank you!

    Windows 10 UE 4.27 Visual Studio 2019

    opened by jessekirbs 8
  • AzureKinectDeviceLog: Error: Cant't open: Failed to create k4abt tracker!

    AzureKinectDeviceLog: Error: Cant't open: Failed to create k4abt tracker!

    AzureKinectDeviceLog: Error: Cant't open: Failed to create k4abt tracker! Has anyone got this error and found a fix for that? Im getting error only when Skeleton Tracking is on.

    opened by MarisFreimanis 1
  • More detailed tutorial

    More detailed tutorial

    Dear Developer, may you create a more detailed tutorial for this plugin? For example, I would need a description of the blueprint methods used to update the skeleton for each frame. Thank you in advance for your reply. Best regards.

    opened by Art3m1s-GITHUB 0
  • How to packaging include this plugin

    How to packaging include this plugin

    Hi, @nama-gatsuo

    I am trying to package (win64) a project using this plugin.

    I'm getting an error in the packaging and I want to know what I'm missing.

    Can you give me an answer about the packaging?

    bug 
    opened by YongsunLee 7
  • k4a_capture_t invalid problem

    k4a_capture_t invalid problem

    image

    k4a.log k4abt.log

    Hi, @nama-gatsuo

    I am using this plugin you provided with gratitude.

    It was found that there were no problems during use.

    That's why I thought the warning in the photo above was caused by my editing.

    But I also found the same warning log in the base project with the variant removed.

    That is why I am writing this article by attaching the log file.

    May I know what is causing this issue?

    opened by YongsunLee 1
Owner
Ayumu Nagamatsu
a creator who loves generative art, audio reactive things, data driven art
Ayumu Nagamatsu
Unreal Engine 4 vulnerability, that allows you to run shellcode directly into the target game process.

Unreal Engine 4 vulnerability, that allows you to run shellcode directly into the target game process, to load any DLL undetected from most game anti cheats, such as Easy Anti Cheat, BattleEye, Ricochet, Vanguard, ATG, and more.

Zebratic 49 Jan 4, 2023
Lambda support for Unreal Engine dynamic delegates

DynamicLambda Lambda support for Unreal Engine dynamic delegates This is experimental feature. Now only parametless lambdas are supported To see more

Andrew Derkach 28 Nov 29, 2022
Unreal Engine 4 plugin for SteamVR passthrough camera support

SteamVR Passthrough Plugin This Unreal Engine 4 plugin adds SteamVR passthrough camera support using the OpenVR TrackedCamera API. Example project: ht

null 10 Dec 5, 2022
Mobile Studio tool integration with C# scripting for the Unity game engine.

Mobile Studio Package This project is a Unity package for integrating the Mobile Studio tool suite into game development workflows. This version of th

Arm Software 76 Dec 7, 2022
C++ game engine inspired by quake. Modern rendering and quake mapping tool integration.

Nuake Feel free to join the discord server for updates: What is it Nuake is a game engine written from scratch by myself. It is not meant to be a end-

Antoine Pilote 25 Oct 17, 2022
Full source code for WarriOrb, a Dark-Souls like action platformer - using Unreal Engine 4

WarriOrb source code WarriOrb is a hardcore action platformer where you play as a demon who is trapped in an unlikely body. The game mixes the difficu

Not Yet 247 Dec 30, 2022
Design-agnostic node editor for scripting game’s flow in Unreal Engine

Flow plug-in for Unreal Engine provides a graph editor tailored for scripting flow of events in virtual worlds. It's based on a decade of experie

Moth Cocoon 555 Jan 4, 2023
Niagara UI Renderer | Free Plugin for Unreal Engine 4

Niagara UI Renderer | Free Plugin for Unreal Engine 4 Niagara UI Plugin adds Niagara Particle System Widget that allows you to render Niagara particle

null 156 Dec 19, 2022
Third-person survival game for Unreal Engine 4 made entirely in C++.

Third-person survival game for Unreal Engine 4 made entirely in C++. Originally built as a 6 section tutorial series, now available as open-source C++ sample project.

Tom Looman 2.8k Dec 30, 2022
RenderStream plugin for Unreal Engine

This project relies on http://disguise.one software to function. For the plugin setup process - please see https://help.disguise.one/Content/Configuri

disguise 41 Dec 19, 2022
Simple CSV localization system for Unreal Engine 4

BYG Localization We wanted to support fan localization for Industries of Titan and found that Unreal's built-in localization system was not exactly wh

Brace Yourself Games 56 Dec 8, 2022
Edycja PianoFall zrobiona na Unreal Engine

PianoFall - Unreal Engine Edition Edycja PianoFall zrobiona na Unreal Engine (mój pierwszy projekt w UE) Obsługa Po uruchomieniu programu i wciśnięciu

Nadwey 4 Jun 17, 2021
Dialogue scripting language for Unreal Engine

Supertalk Welcome to Supertalk! This is a simple dialogue scripting language for Unreal Engine created due to frustration with visual dialogue tree wo

Sam Bloomberg 41 Nov 18, 2022
An Unreal Engine 4 Dungeon Generator

DungeonGenerator A (hopefully) handy Unreal Engine 4 Dungeon Generator Read the full docs here: https://github.com/orfeasel/DungeonGenerator/blob/main

Orfeas Eleftheriou 69 Dec 28, 2022
Procedural Mesh Modeling Toolkit for Unreal Engine Artists

OpenLand Mesh Procedural Mesh Modeling Toolkit for Unreal Engine Artists. Installation Get it via the marketplace ??️ For non-commercial projects, you

GDi4K 26 Nov 19, 2022
Creating Unreal Engine infinite landscapes/oceans using the editor shader graph and rendering them using Geometry ClipMap. It also allows to spawn mesh on landscape surface. UE5 required

Procedural Landscapes and Oceans in Unreal Engine 5 using Editor Shader Graph Latest version of this project is available as a plugin for UE 4.26+ on

Maxime Dupart 10 Oct 4, 2021
Unreal Engine Plugin to wrap Nuitrack SDK ( skeleton tracking solution by 3DiVi )

Nuitrack for Unreal Engine Unreal Engine plugin to bridge Nuitrack. Nuitrack is a middleware to provide 3d skeleton tracking solution using a depth se

Ayumu Nagamatsu 11 Nov 10, 2022
HLSL Material for Unreal Engine

HLSL Material for Unreal Engine Ever wanted to write complex material functions directly in HLSL? Now you can! Unreal Engine 4.26, 4.27 and 5.0 are su

null 417 Dec 28, 2022
Building Escape is a simple room escape game made with Unreal Engine 4.27 and C++.

Building-Escape Building Escape is a simple room escape game made with Unreal Engine and C++. The main purpose of the game is to find a way to escape

Christine Coomans 2 Dec 13, 2021