UE4 MediaPipe plugin

Overview

UE4 MediaPipe plugin

Platforms: Win64

2D features: Face, Iris, Hands, Pose, Holistic

3D features: Face Mesh, World Pose

Demo video: https://www.youtube.com/watch?v=_gRGjGn6FQE

More details: https://github.com/google/mediapipe

Custom build howto

Clone plugin

git clone https://github.com/wongfei/ue4-mediapipe-plugin.git

Clone wrapper

git clone -b unreal https://github.com/wongfei/mediapipe.git ue4-mediapipe-wrapper

Setup workspace

Build wrapper

cd ue4-mediapipe-wrapper

bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 --action_env PYTHON_BIN_PATH="C:\\Python39\\python.exe" mediapipe/unreal:ump_shared

More info: ue4-mediapipe-wrapper\mediapipe\unreal\scripts\build_shared.cmd

Copy ump_shared.dll to ue4-mediapipe-plugin\Plugins\MediaPipe\ThirdParty\mediapipe\Binaries\Win64\

More info: ue4-mediapipe-wrapper\mediapipe\unreal\scripts\deploy.cmd

Other deps

Protobuf 3.11.4

Copy libprotobuf.lib to ue4-mediapipe-plugin\Plugins\MediaPipe\ThirdParty\protobuf\Lib\Win64\

OpenCV 3.4.10 with fixed CAP_MSMF (optional)

git clone -b msmf_fix_3410 https://github.com/wongfei/opencv.git

Copy opencv_world3410.dll to ue4-mediapipe-plugin\Plugins\MediaPipe\ThirdParty\mediapipe\Binaries\Win64\

Issues
  • Some problems in practical use

    Some problems in practical use

    Visibility and presence should mean the same thing. I don't understand the difference between the two image Pose point has only position vector and no directional Euler angle, which leads to low efficiency in driving the model image I try to calculate by mathematical operation, but the effect is not very ideal. image Since last year, I have had the idea of porting mediapipe to fantasy. But my code is very bad. I spend more time using blueprints. Thank you very much for making this plug-in. I want to make it better

    opened by salier 24
  • MacOS and HTML5

    MacOS and HTML5

    Hi Wongfei,

    I am currently looking into the code an dusage of the dlls and protobuf.

    I am sure with some work this plugin could be edited to also include the MacOS platform, right?

    And would it work to build to an HTML5 package?

    Are you available for commission?

    Thanks,

    Aaron

    Edit: I asked my colleague on Windows to open the project but he got the same error of not being able to build. He was on UE4.26.2.

    opened by aaronvdbrugge 7
  •  LogMediaPipe: Error: ObjectId out of range

    LogMediaPipe: Error: ObjectId out of range

    log.txt An error is reported at runtime logmediapipe: error: objectid out of range The number of frames is greatly reduced (from 200 frames to about 60 frames) The array length of 0 is the camera start-up time (or the face is not recognized)

    opened by salier 7
  • Video Feed before MediaPipe processing

    Video Feed before MediaPipe processing

    Hey Wei Fong,

    Is there a setting to turn off the rendering of the Landmarks in the video texture? I'd like to still use the Video without the landmark rendering. Or maybe is there somewhere I can use the video feed before it gets processed by mediapipe?

    opened by tbKorber 5
  • Turning the 'enable overlay' variable to false breaks the hand landmarks.

    Turning the 'enable overlay' variable to false breaks the hand landmarks.

    Hello! I am nearing my project's end using this awesome plugin, but I'm facing a slight issue wrapping things up.

    I don't want the Mediapipe window showing the raw camera feed to pop up on the final packaged project. Turning the 'enable overlay' variable to false seems to do that, but it greatly reduces the performance of the hand landmark tracking.

    My intended outcome is simply to prevent the extra window showing the webcam feed from popping up. Am I doing this the wrong way?

    Thank you.

    opened by EDsizm 4
  • Questions about code location

    Questions about code location

    Questions where can I check the code that connects the landmark of the media pipe to the skeleton bones of the unreal engine? I want to learn that process! thanks

    opened by donguk071 3
  • BlackScreen after Package for windows. UE 5.01

    BlackScreen after Package for windows. UE 5.01

    Good afternoon. The last source. When packing a project under Windows, I can't start the resulting application. The application window freezes and nothing else happens. Even on an empty map. In the editor, everything starts and works like a charm.

    opened by UmkaApps 3
  • holistic landmarks detection does not output hand detection confidence

    holistic landmarks detection does not output hand detection confidence

    Hi! Wongfei, When running with this plugin, I encountered the following problem: The hand landmarks detection (configured by multi_hand_landmarks.pbtxt) output hand detection confidence, while the holistic landmarks detection (configured by holistic_landmarks.pbtxt or holistic_with_iris.pbtxt) does not output the hand detection confidence. How to solve this issue? Can I just modify holistic_landmarks.pbtxt to add the corresponding output without changing the code structure. Looking forward to your reply!

    opened by increase24 3
  • Compiled the UE4 project. How to connect the camera?

    Compiled the UE4 project. How to connect the camera?

    I have compiled the project and when I started to run the level the MediaPipe window appears black. I guess that is not connected to the webcam. Can you explain me where or in which config file can I address the webcam?

    Best regards, Screenshot 2021-07-28 232117

    Mirko

    opened by PinkMediaGroup 3
  • Error LNK2005 & Fatal Error LNK1169 when packaging with other plugins

    Error LNK2005 & Fatal Error LNK1169 when packaging with other plugins

    I am currently using your plugin alongside 2 others, OVRLipSync as well as JsonCsvToDataTableToStruct and they managed to package fine without your plugin in the project.

    But when I include your plugin into the project it fails to do a successful package. Attached is the error log from when it failed to package.

    Would you have any idea why this is happening? I'd appreciate all the help I can get!!

    LogUObjectHash: Compacting FUObjectHashTables data took   1.13ms
    LogMainFrame: Project requires temp target (SunPosition plugin is enabled)
    LogLauncherProfile: Unable to use promoted target - ../../../../../../Users/MyComputer/Desktop/TestingProject/Binaries/Win64/UE4Game-Win64-Shipping.target does not exist.
    UATHelper: Packaging (Windows (64-bit)): Running AutomationTool...
    UATHelper: Packaging (Windows (64-bit)): Parsing command line: -ScriptsForProject=C:/Users/MyComputer/Desktop/TestingProject/MotionTrackTest.uproject BuildCookRun -nocompileeditor -installed -nop4 -project=C:/Users/MyComputer/Desktop/TestingProject/MotionTrackTest.uproject -cook -stage -archive -archivedirectory=C:/Users/Dex-lab/Desktop/3-Combo-Plugin -pac
    kage -ue4exe="C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\Win64\UE4Editor-Cmd.exe" -clean -compressed -ddc=InstalledDerivedDataBackendGraph -pak -prereqs -nodebuginfo -targetplatform=Win64 -build -clientconfig=Shipping -utf8output
    UATHelper: Packaging (Windows (64-bit)): Setting up ProjectParams for C:\Users\MyComputer/Desktop/TestingProject/MotionTrackTest.uproject
    UATHelper: Packaging (Windows (64-bit)): MotionTrackTest.uproject requires a temporary target.cs to be generated (SunPosition plugin is enabled)
    UATHelper: Packaging (Windows (64-bit)): ********** BUILD COMMAND STARTED **********
    UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe MotionTrackTest Win64 Shipping -Project=C:\MyComputer/Desktop/TestingProject/MotionTrackTest.uproject -Clean -NoHotReload C:\MyComputer/Desktop/TestingProject/MotionTrackTest.uproject -NoUBTMakefiles  -remoteini="C:
    \Users\Dex-lab\Desktop\IslandHealth_POC" -skipdeploy -nobuilduht -log="C:\Users\MyComputer\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MotionTrackTest-Win64-Shipping.txt"
    UATHelper: Packaging (Windows (64-bit)):   Cleaning OKI_AnimationTest binaries...
    UATHelper: Packaging (Windows (64-bit)): Took 0.8157848s to run UnrealBuildTool.exe, ExitCode=0
    UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe OKI_AnimationTest Win64 Shipping -Project=C:\Users\Dex-lab\Desktop\IslandHealth_POC\OKI_AnimationTest.uproject  C:\Users\Dex-lab\Desktop\IslandHealth_POC\OKI_AnimationTest.uproject -NoUBTMakefiles  -remoteini="C:\Users\Dex-lab\Desk
    top\IslandHealth_POC" -skipdeploy -Manifest=C:\Users\Dex-lab\Desktop\IslandHealth_POC\Intermediate\Build\Manifest.xml -NoHotReload -log="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-OKI_AnimationTest-Win64-Shipping_2.txt"
    UATHelper: Packaging (Windows (64-bit)):   Using 'git status' to determine working set for adaptive non-unity build (C:\Users\Dex-lab\Desktop\IslandHealth_POC).
    UATHelper: Packaging (Windows (64-bit)):   Parsing headers for OKI_AnimationTest
    UATHelper: Packaging (Windows (64-bit)):     Running UnrealHeaderTool "C:\Users\Dex-lab\Desktop\IslandHealth_POC\OKI_AnimationTest.uproject" "C:\Users\Dex-lab\Desktop\IslandHealth_POC\Intermediate\Build\Win64\OKI_AnimationTest\Shipping\OKI_AnimationTest.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors 
    -abslog="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UHT-OKI_AnimationTest-Win64-Shipping_2.txt" -installed
    UATHelper: Packaging (Windows (64-bit)):   LogInit: Display: Loading text-based GConfig....
    UATHelper: Packaging (Windows (64-bit)):   Reflection code generated for OKI_AnimationTest in 4.8656366 seconds
    UATHelper: Packaging (Windows (64-bit)):   Writing manifest to C:\Users\Dex-lab\Desktop\IslandHealth_POC\Intermediate\Build\Manifest.xml
    UATHelper: Packaging (Windows (64-bit)):   Building OKI_AnimationTest...
    UATHelper: Packaging (Windows (64-bit)):   Using Visual Studio 2019 14.29.30137 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133) and Windows 10.0.19041.0 SDK (C:\Program Files (x86)\Windows Kits\10).
    UATHelper: Packaging (Windows (64-bit)):   [Upgrade]
    UATHelper: Packaging (Windows (64-bit)):   [Upgrade] Using backward-compatible build settings. The latest version of UE4 sets the following values by default, which may require code changes:
    UATHelper: Packaging (Windows (64-bit)):   [Upgrade]     bLegacyPublicIncludePaths = false                 => Omits subfolders from public include paths to reduce compiler command line length. (Previously: true).
    UATHelper: Packaging (Windows (64-bit)):   [Upgrade]     ShadowVariableWarningLevel = WarningLevel.Error   => Treats shadowed variable warnings as errors. (Previously: WarningLevel.Warning).
    UATHelper: Packaging (Windows (64-bit)):   [Upgrade]     PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs   => Set in build.cs files to enables IWYU-style PCH model. See https://docs.unrealengine.com/en-US/Programming/BuildTools/UnrealBuildTool/IWYU/index.html. (Previously: PCHUsageMode.UseSharedPCHs).
    UATHelper: Packaging (Windows (64-bit)):   [Upgrade] Suppress this message by setting 'DefaultBuildSettings = BuildSettingsVersion.V2;' in OKI_AnimationTest.Target.cs, and explicitly overriding settings that differ from the new defaults.
    UATHelper: Packaging (Windows (64-bit)):   [Upgrade]
    UATHelper: Packaging (Windows (64-bit)):   Building 42 actions with 8 processes...
    UATHelper: Packaging (Windows (64-bit)):     [1/42] optical_flow_field_data.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [2/42] anchor.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [3/42] classification.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [4/42] face_geometry.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [5/42] SharedPCH.Core.cpp
    UATHelper: Packaging (Windows (64-bit)):     [6/42] MediaPipe.init.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [7/42] geometry_pipeline_metadata.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [8/42] MediaPipeModule.cpp
    UATHelper: Packaging (Windows (64-bit)):     [9/42] MediaPipeObserverComponent.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [10/42] MediaPipePipelineComponent.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [11/42] MediaPipeLandmarkObserverComponent.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [12/42] MediaPipePresenceObserverComponent.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [13/42] matrix_data.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [14/42] DynamicTexture.cpp
    UATHelper: Packaging (Windows (64-bit)):     [15/42] environment.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [16/42] image_file_properties.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [17/42] MediaPipeClassificationObserverComponent.cpp
    UATHelper: Packaging (Windows (64-bit)):     [18/42] locus.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [19/42] mesh_3d.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [20/42] rasterization.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [21/42] MediaPipeClassificationObserverComponent.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [22/42] rect.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [23/42] MediaPipeObserverComponent.cpp
    UATHelper: Packaging (Windows (64-bit)):     [24/42] time_series_header.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [25/42] image_format.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [26/42] landmark.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [27/42] MediaPipePresenceObserverComponent.cpp
    UATHelper: Packaging (Windows (64-bit)):     [28/42] Default.rc2
    UATHelper: Packaging (Windows (64-bit)):     [29/42] OKI_AnimationTest.cpp
    UATHelper: Packaging (Windows (64-bit)):     [30/42] MediaPipeFaceMeshObserverComponent.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [31/42] detection.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [32/42] location_data.pb.cc
    UATHelper: Packaging (Windows (64-bit)):     [33/42] MediaPipePipelineComponent.cpp
    UATHelper: Packaging (Windows (64-bit)):     [34/42] SharedPCH.Engine.cpp
    UATHelper: Packaging (Windows (64-bit)):     [35/42] MediaPipeLandmarkObserverComponent.cpp
    UATHelper: Packaging (Windows (64-bit)):     [36/42] Module.JsonCsvToDataTableToStruct.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [37/42] Module.OVRLipSync.gen.cpp
    UATHelper: Packaging (Windows (64-bit)):     [38/42] Module.OVRLipSync.cpp
    UATHelper: Packaging (Windows (64-bit)):     [39/42] MediaPipeFaceMeshObserverComponent.cpp
    UATHelper: Packaging (Windows (64-bit)):     [40/42] Module.JsonCsvToDataTableToStruct.cpp
    UATHelper: Packaging (Windows (64-bit)):     [41/42] OKI_AnimationTest-Win64-Shipping.exe
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::VerifyVersion(int,int,char const *)" ([email protected]@[email protected]@@[email protected]) already defined in webrtc.lib(common.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void * __cdecl google::protobuf::internal::ArenaImpl::AllocateAlignedAndAddCleanup(unsigned __int64,void (__cdecl*)(void *))" ([email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in webrtc.lib(arena.obj
    )
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "private: void __cdecl google::protobuf::Arena::OnArenaAllocation(class type_info const *,unsigned __int64)const " ([email protected]@[email protected]@@[email protected]@[email protected]) already defined in webrtc.lib(arena.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::OnShutdownRun(void (__cdecl*)(void const *),void const *)" ([email protected]@[email protected]@@[email protected]@Z) already defined in webrtc.lib(common.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "protected: void __cdecl google::protobuf::internal::RepeatedPtrFieldBase::Reserve(int)" ([email protected]@[email protected]@[email protected]@[email protected]) already defined in webrtc.lib(repeated_field.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "private: void __cdecl google::protobuf::internal::RepeatedPtrFieldBase::MergeFromInternal(class google::protobuf::internal::RepeatedPtrFieldBase const &,void (__cdecl google::protobuf::internal::RepeatedPtrFieldBase::*)(void * *,void * *,int,int))" (?MergeFromInte
    [email protected]@[email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in webrtc.lib(repeated_field.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::InitSCCImpl(struct google::protobuf::internal::SCCInfoBase *)" ([email protected]@[email protected]@@[email protected]@@Z) already defined in webrtc.lib(generated_message_util.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::DestroyMessage(void const *)" ([email protected]@[email protected]@@[email protected]) already defined in webrtc.lib(generated_message_util.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > * __cdecl google::protobuf::internal::ArenaStringPtr::MutableNoArena(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const *)" (
    [email protected]@[email protected]@[email protected]@[email protected][email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in webrtc.lib(arenastring.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ArenaStringPtr::DestroyNoArena(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const *)" ([email protected]@[email protected]@[email protected]@[email protected]?$
    [email protected]@[email protected]@[email protected]@[email protected]@[email protected]@@Z) already defined in webrtc.lib(arenastring.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: static unsigned __int64 __cdecl google::protobuf::internal::WireFormatLite::Int32Size(class google::protobuf::RepeatedField<int> const &)" ([email protected]@[email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in webrtc.lib(w
    ire_format_lite.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: __cdecl google::protobuf::internal::ExtensionSet::ExtensionSet(void)" ([email protected]@[email protected]@@[email protected]) already defined in webrtc.lib(extension_set.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: __cdecl google::protobuf::internal::ExtensionSet::~ExtensionSet(void)" ([email protected]@[email protected]@@[email protected]) already defined in webrtc.lib(extension_set.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ExtensionSet::Clear(void)" ([email protected]@[email protected]@[email protected]@QEAAXXZ) already defined in webrtc.lib(extension_set.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ExtensionSet::MergeFrom(class google::protobuf::internal::ExtensionSet const &)" ([email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in webrtc.lib(extension_set.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ExtensionSet::Swap(class google::protobuf::internal::ExtensionSet *)" ([email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in webrtc.lib(extension_set.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: bool __cdecl google::protobuf::internal::ExtensionSet::IsInitialized(void)const " ([email protected]@[email protected]@[email protected]@QEBA_NXZ) already defined in webrtc.lib(extension_set.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: unsigned __int64 __cdecl google::protobuf::internal::ExtensionSet::ByteSize(void)const " ([email protected]@[email protected]@[email protected]@QEBA_KXZ) already defined in webrtc.lib(extension_set.obj)
    UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: static unsigned __int64 __cdecl google::protobuf::internal::WireFormatLite::UInt32Size(class google::protobuf::RepeatedField<unsigned int> const &)" ([email protected]@[email protected]@[email protected]@[email protected]@[email protected]@Z) already defined in w
    ebrtc.lib(wire_format_lite.obj)
    UATHelper: Packaging (Windows (64-bit)):        Creating library C:\Users\Dex-lab\Desktop\IslandHealth_POC\Binaries\Win64\OKI_AnimationTest-Win64-Shipping.lib and object C:\Users\Dex-lab\Desktop\IslandHealth_POC\Binaries\Win64\OKI_AnimationTest-Win64-Shipping.exp
    UATHelper: Packaging (Windows (64-bit)):     C:\Users\Dex-lab\Desktop\IslandHealth_POC\Binaries\Win64\OKI_AnimationTest-Win64-Shipping.exe : fatal error LNK1169: one or more multiply defined symbols found
    UATHelper: Packaging (Windows (64-bit)): Took 57.1001483s to run UnrealBuildTool.exe, ExitCode=6
    UATHelper: Packaging (Windows (64-bit)): UnrealBuildTool failed. See log for more details. (C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-OKI_AnimationTest-Win64-Shipping_2.txt)
    UATHelper: Packaging (Windows (64-bit)): AutomationTool exiting with ExitCode=6 (6)
    UATHelper: Packaging (Windows (64-bit)): BUILD FAILED
    PackagingResults: Error: Unknown Error
    
    opened by Jeremiah-DexLab 2
  • Packaging MediaPipe with OVRLipSync Error

    Packaging MediaPipe with OVRLipSync Error

    Hi Wong Fei,

    I've tested packaging with only the mediapipe plugin aswell as only the Oculus VR Lip Sync plugin and both are able to be packaged fine on their own. However trying to package them together produces an error that my colleague and I haven't been able to figure out.

    If you have any idea why this is happening know of a fix it would be greatly appreciated.

    (It seems that Github will wrap the text to fit so it's still difficult to read even when marked as code, sorry) The Output Log for the Packaging Process:

    LogUObjectHash: Compacting FUObjectHashTables data took 0.80ms LogMainFrame: Selected target: MediaPipeDemo LogLauncherProfile: Unable to use promoted target - D:/Unreal Projects/ue4-mediapipe-plugin/Binaries/Win64/MediaPipeDemo-Win64-Shipping.target does not exist. UATHelper: Packaging (Windows (64-bit)): Running AutomationTool... UATHelper: Packaging (Windows (64-bit)): Parsing command line: -ScriptsForProject="D:/Unreal Projects/ue4-mediapipe-plugin/MediaPipeDemo.uproject" BuildCookRun -nocompileeditor -installed -nop4 -project="D:/Unreal Projects/ue4-mediapipe-plugin/MediaPipeDemo.uproject" -cook -stage -archive -archivedirectory="D:/Unreal Projects/MediaPipeBuild" -package -ue4exe ="C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\Win64\UE4Editor-Cmd.exe" -clean -compressed -ddc=InstalledDerivedDataBackendGraph -pak -prereqs -distribution -nodebuginfo -targetplatform=Win64 -build -target=MediaPipeDemo -clientconfig=Shipping -utf8output UATHelper: Packaging (Windows (64-bit)): Setting up ProjectParams for D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject UATHelper: Packaging (Windows (64-bit)): ********** BUILD COMMAND STARTED ********** UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe MediaPipeDemo Win64 Shipping -Project="D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" -Clean -NoHotReload "D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" -NoUBTMakefiles -remoteini="D:\Unreal Proj ects\ue4-mediapipe-plugin" -skipdeploy -distribution -nobuilduht -log="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MediaPipeDemo-Win64-Shipping.txt" UATHelper: Packaging (Windows (64-bit)): Cleaning MediaPipeDemo binaries... UATHelper: Packaging (Windows (64-bit)): Took 1.1431025s to run UnrealBuildTool.exe, ExitCode=0 UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe MediaPipeDemo Win64 Shipping -Project="D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" "D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" -NoUBTMakefiles -remoteini="D:\Unreal Projects\ue4-mediapipe- plugin" -skipdeploy -distribution -Manifest="D:\Unreal Projects\ue4-mediapipe-plugin\Intermediate\Build\Manifest.xml" -NoHotReload -log="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MediaPipeDemo-Win64-Shipping_2.txt" UATHelper: Packaging (Windows (64-bit)): Using 'git status' to determine working set for adaptive non-unity build (D:\Unreal Projects\ue4-mediapipe-plugin). UATHelper: Packaging (Windows (64-bit)): Parsing headers for MediaPipeDemo UATHelper: Packaging (Windows (64-bit)): Running UnrealHeaderTool "D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" "D:\Unreal Projects\ue4-mediapipe-plugin\Intermediate\Build\Win64\MediaPipeDemo\Shipping\MediaPipeDemo.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -abslog="C:\User s\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UHT-MediaPipeDemo-Win64-Shipping_2.txt" -installed UATHelper: Packaging (Windows (64-bit)): LogInit: Display: Loading text-based GConfig.... UATHelper: Packaging (Windows (64-bit)): Reflection code generated for MediaPipeDemo in 4.7631709 seconds UATHelper: Packaging (Windows (64-bit)): Writing manifest to D:\Unreal Projects\ue4-mediapipe-plugin\Intermediate\Build\Manifest.xml UATHelper: Packaging (Windows (64-bit)): Building MediaPipeDemo... UATHelper: Packaging (Windows (64-bit)): Using Visual Studio 2019 14.29.30137 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133) and Windows 10.0.19041.0 SDK (C:\Program Files (x86)\Windows Kits\10). UATHelper: Packaging (Windows (64-bit)): Building 42 actions with 8 processes... UATHelper: Packaging (Windows (64-bit)): [1/42] mesh_3d.pb.cc UATHelper: Packaging (Windows (64-bit)): [2/42] classification.pb.cc UATHelper: Packaging (Windows (64-bit)): [3/42] landmark.pb.cc UATHelper: Packaging (Windows (64-bit)): [4/42] locus.pb.cc UATHelper: Packaging (Windows (64-bit)): [5/42] environment.pb.cc UATHelper: Packaging (Windows (64-bit)): [6/42] time_series_header.pb.cc UATHelper: Packaging (Windows (64-bit)): [7/42] rect.pb.cc UATHelper: Packaging (Windows (64-bit)): [8/42] MediaPipePresenceObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [9/42] MediaPipeObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [10/42] detection.pb.cc UATHelper: Packaging (Windows (64-bit)): [11/42] image_file_properties.pb.cc UATHelper: Packaging (Windows (64-bit)): [12/42] rasterization.pb.cc UATHelper: Packaging (Windows (64-bit)): [13/42] MediaPipeClassificationObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [14/42] MediaPipeLandmarkObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [15/42] matrix_data.pb.cc UATHelper: Packaging (Windows (64-bit)): [16/42] MediaPipePipelineComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [17/42] anchor.pb.cc UATHelper: Packaging (Windows (64-bit)): [18/42] image_format.pb.cc UATHelper: Packaging (Windows (64-bit)): [19/42] MediaPipeClassificationObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [20/42] MediaPipeFaceMeshObserverComponent.cpp LogUdpMessaging: Warning: FUdpMessageProcessor::UpdateReassemblers Discarding 1/2 of stale message segements from Sender=10.235.10.219:63482, Sequence=0 UATHelper: Packaging (Windows (64-bit)): [21/42] geometry_pipeline_metadata.pb.cc UATHelper: Packaging (Windows (64-bit)): [22/42] location_data.pb.cc UATHelper: Packaging (Windows (64-bit)): [23/42] MediaPipePipelineComponent.cpp UATHelper: Packaging (Windows (64-bit)): [24/42] MediaPipeFaceMeshObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [25/42] optical_flow_field_data.pb.cc UATHelper: Packaging (Windows (64-bit)): [26/42] MediaPipePresenceObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [27/42] Default.rc2 UATHelper: Packaging (Windows (64-bit)): [28/42] MediaPipeLandmarkObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [29/42] face_geometry.pb.cc UATHelper: Packaging (Windows (64-bit)): [30/42] MediaPipeModule.cpp UATHelper: Packaging (Windows (64-bit)): [31/42] MediaPipeObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [32/42] MediaPipe.init.gen.cpp UATHelper: Packaging (Windows (64-bit)): [33/42] DynamicTexture.cpp UATHelper: Packaging (Windows (64-bit)): [34/42] SharedPCH.Engine.ShadowErrors.cpp UATHelper: Packaging (Windows (64-bit)): [35/42] Module.OVRLipSync.cpp UATHelper: Packaging (Windows (64-bit)): D:/Unreal Projects/ue4-mediapipe-plugin/Plugins/OVRLipSync/Source/OVRLipSync/Private/CookFrameSequenceAsync.cpp(7): fatal error C1083: Cannot open include file: 'Async.h': No such file or directory UATHelper: Packaging (Windows (64-bit)): [36/42] Module.OVRLipSync.gen.cpp UATHelper: Packaging (Windows (64-bit)): D:\Unreal Projects\ue4-mediapipe-plugin\Plugins\OVRLipSync\Source\OVRLipSync/Public/OVRLipSyncPlaybackActorComponent.h(26): fatal error C1083: Cannot open include file: 'Classes/Components/AudioComponent.h': No such file or directory UATHelper: Packaging (Windows (64-bit)): [37/42] MediaPipeDemo.cpp UATHelper: Packaging (Windows (64-bit)): [38/42] MediaPipeDemo.init.gen.cpp UATHelper: Packaging (Windows (64-bit)): [39/42] MediaPipeCppExample.cpp UATHelper: Packaging (Windows (64-bit)): [40/42] MediaPipeCppExample.gen.cpp UATHelper: Packaging (Windows (64-bit)): Took 65.7694112s to run UnrealBuildTool.exe, ExitCode=6 UATHelper: Packaging (Windows (64-bit)): UnrealBuildTool failed. See log for more details. (C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MediaPipeDemo-Win64-Shipping_2.txt) UATHelper: Packaging (Windows (64-bit)): AutomationTool exiting with ExitCode=6 (6) UATHelper: Packaging (Windows (64-bit)): BUILD FAILED PackagingResults: Error: Unknown Error

    opened by tbKorber 2
  • How to use this plugin

    How to use this plugin

    Hello, @wongfei , i am totally new to UE, ( i am familiar with mediapipe), recently i am getting started with learning UE, then i want to start with playing mediapipe in UE, then i find your this plugin, but i don't know how to install and use it in UE, could you please add more tutorial in README.md ? or tell me some related tutorial link ~

    opened by wwdok 1
  • The project does not open in UE 5.0.2

    The project does not open in UE 5.0.2

    If I try to open a project in UE 5.0.2, it will stop at 75% Initializing.

    When I checked the log, I checked that there was no response after opening the ump_shared.dll file for the last time, but I'm not sure if this dll is really the problem.

    How can I fix this?

    Thank you for reading it.

    opened by hyunwoolee0099 8
  • Add heads landmark to avatar

    Add heads landmark to avatar

    Thanks for your excellent job. I find the avatar's hand like thumb could not be controlled. Can I add the landmark by myself to control it in the mapping. I have tried it but it seems something wrong and avatar's thumb do not have any response. Do you know why? image

    opened by AceCHQ 0
  • 3d holistic landmarks could not be drawed well

    3d holistic landmarks could not be drawed well

    great job! When I test holistic landmarks demo, the landmarks are detected correctly, but they are not showed correctly in the right box. Please look at the hand, the image is from front view but the points in right box are seem in side view. image

    opened by ucaswindlike 0
  • eye is not close in ue4

    eye is not close in ue4

    hello,this plgus is very good.

    When I use the 3D detection of face key points, the mouth can be closed, but the eyes will not be closed. Have you encountered this kind of problem there?

    opened by tiantianaixuexi 1
Releases(v4.26-20211110)
A UE4 editor plugin for better artist/designer UI.

Details Workspace (for UE4) Details Workspace is a UE4 editor plugin for better artist/designer workflow. It allows you to create any number of "Detai

null 4 Apr 29, 2022
MediaPipe offers cross-platform, customizable ML solutions for live and streaming media.

Cross-platform, customizable ML solutions for live and streaming media.

Google 17.7k Jun 25, 2022
Example Qt application that demonstrates how to integrate Mediapipe

Mediapipe integration to Qt application example Example on how to integrate mediapipe as a dynamic library into Qt applicaton on Linux. Resulting appl

null 25 Jun 21, 2022
A mediapipe-hand demo infer by ncnn

The Android demo of Mediapipe Hand infer by ncnn Please enjoy the mediapipe hand demo on ncnn You can try this APK demo https://pan.baidu.com/s/1ArAMH

FeiGeChuanShu 30 Jun 22, 2022
The purpose of this project is to apply mediapipe to more AI chips.

1.About This Project Our Official Website: www.houmo.ai Who We Are: We are Houmo - A Great AI Company. We wish to change the world with unlimited comp

null 31 Jun 7, 2022
Native runtime package for MediaPipe.NET.

MediaPipe.NET.Runtime Native library package for MediaPipe.NET. This is the first half of the port of MediaPipeUnityPlugin, in order to use MediaPipe

Vignette 7 Feb 17, 2022
🦴 Takes base Rokoko animation export and uses my pre-made bone remapping to map to the default UE4 skeleton.

?? RokokoToUE4-pipeline ?? Takes base rokoko animation export and uses my pre-made bone remapping to map to the default UE4 skeleton. How to use: Down

Matthew Jigalin 3 Jan 10, 2022
A light and fast internet speed plugin(DDE).

lfxNet English | 简体中文 | 繁體中文 lfxNet 是一款轻量、快速的实时显示系统资源信息的应用程序。 目录 背景 编译 下载 作者 鸣谢 协议 背景 喜爱 DDE ,为 Deepin 爱好者、也是开发者之一。因习惯其它系统上有一个任务栏网速插件,但在 Deepin/UOS上没有

偕臧 53 May 31, 2022
Face tracking plugin for OBS Studio

OBS Face Tracker Plugin Introduction This plugin provide a filter for face detection and face tracking for mainly a speaking person. This plugin emplo

Norihiro Kamae 148 Jun 20, 2022
Gstreamer plugin that allows use of NVIDIA Maxine SDK in a generic pipeline.

GST-NVMAXINE Gstreamer plugin that allows use of NVIDIA MaxineTM sdk in a generic pipeline. This plugin is intended for use with NVIDIA hardware. Visi

Alex Pitrolo 14 May 11, 2022
VST/AU Plugin for Auditioning RAVE Models in Real-time

RAVE-audition VST/AU Plugin for Auditioning RAVE Models in Real-time Check out Antoine's great work in the original RAVE repository! Building the C++

Andrew Fyfe 55 Jun 4, 2022
A Qt platform plugin for running Qt/OpenGL applications offscreen

A Qt platform plugin for running Qt/OpenGL applications offscreen https://doc.qt.io/qt-6/qpa.html https://www.khronos.org/registry/EGL/extensions/MESA

null 3 Jun 5, 2022
Insomniac games cache simulation tool plugin for UE4

Insomniac Games CacheSim plugin for UE4 This plugin for Unreal Engine 4 lets you use the Insomniac Games Cache Simulation tool to detect cache misses

Toni Rebollo Berná 26 Jan 19, 2022
UE4 Plugin to execute trained PyTorch modules

SimplePyTorch UE4 Plugin to execute trained PyTorch modules ------- Packaging ------- Download PyTorch C++ distributions: https://pytorch.org/cppdocs/

null 41 Jun 28, 2022
Footsteps Plugin for UE4.

Footsteps Plugin for UE4 Packaged for UE4.27. Example Project See Footsteps Component Propertis if you dont use skeletal mesh or you dont want to use

null 3 Oct 6, 2021
A UE4 plugin used ffmpeg library to record/publish game viewport output and in-game audio.

Streaming A UE4 plugin used ffmpeg library to record/publish game viewport output and in-game audio. GPL license beacuse used ffmpeg. Structures GameV

Luo Yu 2 Feb 25, 2022
A UE4 editor plugin for better artist/designer UI.

Details Workspace (for UE4) Details Workspace is a UE4 editor plugin for better artist/designer workflow. It allows you to create any number of "Detai

null 4 Apr 29, 2022
A UE4 plugin wrapper for Molecular Matter's Live++ Hot-Reloading Library

Good News Everyone: Natively integrated in UE4.22+ Details here: https://docs.unrealengine.com/en-us/Builds/4_22 UE4 LivePP: C/C++ live coding A UE4 p

Kite & Lightning 99 May 1, 2022
MediaPipe offers cross-platform, customizable ML solutions for live and streaming media.

Cross-platform, customizable ML solutions for live and streaming media.

Google 17.7k Jun 25, 2022
Example Qt application that demonstrates how to integrate Mediapipe

Mediapipe integration to Qt application example Example on how to integrate mediapipe as a dynamic library into Qt applicaton on Linux. Resulting appl

null 25 Jun 21, 2022
A mediapipe-hand demo infer by ncnn

The Android demo of Mediapipe Hand infer by ncnn Please enjoy the mediapipe hand demo on ncnn You can try this APK demo https://pan.baidu.com/s/1ArAMH

FeiGeChuanShu 30 Jun 22, 2022
The purpose of this project is to apply mediapipe to more AI chips.

1.About This Project Our Official Website: www.houmo.ai Who We Are: We are Houmo - A Great AI Company. We wish to change the world with unlimited comp

null 31 Jun 7, 2022
Native runtime package for MediaPipe.NET.

MediaPipe.NET.Runtime Native library package for MediaPipe.NET. This is the first half of the port of MediaPipeUnityPlugin, in order to use MediaPipe

Vignette 7 Feb 17, 2022
Steve's Unreal Quest System: data-driven quest system for UE4

Steve's Unreal Quest System (SUQS) What Is It? SUQS is a simple, data-driven quest system for UE4. It helps you define quest structures for your game,

Steve Streeting 57 May 28, 2022
Helper to create a SkeletalMeshComponent in UE4 at runtime.

Runtime Skeletal Mesh Generator for UE4 Helper to create a SkeletalMeshComponent in UE4 at runtime. This is a header only library that simplify the pr

Andrea Catania 23 Jun 17, 2022
🦴 Takes base Rokoko animation export and uses my pre-made bone remapping to map to the default UE4 skeleton.

?? RokokoToUE4-pipeline ?? Takes base rokoko animation export and uses my pre-made bone remapping to map to the default UE4 skeleton. How to use: Down

Matthew Jigalin 3 Jan 10, 2022
FSD-Template - A template UE4.25 project for BP modding.

FSD-Template Project generated by Archengius' UE4 Template Generator. Reflected C++ classes generated by CheatingMuppet & Archengius' UE4SS UHT Genera

null 8 Jun 18, 2022
This a small cheat menu for Sword With Sauce (a UE4 game) that I made in a week or so.

This a small cheat menu for Sword With Sauce (a UE4 game) that I made in a week or so. The reason why I am sharing the project is that it is probably one of the best sources to learn UE game hacking from. The project has many comments.

1hAck 25 Jun 27, 2022
Useful UE4 Visual Studio extensions.

UE4 Smarter Macro Indenting This extension was designed to fix the unnecessary and annoying "smart" indenting that Visual Studio likes to do around va

Chris Pawlukowsky 241 Jun 14, 2022