Visibility and presence should mean the same thing. I don't understand the difference between the two Pose point has only position vector and no directional Euler angle, which leads to low efficiency in driving the model I try to calculate by mathematical operation, but the effect is not very ideal. Since last year, I have had the idea of porting mediapipe to fantasy. But my code is very bad. I spend more time using blueprints. Thank you very much for making this plug-in. I want to make it better

If I try to open a project in UE 5.0.2, it will stop at 75% Initializing.

When I checked the log, I checked that there was no response after opening the ump_shared.dll file for the last time, but I'm not sure if this dll is really the problem.

How can I fix this?

Thank you for reading it.

Hi Wongfei,

I am currently looking into the code an dusage of the dlls and protobuf.

I am sure with some work this plugin could be edited to also include the MacOS platform, right?

And would it work to build to an HTML5 package?

Are you available for commission?

Thanks,

Aaron

Edit: I asked my colleague on Windows to open the project but he got the same error of not being able to build. He was on UE4.26.2.

log.txt An error is reported at runtime logmediapipe: error: objectid out of range The number of frames is greatly reduced (from 200 frames to about 60 frames) The array length of 0 is the camera start-up time (or the face is not recognized)

Hey Wei Fong,

Is there a setting to turn off the rendering of the Landmarks in the video texture? I'd like to still use the Video without the landmark rendering. Or maybe is there somewhere I can use the video feed before it gets processed by mediapipe?

Hello! I am nearing my project's end using this awesome plugin, but I'm facing a slight issue wrapping things up.

I don't want the Mediapipe window showing the raw camera feed to pop up on the final packaged project. Turning the 'enable overlay' variable to false seems to do that, but it greatly reduces the performance of the hand landmark tracking.

My intended outcome is simply to prevent the extra window showing the webcam feed from popping up. Am I doing this the wrong way?

Thank you.

The plugin seems compiling correctly, but remains stuck at 75% while loading the demo project.

Recompile through VS19 going positive:

Ricompilazione avviata... 1>------ Inizio ricompilazione: Progetto: MediaPipeDemo, Configurazione: Development_Editor x64 ------ 1>Cleaning MediaPipeDemoEditor and UnrealHeaderTool binaries... 1>Using bundled DotNet SDK 1>Log file: C:\Users\Francesco\AppData\Local\UnrealBuildTool\Log.txt 1>Creating makefile for MediaPipeDemoEditor (no existing makefile) 1>Parsing headers for MediaPipeDemoEditor 1> Running UnrealHeaderTool "C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\MediaPipeDemo.uproject" "C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Intermediate\Build\Win64\MediaPipeDemoEditor\Development\MediaPipeDemoEditor.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -abslog="C:\Users\Francesco\AppData\Local\UnrealBuildTool\Log_UHT.txt" -installed 1>Reflection code generated for MediaPipeDemoEditor in 4,5386259 seconds 1>Building MediaPipeDemoEditor... 1>Using Visual Studio 2019 14.29.30145 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133) and Windows 10.0.18362.0 SDK (C:\Program Files (x86)\Windows Kits\10). 1>[Adaptive Build] Excluded from MediaPipe unity file: DynamicTexture.cpp, MediaPipeClassificationObserverComponent.cpp, MediaPipeFaceMeshObserverComponent.cpp, MediaPipeLandmarkObserverComponent.cpp, MediaPipeModule.cpp, MediaPipeObserverComponent.cpp, MediaPipePipelineComponent.cpp, MediaPipePresenceObserverComponent.cpp 1>[Adaptive Build] Excluded from MediaPipeDemo unity file: MediaPipeCppExample.cpp, MediaPipeDemo.cpp 1>Determining max actions to execute in parallel (4 physical cores, 8 logical cores) 1> Executing up to 4 processes, one per physical core 1>Building 44 actions with 4 processes... 1>[1/44] Resource Default.rc2 1>[2/44] Compile optical_flow_field_data.pb.cc 1>[3/44] Compile time_series_header.pb.cc 1>[4/44] Compile environment.pb.cc 1>[5/44] Compile MediaPipe.init.gen.cpp 1>[6/44] Compile anchor.pb.cc 1>[7/44] Compile mesh_3d.pb.cc 1>[8/44] Compile image_file_properties.pb.cc 1>[9/44] Compile MediaPipeClassificationObserverComponent.gen.cpp 1>[10/44] Compile image_format.pb.cc 1>[11/44] Compile landmark.pb.cc 1>[12/44] Compile rect.pb.cc 1>[13/44] Compile rasterization.pb.cc 1>[14/44] Compile MediaPipeLandmarkObserverComponent.gen.cpp 1>[15/44] Compile locus.pb.cc 1>[16/44] Compile MediaPipePresenceObserverComponent.cpp 1>[17/44] Compile matrix_data.pb.cc 1>[18/44] Compile MediaPipeObserverComponent.cpp 1>[19/44] Compile detection.pb.cc 1>[20/44] Compile MediaPipeFaceMeshObserverComponent.cpp 1>[21/44] Compile MediaPipeObserverComponent.gen.cpp 1>[22/44] Compile classification.pb.cc 1>[23/44] Compile SharedPCH.Engine.ShadowErrors.cpp 1>[24/44] Compile MediaPipePipelineComponent.gen.cpp 1>[25/44] Compile geometry_pipeline_metadata.pb.cc 1>[26/44] Compile MediaPipePresenceObserverComponent.gen.cpp 1>[27/44] Compile DynamicTexture.cpp 1>[28/44] Compile location_data.pb.cc 1>[29/44] Compile MediaPipeModule.cpp 1>[30/44] Compile MediaPipePipelineComponent.cpp 1>[31/44] Compile face_geometry.pb.cc 1>[32/44] Compile MediaPipeDemo.init.gen.cpp 1>[33/44] Compile MediaPipeFaceMeshObserverComponent.gen.cpp 1>[34/44] Resource Default.rc2 1>[35/44] Compile MediaPipeDemo.cpp 1>[36/44] Compile MediaPipeCppExample.gen.cpp 1>[37/44] Compile MediaPipeCppExample.cpp 1>[38/44] Link UnrealEditor-MediaPipeDemo.lib 1> Creazione della libreria C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipeDemo\UnrealEditor-MediaPipeDemo.lib e dell'oggetto C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipeDemo\UnrealEditor-MediaPipeDemo.exp 1>[39/44] Compile MediaPipeLandmarkObserverComponent.cpp 1>[40/44] Compile MediaPipeClassificationObserverComponent.cpp 1>[41/44] Link UnrealEditor-MediaPipe.lib 1> Creazione della libreria C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Plugins\MediaPipe\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipe\UnrealEditor-MediaPipe.lib e dell'oggetto C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Plugins\MediaPipe\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipe\UnrealEditor-MediaPipe.exp 1>[42/44] Link UnrealEditor-MediaPipe.dll 1> Creazione della libreria C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Plugins\MediaPipe\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipe\UnrealEditor-MediaPipe.suppressed.lib e dell'oggetto C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Plugins\MediaPipe\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipe\UnrealEditor-MediaPipe.suppressed.exp 1>[43/44] Link UnrealEditor-MediaPipeDemo.dll 1> Creazione della libreria C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipeDemo\UnrealEditor-MediaPipeDemo.suppressed.lib e dell'oggetto C:\Users\Francesco\Desktop\ue4-mediapipe-plugin-master\Intermediate\Build\Win64\UnrealEditor\Development\MediaPipeDemo\UnrealEditor-MediaPipeDemo.suppressed.exp 1>[44/44] WriteMetadata MediaPipeDemoEditor.target 1>Total time in Parallel executor: 139,31 seconds 1>Total execution time: 149,50 seconds ========== Ricompilazione: 1 completate, 0 non riuscite, 0 ignorate ==========

Questions where can I check the code that connects the landmark of the media pipe to the skeleton bones of the unreal engine? I want to learn that process! thanks

Good afternoon. The last source. When packing a project under Windows, I can't start the resulting application. The application window freezes and nothing else happens. Even on an empty map. In the editor, everything starts and works like a charm.

Hi! Wongfei, When running with this plugin, I encountered the following problem: The hand landmarks detection (configured by multi_hand_landmarks.pbtxt) output hand detection confidence, while the holistic landmarks detection (configured by holistic_landmarks.pbtxt or holistic_with_iris.pbtxt) does not output the hand detection confidence. How to solve this issue? Can I just modify holistic_landmarks.pbtxt to add the corresponding output without changing the code structure. Looking forward to your reply!

I have compiled the project and when I started to run the level the MediaPipe window appears black. I guess that is not connected to the webcam. Can you explain me where or in which config file can I address the webcam?

Best regards,

Mirko