A sample project combining Epic Games' MetaHuman digital characters with Amazon Polly text-to-speech.

Overview

Amazon Polly & MetaHumans Sample Project

A sample project combining Epic Games' MetaHuman digital characters with Amazon Polly text-to-speech.

This Unreal Engine sample project demonstrates how to bring Epic Games' MetaHuman digital characters to life using the Amazon Polly text-to-speech service from AWS. Use this project as a starting point for creating your own Unreal Engine applications that leverage Amazon Polly to give voice to your MetaHumans using one of 16 different English language voices spanning 5 dialects. Or extend this project to use any of Polly's 60+ voices covering 20+ languages and 13+ dialects.

Polly.MetaHuman.1080p.mov

With Amazon Polly, you only pay for what you use. You are charged based on the number of characters of text that you convert either to speech audio or to speech metadata. In addition, you can cache and replay Amazon Polly’s generated speech at no additional cost. For full pricing details, see Amazon Polly Pricing.

Contents

Quick Start

🛑 Before proceeding you must have Unreal Engine 4.26 or later installed as well as the Microsoft Visual Studio development tools required for UE4 C++ development (Windows) or the Xcode development tools (Mac). If you need help with these setup steps, refer to the Unreal Engine 4 documentation, especially "Setting Up Visual Studio for Unreal Engine".

1. Create AWS credentials for the project

In order for this Unreal Engine project to interact with the Amazon Polly service, you must provide it with AWS credentials that allow access to that service. The easiest way to generate these credentials is to create a new AWS Identity Access & Management (IAM) user in your AWS account.

Create a new IAM user and assign to it the permissions policy named "AmazonPollyReadOnlyAccess". Although the name you give this user is not important, we suggest naming it "MetaHumans Sample" or something equally distinctive. Be sure to save the Access Key ID and the Secret Access Key that are generated during the user creation process. You'll need them later.

💡 Tip: For more help, see "Creating an IAM user in your AWS account" in the AWS IAM documentation.

2. Install and configure the AWS Command Line Interface

You will need to configure your local computer to communicate with AWS services using the credentials you created above. The easiest way to do this is to install and configure the AWS Command Line Interface (AWS CLI).

Install the AWS CLI to your local computer following these instructions.

Use the aws configure command to create a default profile for the AWS CLI. Be sure to use the Access Key ID and Secret Access Key values you saved above.

💡 Tip: For more help, see "Configuration basics" in the AWS CLI documentation.

3. Compile the Polly C++ SDK

🛑 This next step requires cmake. If you don't already have cmake installed, you can download it here. After you download cmake, launch cmake and click 'Tools' -> 'How To Install For Command Line Use' and follow one of the instructions. E.g. for Mac - One may add CMake to the PATH: PATH="/Applications/CMake.app/Contents/bin":"$PATH"

This project makes use of the C++ Polly API – a part of the AWS SDK for C++ – to communicate with the Polly service. We've provided scripts to automatically download and compile the appropriate binaries for you. Run one of the following scripts:

Windows: Source/AmazonPollyMetaHuman/ThirdParty/AwsSdk/BuildAwsSdkWin64.bat

Mac: Source/AmazonPollyMetaHuman/ThirdParty/AwsSdk/BuildAwsSdkMac.sh

4. Open the Unreal Engine project

Open the project by double-clicking on the AmazonPollyMetaHuman.uproject file.

Click "Yes" on the dialog that appears.

Module compile prompt

5. Run the project

To try out the project, simply click the "Play" button in the Unreal Engine editor. The MetaHuman will come alive using speech and lip sync generated by Amazon Polly. After the MetaHuman stops speaking you can enter your own custom speech text into the on-screen text field.

Play button

⚠️ Wait until the "Compiling Shaders" process completes before running this project for the first time.

🛠 Troubleshooting: This project includes extensive error messaging that can help you debug common problems. If the project doesn't work properly, open the Output Log tab in the Unreal Engine editor and look for error messages.

Developer Guide

This repository includes a full Developer Guide which describes the project's architecture and explains how to customize the project with your own MetaHuman characters.

Getting Help

If you have questions as you explore this sample project post them to the Issues section of this repository. To report bugs, request new features, or contribute to this open source project see CONTRIBUTING.md.

Security

See CONTRIBUTING for more information.

License

This sample code is licensed under the MIT-0 License. See the LICENSE file.

Comments
  • Cannot build AmazonPollyMetahuman (and AmazonPolly is missing?)

    Cannot build AmazonPollyMetahuman (and AmazonPolly is missing?)

    Hello, first let me thank you a lot for this project, it really is interesting.

    I have followed the quick-guide to the letter, but I get 4 errors at the end with the .bat file.

    When I try to open the .uproject file, I get:

    Missing AmazonPollyMetaHuman Modules

    The following modules are missing or built with a different engine version:

    AmazonPollyMetaHuman

    Would you like to rebuild them now?

    After trying to compile, the error tells me to compile manually, which i have tried with both VS2022 and VS2019, but I still get errors.

    I have tried a few times the zip downloaded, but I get different sizes each time (and can only be extracted with 7Zip). Git results in stable size.

    I am using W10, UE4..26.2, VS2022 and VS20219.

    I'm at my wits' end. Could anyone help here, please?

    opened by DrinkSlurp 10
  • USpeechComponent crash on mac

    USpeechComponent crash on mac

    System setup

    machine: M1 mac pro build aws-sdk-cpp: cmake "$SDK_REPO_DIR" -DCMAKE_INSTALL_PREFIX="$SDK_INSTALL_DIR" -DBUILD_ONLY="polly" -DCMAKE_SYSTEM_PROCESSOR=x86_64 -DCMAKE_OSX_ARCHITECTURES=x86_64 (have to switch off -DCUSTOM_MEMORY_MANAGEMENT=ON to PASS the build) UE version: 4.27.2


    how I trigger the bug

    It successfully start the UE, and I can see the metahuman and the scene, and I can do "Play", there is a text box to input text. When I input some text, no speech is generated. Then I end the "Play", it cratshes:


    log

    LoginId:1df5aecc93402662665fcead18debcbe EpicAccountId:c1550b314dfc4ca8b5464c64c553cfc6

    Caught signal

    Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const Address = 0x1c889e19d (filename not found) [in libaws-cpp-sdk-core.dylib] Aws::Client::AWSClient::MakeRequestWithUnparsedResponse(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const Address = 0x1c88a3c55 (filename not found) [in libaws-cpp-sdk-core.dylib] Aws::Polly::PollyClient::SynthesizeSpeech(Aws::Polly::Model::SynthesizeSpeechRequest const&) const Address = 0x1c7d29b5e (filename not found) [in libaws-cpp-sdk-polly.dylib] PollyClient::SynthesizeSpeech(Aws::Polly::Model::SynthesizeSpeechRequest const&) Address = 0x1c7b0ae28 [/Users/test/repo/amazon-polly-metahumans/Source/AmazonPollyMetaHuman/Private/PollyClient.cpp, line 36] [in UE4Editor-AmazonPollyMetaHuman.dylib] USpeechComponent::SynthesizeAudio(FString const&, EVoiceId) Address = 0x1c7b0c113 (filename not found) [in UE4Editor-AmazonPollyMetaHuman.dylib] USpeechComponent::GenerateSpeechSync(FString, EVoiceId) Address = 0x1c7b0bf8f [/Users/test/repo/amazon-polly-metahumans/Source/AmazonPollyMetaHuman/Private/SpeechComponent.cpp, line 118] [in UE4Editor-AmazonPollyMetaHuman.dylib] UE4Function_Private::TFunctionRefCaller<FGenerateSpeechAction::FGenerateSpeechAction(FLatentActionInfo const&, USpeechComponent*, FString const&, EVoiceId const&, EGenerateSpeechExecPins&)::$_0, void ()>::Call(void*) Address = 0x1c7b0a711 [/Users/Shared/Epic Games/UE_4.27/Engine/Source/Runtime/Core/Public/Templates/Function.h, line 548] [in UE4Editor-AmazonPollyMetaHuman.dylib] TGraphTask::ExecuteTask(TArray<FBaseGraphTask*, TSizedDefaultAllocator<32> >&, ENamedThreads::Type) Address = 0x10cfb5d3c (filename not found) [in UE4Editor-Core.dylib] FTaskThreadAnyThread::ProcessTasks() Address = 0x10cf987ca (filename not found) [in UE4Editor-Core.dylib] FTaskThreadAnyThread::ProcessTasksUntilQuit(int) Address = 0x10cf97818 (filename not found) [in UE4Editor-Core.dylib] FTaskThreadAnyThread::Run() Address = 0x10cf976c9 (filename not found) [in UE4Editor-Core.dylib] FRunnableThreadPThread::Run() Address = 0x10d0c9a18 (filename not found) [in UE4Editor-Core.dylib] FRunnableThreadPThread::_ThreadProc(void*) Address = 0x10d03ab84 (filename not found) [in UE4Editor-Core.dylib] _pthread_start Address = 0x7ff81a5194e1 (filename not found) [in libsystem_pthread.dylib] thread_start Address = 0x7ff81a514f6b (filename not found) [in libsystem_pthread.dylib]

    opened by KingStorm 9
  • The speech component crashes

    The speech component crashes

    The editor crashes as soon as the Speak function is executed. In debug mode, it hits a breakpoint, and when I press "continue" in VS, it resumes. Please help. I'm really close

    opened by Ishu07 7
  • Aws sdk build - cmake error

    Aws sdk build - cmake error

    Hello, i have some trouble with building aws. I have done everything in readme,

    1. Create AWS credentials for the project DONE
    2. Install and configure the AWS Command Line Interface DONE
    3. Compile the Polly C++ SDK. (Cmake installed, path set) But when i start AmazonPollyMetaHuman/ThirdParty/AwsSdk/BuildAwsSdkWin64.bat it getting that CMakelist is missing.
    CMake Warning:
      Ignoring extra path from command line:
    
       ".."
    
    D:\amazon-polly-metahumans-main\Source\AmazonPollyMetaHuman\ThirdParty\AwsSdk\BuildAwsSdkWin64.bat
    **********************************************************************
    ** Visual Studio 2019 Developer Command Prompt v16.6.2
    ** Copyright (c) 2020 Microsoft Corporation
    **********************************************************************
    CMake Warning:
      Ignoring extra path from command line:
    
       ".."
    
    
    CMake Error: The source directory "D:/amazon-polly-metahumans-main/Source/AmazonPollyMetaHuman/ThirdParty/AwsSdk/aws-sdk-cpp" does not appear to contain CMakeLists.txt.
    Specify --help for usage, or press the help button on the CMake GUI.
    

    Is it a bug or im missing some configuration with cmake?

    Windows 10, visual 2019, cmake tested 3.24.2, 3.23.3 and 3.18.0

    Have nice day, O.P.

    opened by OndrejPliva 1
  • Error out when starting Unreal

    Error out when starting Unreal

    Hello,

    I have succesfull done steps 1-3 I have run the bat file and it has created a Win64 folder with all the files.
    I then downloaded the git project. When double clicking the uproject i get

    The following modules are missing or built with a different engine version AmazonPollyMetaHuman would you like to rebuild now

    click yes and then it comes up with an error it can not be compiled. Do i need to put the win64 folder somewhere... is there any other idea on what can be causing this.

    opened by TheShoes 1
  • Enable easier custom MetaHuman use

    Enable easier custom MetaHuman use

    Issue:

    https://github.com/aws-samples/amazon-polly-metahumans/issues/6

    Description of changes:

    The previous Content folder organization made it very difficult to successfully incorporate new MetaHumans into this sample project. This resulted in a number of users reporting issues. This PR addresses these problems in two ways:

    1. The Content folder has been reorganized to better accommodate custom MetaHuman import.
    2. The "Developer Guide" has been updated with more detailed instructions on how to import and use custom MetaHumans.

    Additionally, this PR fixes an error in one of the documentation images.


    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by Krxtopher 1
  • build polly c++ sdk Failed on mac

    build polly c++ sdk Failed on mac

    Build failed on mac:

    [----------] Global test environment tear-down [==========] 475 tests from 76 test cases ran. (7633 ms total) [ PASSED ] 475 tests. /repo/aws_downloads/aws-sdk-cpp/aws-cpp-sdk-core-tests/RunTests.cpp:40: Failure Expected equality of these values: memorySystem.GetCurrentOutstandingAllocations() Which is: 16 0ULL Which is: 0 /repo/aws_downloads/aws-sdk-cpp/aws-cpp-sdk-core-tests/RunTests.cpp:40: Failure Expected equality of these values: memorySystem.GetCurrentBytesAllocated() Which is: 1440 0ULL Which is: 0 /repo/aws_downloads/aws-sdk-cpp/aws-cpp-sdk-core-tests/RunTests.cpp:40: Failure Value of: memorySystem.IsClean() Actual: false Expected: true aws-cpp-sdk-core-tests(10932,0x102cac580) malloc: *** error for object 0x6000038f8858: pointer being freed was not allocated aws-cpp-sdk-core-tests(10932,0x102cac580) malloc: *** set a breakpoint in malloc_error_break to debug /bin/sh: line 1: 10932 Abort trap: 6 ./aws-cpp-sdk-core-tests --gtest_shuffle --gtest_repeat=3 make[2]: *** [aws-cpp-sdk-core-tests/aws-cpp-sdk-core-tests] Error 134 make[2]: *** Deleting file `aws-cpp-sdk-core-tests/aws-cpp-sdk-core-tests' make[1]: *** [aws-cpp-sdk-core-tests/CMakeFiles/aws-cpp-sdk-core-tests.dir/all] Error 2 make: *** [all] Error 2 mkdir: /repo/aws_downloads/Mac: File exists cp: /repo/aws_downloads/aws-sdk-cpp/_install/include: No such file or directory ls: /repo/aws_downloads/aws-sdk-cpp/_install/lib/*.dylib: No such file or directory

    cmake version 3.24.0-rc2

    CMake suite maintained and supported by Kitware (kitware.com/cmake).

    opened by KingStorm 1
  • hardcode aws sdk to 1.9.0

    hardcode aws sdk to 1.9.0

    Description of changes:

    Modify build scripts to clone the aws sdk at version 1.9.0.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by cwalkere 0
  • update windows bat script

    update windows bat script

    Description of changes: Allow the windows install script to discover the VS 2022 location, prompt the user for vs install path if it's not found in the default location.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by cwalkere 0
  • Feature/documentation review

    Feature/documentation review

    Description of changes: This purpose of this PR is to provide a mechanism for team members to review the documentation for this project, specifically README.md and Documentation/DeveloperGuide.md.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by kris-work 0
  • initial commit to setup repo

    initial commit to setup repo

    Description of changes: Initial commit to setup github repo.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by cwalkere 0
  • The polly sdk not working correctly with some unicode  language ?

    The polly sdk not working correctly with some unicode language ?

    I download the repo and built all SDKs. then add more voiceIds (like Zhiyu for chinse), after the pixel streaming started, when enter some Chinese text in the text box, the polly can only speak few characters. Same for Japanese's voice id. But for English works. Did I missing some configurations ?

    opened by eggcaker 0
  • error when speaking

    error when speaking

    Any idea why i get this error on start LogPollyMsg: Error: Polly failed to generate audio file. Error: Invalid DNS Label found in URI host LogPollyMsg: Error: Failed to start speech. GenerateSpeech must be invoked before StartSpeech.

    also get it when i type a message in the box and click speak

    opened by TheShoes 2
  • Awssdk build error

    Awssdk build error

    hirdParty/AwsSdk/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/posix/system_info.c:196:34: error: this old-style function definition is not preceded by a prototype [-Werror,-Wstrict-prototypes] const char *s_get_executable_path() { ^ 1 error generated. make[2]: *** [crt/aws-crt-cpp/crt/aws-c-common/CMakeFiles/aws-c-common.dir/source/posix/system_info.c.o] Error 1 make[1]: *** [crt/aws-crt-cpp/crt/aws-c-common/CMakeFiles/aws-c-common.dir/all] Error 2 make: *** [all] Error 2

    opened by jinfagang 2
  • Not able to produce the animation on adding new metahuman

    Not able to produce the animation on adding new metahuman

    Hello, first of all, thanks for all making it easy for Polly's integration. I am successfully able to run the sample given, now I am trying to add another metahuman as well and followed the steps given in the developer's guide, I am successfully able to import the metahuman and add all the speech blueprints with sequence, now on the next steps I am stuck.

    I have some questions 1> after adding the Speech_AnimBP to the face component, the facial animation does not work but the audio plays back successfully. 2.> similar to above but this time on the body component I cant add the Bodyidle animation, even dragging n dropping the Bodyidle animation into anim class doesn't work,

    Now I observed that if the face mesh of the new metahuman is changed back to ada_Facemesh with the Speech_AnimBP, it works fine. So basically animation is only for ada_FaceMesh (Ada)?

    I tried to change the face mesh directly by editing the Speech_AnimBP, but was unable, as it doesn't show other face mesh other than Ada.

    Please guide. @cwalkere

    opened by OnlinePage 16
Owner
AWS Samples
AWS Samples
Amazon Lumberyard is a free AAA game engine deeply integrated with AWS and Twitch – with full source.

Amazon Lumberyard Amazon Lumberyard is a free, AAA game engine that gives you the tools you need to create high quality games. Deeply integrated with

Amazon Web Services 1.9k Sep 24, 2022
This is a list of different open-source video games and commercial video games open-source remakes.

This is a list of different open-source video games and commercial video games open-source remakes.

Ivan Bobev 108 Sep 27, 2022
Project DELTA - An open-source trainer built on the Void Engine for Toby Fox's games and their spin-offs.

Project DELTA v3 Project DELTA - An open-source, modular mod menu for Toby Fox's games and their spin-offs. Important note to Grossley: Yes, it is out

Archie 7 Apr 20, 2022
MZ-700 Original Games Project

MZ-700 Game Projects ある日唐突に MZ-700 で動くゲームを作ってみたくなったので, 作りました! One day I suddenly wanted to make a game that runs on the MZ-700. Nowadays, there are Z8

null 12 May 21, 2022
This repository is about a school project about games with physics, written in C++.

Eight Ball Pool ?? A game about playing physics-based eight ball pool. ?? Table of Contents About Getting Started Usage Built Using About the creators

Valeri Ivanov 17 Sep 3, 2022
A simple text-based adventure game

The Lighthouse of Doom This repository contain a simple text-based adventure game, implemented twice, once in portable C, and once in Z80 assembly lan

Steve Kemp 54 Aug 18, 2022
A text-based space adventure role-playing game developed by a team of 5.

SpaceAdventureRPG - Text-Based Game A text-based space adventure role-playing game developed by a team of 5. Module: 4007CEM, Class: B, Group: 3 Contr

Jonas Djondo 1 Nov 18, 2021
Open-source, cross-platform, C++ game engine for creating 2D/3D games.

GamePlay v3.0.0 GamePlay is an open-source, cross-platform, C++ game framework/engine for creating 2D/3D mobile and desktop games. Website Wiki API De

gameplay3d 3.8k Oct 2, 2022
A C math library targeted at games

Kazmath Kazmath is a simple 3D maths library written in C. It was initially coded for use in my book, Beginning OpenGL Game Programming - Second editi

Luke Benstead 505 Sep 24, 2022
3D games console based on RP2040 and iCE40 UP5k

PicoStation 3D This is an unfinished, untested project to develop a 3D games console based on an RP2040 microcontroller and an iCE40 UP5k FPGA. Quick

Luke Wren 37 Sep 12, 2022
A set of libraries and tools to make MSX games using the C programming language.

ubox MSX lib This is a set of libraries and tools to make MSX games using the C programming language. There are three main components: ubox: thin wrap

Juan J. Martínez 42 May 30, 2022
TIC-80 is a fantasy computer for making, playing and sharing tiny games.

TIC-80 is a fantasy computer for making, playing and sharing tiny games.

Vadim Grigoruk 3.6k Sep 30, 2022
Enfusion Artifical Intelligence for DayZ and future Bohemia Interactive games.

Enfusion AI Project (eAI) This mod adds headless player units under the control of a script on the server. Although the script is very rudimentary now

William Bowers 58 Sep 23, 2022
Game engine behind Sea Dogs, Pirates of the Caribbean and Age of Pirates games.

Game engine behind Sea Dogs, Pirates of the Caribbean and Age of Pirates games.

Storm Devs 659 Oct 5, 2022
null 5.1k Oct 1, 2022
Cute Framework (CF for short) is the cutest framework available for making 2D games in C/C++

Cute Framework (CF for short) is the cutest framework available for making 2D games in C/C++. CF comprises of different features, where the various features avoid inter-dependencies. In this way using CF is about picking and choosing which pieces are needed for your game

null 249 Sep 20, 2022
Modding (hacking) il2cpp games by classes, methods, fields names.

ByNameModding Modding (hacking) il2cpp games by classes, methods, fields names. Status: Ready to use Why did I do it 1. In order not to update the off

null 88 Sep 24, 2022
OGRE is a scene-oriented, flexible 3D engine written in C++ designed to make it easier and more intuitive for developers to produce games and demos utilising 3D hardware.

OGRE (Object-Oriented Graphics Rendering Engine) is a scene-oriented, flexible 3D engine written in C++ designed to make it easier and more intuitive for developers to produce games and demos utilising 3D hardware. The class library abstracts all the details of using the underlying system libraries like Direct3D and OpenGL and provides an interface based on world objects and other intuitive classes.

null 3k Oct 5, 2022
Insomniac games cache simulation tool plugin for UE4

Insomniac Games CacheSim plugin for UE4 This plugin for Unreal Engine 4 lets you use the Insomniac Games Cache Simulation tool to detect cache misses

Toni Rebollo Berná 29 Aug 27, 2022