AI Archives - ab https://alexandrebruffa.com/category/ai/ Thu, 01 Feb 2024 02:38:43 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.2 https://alexandrebruffa.com/wp-content/uploads/2022/04/cropped-cropped-elior-favicon-270x270-1-32x32.png AI Archives - ab https://alexandrebruffa.com/category/ai/ 32 32 Enhance Your Real-time Multiplayer Game with Generative AI (Bedrock)! https://alexandrebruffa.com/enhance-your-real-time-multiplayer-game-with-generative-ai-bedrock/ Fri, 12 Jan 2024 03:25:07 +0000 https://alexandrebruffa.com/?p=1864 Can an NPC be smart? In this post, I show how to enhance a real-time multiplayer game with AI.

The post Enhance Your Real-time Multiplayer Game with Generative AI (Bedrock)! appeared first on ab.

]]>
This article was initially published on my Medium Page.

Can an NPC be smart? What about having a real conversation with an NPC? You may have watched those incredible videos of gamers talking with AI NPCs. Can it be done with Unity3D and Amazon services? Let’s see!

Check my video to learn how to do it:

Preamble

Lately, I have published a series of articles and videos about building a real-time multiplayer game with Unity3D and Amazon GameLift.

I also entered the AI field and produced content about integrating Unity3D and AI services: ChatGPT, Dall-E, and Amazon Bedrock.

On my video about Amazon Bedrock, a clever follower left this comment:

What an outstanding idea! Technically, it would be a perfect mix between my work about Amazon GameLift and the one about Bedrock. Thanks for the idea, bro; let’s do it!

About NPCs

A non-player character (NPC), as its name indicates, is a character that is not controlled by a player but can eventually interact with the real players of a game. Therefore, every action or decision taken by the NPC is server-managed. The flow is the following:

  1. A player interacts with an NPC; the player notifies the game server.
  2. The game server processes the information and eventually connects to other services (Database, AI, etc. )
  3. The game server sends an answer to all the players, and Unity clients are updated.

General Architecture

The architecture of this project contains 3 main parts:

  1. A simple architecture of a real-time multiplayer game implicating the services Cognito, Lambda, and GameLift.
  2. A Lambda function that calls the generative AI service Amazon Bedrock.
  3. A connection between the GameLift’s game server and the Lambda function.

For the first two parts, please refer to the following articles and videos: Building a Real-Time Multiplayer Game With Unity3D and Amazon GameLift and Generative AI With Amazon Bedrock and Unity3D. This article will focus on how to invoke a Lambda function from a GameLift’s game server.

Accessing AWS Resources From GameLift

We want the game server to access another AWS resource here, Lambda. The classical way to do it would be to create a role with associated policies to allow GameLift to invoke a specific Lambda function. But this is an uncommon case and requires an extra step. The GameLift documentation says you can achieve doing the following:

  1. Set up an IAM service role for Amazon GameLift.
  2. Associate the service role with an Amazon GameLift fleet.
  3. Add code to your game server to assume the service role.

So, why this extra step? Remember that GameLift game servers are actually composed of one or various EC2 instances (fleet); therefore, we must associate a role to the fleet and also to the EC2 instance where the game session is running. After that, we can invoke our Lambda function from the game server.

It’s not so straightforward, so fasten your seatbelt, and let’s start!

The IAM role

The new role we are creating will allow GameLift to access other resources. To do so, we select GameLift as a trusted entity when creating the role.

By default, an AWS-managed policy called GameLiftGameServerGroupPolicy is attached to the role. We create a new policy to allow access to the Lambda function and attach it to the role.

Okay, so we have the role ready; we can attach it to a new GameLift fleet. But don’t create the fleet yet! We need to work on the Realtime script and dependencies first.

The Realtime Script

This is the spicy part. The role we have created is assumed by the GameLift fleet, but as mentioned before, we also need to assume the role from the Realtime script. Realtime servers use Node.js and JavaScript, so we will use the AWS SDK for JavaScript.

For some reason, I could not use the AWS SDK for Javascript version 3; it made my game server crash. I used the AWS SDK for Javascript version 2 instead, and everything worked like a charm. If you could achieve it with version 3, please reach out to me!

First, we need the game server to assume the role we have created. In the init function of the Realtime Script, we call the STS service and assume the role:

Notes:

  • We call the function assumeRole as specified in the STS documentation

As indicated in the STS documentation, the response has the following structure containing credentials:

Thanks to the credentials, we can now call our lambda function:

Notes:

  • We call the Lambda constructor thanks to the credentials returned by STS, as specified in the documentation.
  • We invoke the Lambda function (in my case npcFunction), as specified in the documentation.

To get more details about how the Realtime script works, please refer to this previous article:

The Dependencies

In the previous section, we worked with the AWS SDK for JavaScript in the Realtime script. Small but very important detail: the AWS SDK for JavaScript is not installed on the game server! Good news: we can upload the dependencies along with the Realtime script. Therefore, we will follow those steps:

Install Node.js in your local environment.

First, go to the Node.js from the official website. Select the operating system running on your local machine and install Node. Once installed, open a new terminal (or command prompt) and type “node”. You should be able to enter the Node console:

Install the AWS SDK for Javascript

In a new terminal, go to the folder containing your Realtime script and install the AWS SDK for Javascript version 2 thanks to Node’s package manager, NPM:npm install aws-sdk

Once installed, 3 new items have been created: the package.json and package-lock.json files and the node_modules folder.

Wrap the script and the dependencies

Zip the folder containing the Realtime script and the dependencies (4 files in total), and go to the GameLift console. In the script section, create a new script and select the zip file.

🚨 Houston, we have a problem: our zip file is too big! The reason is the following: AWS SDK for Javascript version 2 is not modular, so we had no choice: we have installed and zipped the whole SDK (12 MB). With the AWS SDK for Javascript version 3, we could have installed only the services we need (STS and Lambda), and this would have resulted in a lightweight zip file (2–3 MB).

Hopefully, another option is available: hosting our zip file in an Amazon S3 bucket. This is our updated architecture:

Create an S3 bucket and a related role

Create a new S3 private bucket and upload the zip file.

In the IAM console, create a new role with GameLift as a trusted entity and attach a policy that allows the private bucket and its content to be read:

Upload the script and the dependencies to GameLift

Come back to the GameLift console! In the script section, create a new script, but this time, select the zip file from S3 and the role we have just created:

GameLift can now access your Realtime script and dependencies during fleet creation through an S3 bucket.

Creating a new Fleet

When creating a new fleet, in the additional details section, indicate the role we have created at the beginning.

You will be asked for the Realtime script. Select the script you have created before, and in the Runtime configuration section, indicate the script’s path. Remember that we have zipped a folder containing the script; therefore, the path will be folder/script, in my case, GameLift/Multiplayer.js.

The Unity Client

The NPC

As I mentioned in other posts, I’m not a modeler or a 3D artist; therefore, I always look for free assets for my projects. The characters I’m using for this project, like the previous ones, are from the website Mixamo. This time, I looked for something different for the NPC and found this great Moai model on free3D.com. OBJ format is compatible with Unity, perfect!

Moai on free3D.com

The events

For this project, I added 4 new events related to the NPC:

  • PLAYER_MEETS_NPC: A player notifies the game server that he wants to interact with the NPC. If the game server allows it, the same event is sent back to all players.
  • PLAYER_TO_NPC: The player can now interact with the NPC; he can send this event along with a message.
  • NPC_TO_PLAYER: The game server sends a message back to the player.
  • PLAYER_LEAVES_NPC: The interaction ends, and all players are notified.

Final Result

This is the final result of one of the characters interacting with Moai; it looks pretty cool!

Closing Thoughts

Weeks ago, I had the opportunity to share my knowledge as a speaker at the AWS UG Peru Conference 2023. One of the attendees asked me: “What is the advantage of using GameLift over another similar service like Photon?”. My answer was the following: using GameLift, YOU build the cloud architecture of your game, and therefore, you can do absolutely what you want. You could, for example, connect your game to an AI service, a database, a notification system, etc.

And that is totally what I meant with this article: accessing AWS Resources from a GameLift game server opens the door to infinite possibilities! The implementation was a little bit complex, but hey, it is totally worth it!

Resources

Here are the resources mentioned in the article:

I also used free Mixamo characters and a model from free3D.com.

The post Enhance Your Real-time Multiplayer Game with Generative AI (Bedrock)! appeared first on ab.

]]>
Generative AI With Amazon Bedrock and Unity3D https://alexandrebruffa.com/generative-ai-with-amazon-bedrock-and-unity3d/ Thu, 19 Oct 2023 16:08:19 +0000 https://alexandrebruffa.com/?p=1855 Have you heard about Bedrock, the new generative AI service from Amazon? Let's build a cool AI chat with Unity3D and Amazon Bedrock!

The post Generative AI With Amazon Bedrock and Unity3D appeared first on ab.

]]>
A few days ago, Amazon made a big announcement: the service Bedrock was finally released.

If you missed the news, here it is: Amazon decided to enter massively into the generative AI field and invested in the company Anthropic for $4 billion in order to compete with popular AI services like ChatGPT, Dall-E, MidJourney, etc. The first proposal of Amazon is Bedrock, a fully managed AI service that offers multiple foundational models such as Claude, Jurassic, and Stable Diffusion XL, among others.

In my previous articles, I created Unity3D applications communicating with ChatGPT and Dall-E. Let’s do the same with Bedrock!

Do you prefer something more interactive instead of reading? Check my video!

General Architecture

Here is the general architecture of our application:

We use 3 Amazon services:

  • API Gateway to create, expose, and consume endpoints.
  • Lambda to receive the client’s requests and communicate with Bedrock.
  • Bedrock to generate answers based on the client’s prompts.

If you want to get further adding a login mechanism with Cognito, please refer to my previous article:

Bedrock

First, we have to enable the foundational models we want to use. By default, all the models are disabled. Go to the Bedrock console and enable the models you will use in the model access section. If you have an updated payment method, activating the models only takes a few minutes.

Model access section in the Bedrock console

If you don’t enable the models in the Bedrock console, you will receive this error in Lambda:

An error occurred (AccessDeniedException) when calling the InvokeModel operation: Your account is not authorized to invoke this API operation.

Lambda, Boto3 and Bedrock

If you follow my work, you know I’m a Python guy, so I will use the Python SDK for AWS, Boto3. Good news: Bedrock is available on Boto3; bad news: Lambda runs an old version of Boto3 in which Bedrock does not exist yet.

💡 Here is a workaround: creating a new Lambda layer with an up-to-date version of Boto3. This is how to do it:

  1. In PyCharm, create a new project and install boto3 in a new virtual environment. Boto3 and all the dependencies will be installed.
PyCharm project

2. Go to the venv folder of your project and copy the lib folder into a new folder called python. Then, zip it.

venv folder

3. Go to the Layers section of the Lambda console and create a new layer. Upload the zip file we have created before and choose the same Python version as your local Python virtual environment for the runtime.

Creating a new Layer

The Lambda Functions

We create 2 Lambda functions, one for text generation and the other one for image generation, both with the same version as previously for the runtime (In my case, Python 3.9), and we add the layer we have created before.

Adding a new layer to the Lambda function

The text generation function

For the text generation, I use the Claude model and implement it as described in the Amazon documentation.

The image generation function

For the image generation, I use the Stable Diffusion XL model and implement it, as described in the Amazon documentation.

Note that the model IDs can be found in the Bedrock documentation.

The Lambda Policy

In order to allow Lambda to invoke a Bedrock model, we create a new managed policy and add it to the Lambda functions roles:

API Gateway

In the API Gateway console, we create a new REST API with 2 resources: image and text. Each resource has a post method integrating with the Lambda functions we have created before.

API resources

When deploying the API, you will be asked to create a new stage. After the creation of the stage, the final endpoint will be shown. You will have to use this URL in the Unity client to perform the request.

API stages

The Unity Client

the Unity client allows the user to send a message to generate a text or an image as an answer:

Handling the request

First, we need to create serializable classes to embed the sending and receiving data:

Sending a request in Unity is an asynchronous process. Therefore, we create a coroutine that sends the request to the server:

Dealing with the generated image

If we ask for an image generation, the result sent by Bedrock is a base64 image. To show it on the Unity UI, we must convert it into bytes and create a Texture2D. Then, we assign the texture to the RawImage component of the UI.

Result

Here is the result with text and image generations:

Costs

Let’s check with the AWS Calculator and the Bedrock pricing sheet what our system cost would be for a very pessimistic scenario: you love the app, and you perform 2,000 requests a month. 1,000 for text generation with the Claude model and 1,000 for image generation with Stable Diffusion XL.

  • API Gateway: With 2,000 requests to our REST API, the monthly cost is 0,01 USD
  • Lambda: With 2,000 requests with an average time of 5 seconds and 512 MB of memory allocation, the monthly cost is 0.08 USD.
  • Bedrock: 1,000 text generations by month with a maximum of 300 output tokens and 20 input tokens per request would result, in the worst case, in $0.2204 + $9.804. 1,000 image generations would give us a bill of $18.

Total: The total bill for our system would be approximately 28 USD monthly. It’s quite affordable!

Final Thoughts

Using Bedrock is very intuitive, and integrating it with other Amazon services is absolutely straightforward. The fact that Bedrock is an in-house product makes the integration easier: there is no need for API keys or external requests to make it work, resulting in greater performance and security.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

Thanks for reading until the end!

Resources

Here is the resource mentioned in the article:

The post Generative AI With Amazon Bedrock and Unity3D appeared first on ab.

]]>
Detecting Unsafe Content With Unity3D and Amazon Rekognition https://alexandrebruffa.com/detecting-unsafe-content-with-unity3d-and-amazon-rekognition/ Wed, 19 Jul 2023 17:31:08 +0000 https://alexandrebruffa.com/?p=1769 Is my profile picture suggestive? In this article, I show how to detect unsafe content thanks to AWS Rekognition and Unity3D.

The post Detecting Unsafe Content With Unity3D and Amazon Rekognition appeared first on ab.

]]>
This article was initially published on my Medium Page.

If you prefer watching a video instead of reading, here you have:

My Brief Journey on TikTok

Weeks ago, I finally decided to download TikTok. I download the app and create a new account. The first thing I always do just after registering on a social network is upload a profile picture. Strangely, TikTok did not let me do it. I tried two more times the same day, but I had no success. After that, I also tried to add friends and change my username, but still no success. I waited for a couple of days, I reopened the app, and I got this message:

Permanently banned message in Spanish.

I have been permanently banned from TikTok, and I don’t know why! Then I started to think. What did I do wrong? I did not post any publications or messages. I used a valid email to register, entered my real name, and tried to upload a real picture of me. I don’t believe my email and my name are a problem. What about my profile picture? Did TikTok detect it as inappropriate? Is it because I have no hair? 👨🏻‍🦲 How to find it out?

The System I built

Then I remembered that Amazon has a service for unsafe content detection: Rekognition; let’s try it! Additionally, you may remember this previous article of mine where I created a chat in Unity showing images from Dall-E. I will reuse the interface for this article.

This is how it works:

  • The Unity client login thanks to Cognito User Pool.
  • The client uploads then a picture to S3.
  • Finally, the client calls a Lambda function that works directly with Rekognition to define if the picture contains unsafe content.

Amazon Services

➡ About Cognito, I created an authenticated access using a User Pool coupled with an Identity Pool. A policy attached to the authenticated role of the Identity Pool allows users to access S3 (PutObject) and Lambda (InvokeFunction). If you want to know in detail how to do it, please check this previous article of mine:

➡ The S3 Bucket is private, so nobody but the authenticated users can access it. I called it censorimagesbucket.

➡ I wrote the Lambda function in Python:

Notes:

  • I used boto3, the AWS SDK for Python, to connect with Rekognition.
  • I used all the Rekognition’s top-level category moderation labels. My face may be offensive in many ways; we never know.
  • I used Rekognition’s DetectModeration API thanks to the detect_moderation_labels function and the S3Object parameter.
  • I used the very cool Pythonic any function.

Don’t forget to attach to the function’s role a policy allowing access to Rekognition and S3:

Unity

The connection between Unity3D and Amazon services is done through the AWS SDK for .NET, as described in my previous article. However, the function invocation is different this time because we have to send the filename to Lambda. We do it thanks to the Payload parameter.

Result

I performed three tests: one with a picture of Bibi, my daughter’s favorite teddy bear, a picture of a girl wearing a bikini, and a picture of myself.

It’s working great! The only unsafe content (the girl in a bikini) has been filtered successfully, and the following moderation labels have been detected:

The result is very accurate, with high confidence rates.

Conclusion

Apparently, my face is not unsafe content! I still don’t know why I have been permanently banned from TikTok, but thanks to this misadventure, I had the opportunity to discover Amazon Rekognition and realize an implementation with Unity3D.

All ids and tokens shown in this article are fake or expired; if you try to use them, you will not be able to establish any connections.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

Resources

Here are all the resources mentioned in the article:

The post Detecting Unsafe Content With Unity3D and Amazon Rekognition appeared first on ab.

]]>
Generating AI Images With DALL-E, AWS, and Unity3D https://alexandrebruffa.com/generating-ai-images-with-dall-e-aws-and-unity3d/ Sat, 11 Mar 2023 17:05:06 +0000 https://alexandrebruffa.com/?p=1714 In this article, I will explain how I built a cool Unity3D chat showing Dall-E images thanks to AWS and the OpenAI API.

The post Generating AI Images With DALL-E, AWS, and Unity3D appeared first on ab.

]]>
This article was initially published on my Medium Page.

In my last article, I showed how I built a talking chat thanks to ChatGPT, AWS, and Unity3D. We will reuse the main components to create a chat showing images generated by Dall-E, the OpenAI images generation system.

Here is a video of the final result:

General Architecture

This time, it will be much more straightforward. We use API Gateway to expose our endpoint, Cognito for the authentication, and Lambda to perform the OpenAI API requests:

Notes:

  • OpenAI stores the image created by Dall-E in its repository and returns the URL.
  • The Unity client app reads the image directly from the URL.

AWS Implementation

This is our new Lambda function:

Notes:

  • We re-use the openai layer we created in my previous article, and the same environment variable openai_api_key.
  • We use the image completion endpoint of OpenAI, as specified in the documentation.
  • The default image size generation by the endpoint is 1024×1024. We will show the image inside a chat message; then, we can work with a lower resolution (512×512).

Unity implementation

Compared with my previous article, we use the same application with a few changes.

First, our Lambda function only returns the image URL, so we need to change the result data class:

Then, we need to show the image generated by the OpenAI API, so we replace the message Text component by an empty Image in the message object:

Note that the Image width is 512px, the same size we specify in the Lambda function.

And we retrieve the image thanks to the GetTexture function:

Note: Once we have received the texture, we convert it into a Sprite and assign it to the Image of the new instantiated chat message.

Costs

Let’s check with the AWS Calculator what our system cost would be for a very pessimistic scenario: you love the app and perform 50 daily, 1,500 requests monthly.

  • Cognito: This project only has one MAU (monthly active user). Cost: 0.00 USD
  • API Gateway: With 1,500 requests to our REST API, the monthly cost is 0,00 USD
  • Lambda: With 1,500 requests with an average time of 5 seconds and a 1,024 MB of memory allocation, the monthly cost is 0.00 USD; excellent!
  • OpenAI: According to the OpenAI pricing sheet, each image generated costs 0.018 USD for a 512×512 resolution. Then, for 1,500 images generated monthly, we have a cost of 27.00 USD.

Total: The total bill for our system would be 27.00 USD monthly, with a pessimistic scenario. That is expensive, but image generation is a complex process requiring high resources, so it can be understandable.

Closing Thoughts

In this article, we could figure out how to build an entire cloud architecture on AWS and how easy it is to integrate the OpenAI API with a Lambda function. We could also evaluate the cost of the whole system thanks to the AWS Calculator.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

All ids and tokens shown in this article are fake or expired; if you try to use them, you will not be able to establish any connections.

You can download the Unity package of the client app specially designed for this article.

A special thanks to Gianca Chavest for designing the fantastic illustration.

The post Generating AI Images With DALL-E, AWS, and Unity3D appeared first on ab.

]]>
I made ChatGPT talk using Unity3D and AWS https://alexandrebruffa.com/i-made-chatgpt-talk-using-unity3d-and-aws/ Sat, 11 Mar 2023 06:38:20 +0000 https://alexandrebruffa.com/?p=1710 In this article, I will explain how I built a Unity application that can communicate with ChatGPT and made it talk thanks to AWS.

The post I made ChatGPT talk using Unity3D and AWS appeared first on ab.

]]>
This article was initially published on my Medium Page.

Those last weeks, the whole Internet has been upside down. A new actor arrived and shook the AI game: ChatGPT. If you have tried it, you have probably figured out that ChatGPT is incredible: it can give you a detailed answer to almost every question, create poems or jokes, and help developers to program, among others.

After playing with the platform for a while, I remembered that years ago, I built for a client a chatbot that could talk. What about making ChatGPT talk? Is it even possible? Let’s see.

Spoil alert: I could make it! Here is a video showing the final result:

ChatGPT

The ChatGPT interface is fantastic, but even better, OpenAI has an API and an official library for Python—a gold mine for the developers. Once logged, we can figure out the following: OpenAI gives us two months of free trial usage with an $18 credit. Thanks, dudes, that’s cool.

ChatGPT account

The OpenAI console is straightforward; the only thing you have to do is generate a new API key to allow any external program to connect:

API key creation in the OpenAI console

General Architecture

Here is the general architecture of the project:

Notes:

  • Do you remember my article about the hotel platform? We will reuse the main components for authentication: we will connect a Unity app to AWS thanks to a login system with Cognito and API Gateway.
  • We will create a user pool and a new user with a name and a password in Cognito.
  • We will create an endpoint in API Gateway and an associated Authorizer so that only the users of the user pool can consume the API.
  • The Lambda function will receive the text from Unity and call the OpenAI API.
  • Once the OpenAI API has answered, we call Polly, the text-to-voice converter of AWS, which will convert the answer into a voice stream.
  • We keep the audio file in an S3 bucket and generate a pre-signed URL to restrict access to the file.

AWS implementation

S3

In the same way that my previous article, we create a private repository:

Lambda

First, we create a Lambda layer with the OpenAI library. Do you remember my previous article about making a homemade CCTV? I explain there in detail how to create a Lambda layer from a local environment, so we will follow the same method with the OpenAI library:

Now, we create our Lambda function:

Don’t forget to add the openai Layer to the function:

Inside the Lambda function, we define a new environment variable called openai_api_key with the OpenAI API key value.

Inside the function’s role, we create 2 inline policies, one for Polly, and the other for S3.

Inline policy for Polly
Inline policy for S3

And here is the function:

Notes:

  • We store the OpenAI API key in a Lambda environment variable called openai_api_key, and we call it in Lambda thanks to the os.getenv function.
  • We parse the Lambda function’s entry parameters and retrieve the message sent from Unity.
  • We call the Create completion function of the OpenAI API with the message sent from Unity. We extract the answer, as specified in the OpenAI documentation, and trim it with the strip function to avoid spaces or line breaks.
  • When we call the Create completion function, we concatenate the message with the sentence “Please give me a short answer.” to be sure that the response given by ChatGPT will not be too elaborate.
  • We call the synthesize_speech function of Polly with boto3, passing the answer as a parameter. We chose the ogg format, the best choice to work in Unity with, as I demonstrated in this previous article.
  • I chose Aria, a friendly New Zealand vocal option of Polly, but it’s up to you to choose your favorite one!
  • We keep the audio stream locally as a file thanks to the open and write functions, and we upload it to an S3 bucket thanks to the upload_file function of the boto3 library. After finishing, we remove the local file thanks to the os.remove function.
  • We generate a pre-signed URL with 1 minute of time life thanks to the generate_presigned_url function of the boto3 library, so only the user using the Unity app will be able to access the audio file.

Cognito

In the same way that in the hotel platform article, we create a new user pool in Cognito:

In the Pool, we create a new user with a name and a password:

API Gateway

In API Gateway, we create a new REST API with a POST method until our Lambda function, and we deploy it:

And we create an authorizer to allow access to the endpoint only for the Cognito users from the Pool we have created:

Unity3D Implementation

The Audio Component

Our app will be able to talk, so we need an AudioSource component in our project!

AudioSource component

Note: We let the AudioClip parameter empty; we will fill it with the audio file from S3.

The UI components

I usually detail little about the UI building of my Unity apps because I’m not a designer nor a front-end developer. Still, in this case, I found it interesting to explain how I built the client app mainly because of the complex layout of the chat.

That’s how I built the client app:

Client application layout

Notes:

  • We use a Canvas with a ScrollView (without scrollbars) to show the messages.
  • We use a vertical Content Size Fitter to resize the content of the ScrollView automatically, and a Vertical Layout Group to place the messages vertically.
  • We use a combination of horizontal and vertical Content Size Fitter and Layout Groups to resize the box containing the message.
  • We use a sliced image for the box, so all the messages will always have the same rounder corners, no matter the text size.
Sliced image in the Sprite Editor

The code

Okay, so we have a functional chat in Unity. Let’s connect it with the backend!

First of all, we login to Cognito when the application starts, and we store the token id returned by Cognito in a PlayPrefs parameter:

Please refer to my previous article for an extensive explanation of the above code.

Then, we write the functions to show and hide the user’s device keyboard:

Notes:

  • Unity work with the native keyboard of the device where the application is running. That means the keyboard will look different if you run it on iOS or Android.
  • We use the class TouchScreenKeyboard as specified in the Unity documentation and the related function Open.

Then, here is the most exciting part: we call our endpoint, and we pass the message written as a parameter:

Well, our endpoint returned a URL of the audio file, so we use it now to retrieve the file and play it:

Notes:

  • We use the function GetAudioClip of UnityWebRequestMultimedia to retrieve the audio stream in ogg format.
  • We assign the audio stream to the clip parameter of our AudioSource object.

And now, we can add the message to the chat:

Notes:

  • We instantiate the user message object or the friend message object according to the needs.
  • We use the function ForceRebuildLayoutImmediate to refresh the ScrollView content and avoid graphical bugs.
  • We set the verticalNormalizedPosition parameter of the ScrollView to 0, so the scroll position is at the bottom, and we can see the last messages.

Costs

Let’s check with the AWS Calculator what our system cost would be for a very pessimistic scenario: you love the app, and you perform 100 daily, 3,000 requests a month.

  • Cognito: We only have one MAU (monthly active user) for this project. Cost: 0.00 USD
  • API Gateway: With 3,000 requests to our REST API, the monthly cost is 0,01 USD
  • S3: Suppose that 50 KB could be the average size of an audio file; we would have 150 MB stored each month. Additionally, we would have 3,000 put requests and 3,000 get requests, leading to a monthly cost of 0.02 USD.
  • Lambda: With 3,000 requests with an average time of 3 seconds and a 1,024 MB of memory allocation, the monthly cost is 0.00 USD; excellent!
  • Polly: Polly is undoubtedly the more expensive AWS service here. Let’s suppose chatGPT answers have an average of 100 characters; the monthly bill will be 1.20 USD.
  • OpenAI: Based on the OpenAI tokenizer tool, suppose that every question we ask ChatGPT represents 15 tokens, so we use 45,000 tokens monthly. According to the OpenAI pricing, this gives us a total of 0.9 USD monthly.

Total: The total bill for our system would be 2.13 USD monthly. It’s totally affordable, taking into account that this is a very pessimistic scenario.

Closing Thoughts

In this article, we could figure out how to build an entire cloud architecture on AWS and how easy it is to integrate the OpenAI API with a Lambda function. We also had the opportunity to discover Polly, the text-to-voice service of AWS. Furthermore, we could evaluate the cost of the entire system thanks to the AWS Calculator.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

All ids and tokens shown in this article are fake or expired; if you try to use them, you will not be able to establish any connections.

You can download the Unity package of the client app specially designed for this article.

A special thanks to Gianca Chavest for designing the amazing illustration.

The post I made ChatGPT talk using Unity3D and AWS appeared first on ab.

]]>