You are talented with Unity and game creation in general, and it’s time for you to move up a gear by creating a real-time multiplayer game, but you have no idea how to do it, and you know very little about cloud computing. Don’t worry, we will see how to do it step by step.

This article is based on the excellent tutorial by Chris Blackwell. Please take a look at it!

Disclaimer: This is not an article about game design. I will focus on the technical aspect of creating a real-time multiplayer game.

Here is the final result:

Preamble

When one starts to work on a real-time multiplayer game, a lot of questions come to mind:

How can I match the players in one or multiple rooms (game sessions)? How can my players communicate together? How can I achieve to have a very low latency for my game?

Short answer: Amazon GameLift.

Amazon GameLift is a managed service for multiplayer games with the following characteristics:

• It deploys a set of EC2 instances (fleet) for you, so you won’t have to worry about mounting EC2 instances, securing them, etc.
• It operates the EC2 instances for you: you will never connect to the instances nor install anything. A game server is automatically set up when you run a new GameLift fleet.
• It scales the service for you. No matter how many users your game is played by, GameLift automatically scales up or down according to the demand.
• It connects the players through the game server, creating game and player sessions.

The Architecture

We can achieve building a real-time multiplayer game with few components. Here is our architecture:

This is how it works:

• The client (Unity application on a mobile device) joins the game by calling a Lambda function.
• The Lambda function indicates GameLift to create a new game session if no game session is available and to create a new player session. The function returns the necessary information to connect to the game server (IP, port, credentials, etc.).
• The client establishes a new connection with the game server, and the user is ready to play.

Lambda

The Lambda function (Python 3.10) will work directly with GameLift and will have the following behavior:

The origin tutorial exposed a Node.js function, but since I’m a Python guy, I rewrote the function in Python. You can find the whole function in the last section of this article.

Also, the Lambda function needs the following permissions to access the GameLift service: CreateGameSession, DescribeGameSessionDetails, CreatePlayerSession, and SearchGameSessions. You can create a new manage IAM policy with those permissions and add it to the Lambda function role.

You can find the policy at the end of this article.

Cognito

In previous articles of mine, I performed user authentication through IAM users or Cognito User Pools. We don’t need user authentication this time: we will use guest access to avoid any restrictions; everyone who downloads our game can play immediately.

In this opportunity, the Unity client invokes a Lambda function, and the best way to give access to AWS resources without credentials (guest access) is through a Cognito Identity Pool. A Cognito Identity Pool allows both authenticated and guest users, so if you want to enable some authentication feature in the future (for example, linking users’ Facebook or Twitter accounts to save game progress), you could do it easily.

We must attach the necessary permissions (Lambda Invoke) to the Identity Pool we created. You can edit the guest role associated with the Identity Pool and create a new policy to achieve it. You can find the policy at the end of this article.

GameLift

Before running a new GameLift fleet, we need to elaborate a script that will be attached to the fleet. The first step is defining the behavior of the game server. This is how a game session will be managed:

Based on the above flow, we can write and upload a JavaScript script to GameLift. Then, we can create a new fleet and include our script during the fleet creation. Check the end of this article for the script.

Before configuring the auto-scaling of your fleet, you need first set limits for it. Imagine that your game has an unexpected success; you don’t want to run out of budget in a few days! In the scaling capacity section, you can set a minimum and a maximum of instances for your fleet:

Now, let’s create an auto-scaling policy. An EC2 instance usually takes one or two minutes to start and can inconvenience the players joining the game. Furthermore, it’s better to start an instance preventively before reaching the player limit:

You can see how many available instances you have for your fleet here and request a limit increase here if necessary. In my case, I had only one available instance for my fleet (c4.large); the support team gently increased the limit to 15.

Unity

The plugins

We will integrate the Unity client directly with a Lambda function, so we need some AWS classes and functions. In the original tutorial, Chris Blackwell recommends using the AWS Mobile SDK for Unity to achieve it. Unfortunately, the AWS Mobile SDK for Unity has been deprecated, and Amazon recommends now using the AWS SDK for .NET. Let’s see.

In previous articles, I recommended downloading the AWS packages on NuGet.org, the official package manager for .NET. Although it could work, the Amazon documentation recommends doing it here for Unity. Since Unity uses .NET Standard 2.1, download and unzips the .NET Standard 2.0 zip file (it also supports 2.1).

Then, copy the following DLL files to the Plugins folder of your Unity project: AWSSDK.Core, AWSSDK.Lambda, AWSSDK.CognitoIdentity, and AWSSDK.SecurityToken. The Lambda function call is asynchronous, so you also need the Microsoft AsyncInterfaces package.

Then, we will need an extra plugin to connect to the game server: the GameLift Realtime Client SDK. Go to the GameLift page and download it. Once unzipped, you can figure out this is a whole .NET project. Open the project with Visual Studio and compile it. Five DLL files will be generated: GameLiftRealtimeClientSdkNet45.dll, Google.Protobuf.dll, log4net.dll, SuperSocket.ClientEngine.dll, and WebSocket4Net.dll. Move those DLL files to the Plugins folder of your Unity project.

Characters and animations

Since I’m not a designer or a 3D modeler, I had to look for a third party who could provide the animations I needed for my Unity project. I found out that Mixamo has a lot of fantastic free characters and animations.

I tried it by downloading the character Remy as an FBX file for Unity and moved it to my Unity project.

It looks good! After that, I downloaded multiple animations to give life to Remy. The animations should be downloaded without skin. We indicate the animation in the editor to use the character’s skin we previously downloaded.

Then I defined the animations flow inside a new Animator. It looks great!

Building the App

In this section, I will give you some hints about how I built the Unity app. Don’t worry about the code and the project hierarchy! The entire Unity project is available for download at the end of this article.

Character movements

Our character can run, walk back, and rotate. We use the function OnAnimatorMove to have total control over it.

Sending data to the game server

When one starts to build the client, the following question comes to mind:

When should I send data to the game server?

The game server needs to be aware of all actions a player performs to inform the other players, but sending too much information would saturate the game server and burn resources. Remember: don’t send data every frame; otherwise, your game server will die!

I found that data can be sent in those cases: when the player starts and ends any movement (running and walking back), any rotation (turning left and right), and any action (punching, etc.).

If you have played online, you may have experienced the famous “lag,” meaning that what you see is delayed over reality. That can be caused by a slow internet connection or a high-latency game server, but it can also mean the server data must be updated. That is why this is important to “fix” the data occasionally, sending the server updated data. I decided to do it every second in the movement functions.

Player data

For the basic game I made, I used the following data:

Note that the PlayerCoord class has four components: I used this for position or rotation (quaternions).

Costs

Let’s consider the following case: you published a successful FPS game with the following characteristics:

• Every game session has a limit of 30 players.
• In the peak hour, your game receives approximately 40 concurrent connections from players located in the United States.
• Your GameLift fleet uses c4.large Linux Spot instances, and each instance handles a single game server.

Based on the Steam statistics, we can speculate that roughly 35% of the day, a single game server is used versus two games servers in the peak hour:

The pricing sheet of Amazon GameLift indicates that a c4.large Linux Spot instance costs $0.07848 per hour, $56.51 per month. We can deduce the following calculation:

Total cost per month = $56.51 x 1.35 ≈ $76.28

GameLift is expensive, but hey, it is worth it! You save time and money instead of implementing your solution and possibly needing help with scaling issues. If you are using GameLift for your game, you should find a way to monetize it (ads, in-app purchases, etc.).

Here are the other costs:

• Unity: As always, Unity is free. The personal plan is a good fit unless you need specific advanced features for your game.
• Cognito: There is no charge for Cognito Identity Pools. Awesome!
• Lambda: The Lambda function is called only once when a new player joins the game. Estimating an exact number of monthly connections is complex, but let’s say 100,000. With 128 MB allocated memory, and each request duration of 500 ms, the cost would be $0.12.

Total: Our real-time multiplayer game bill would be approximately $77 per month.

Final Thoughts

Building a real-time multiplayer game is relatively easy! Amazon GameLift provides all the infrastructure required for this type of application and allows game developers to focus on the visual part.

You are free to use all the resources I provided in this article. Please ping me if you use them to build your game; I would be delighted to see the result!

Resources

Here are all the resources mentioned in the article:

• The Lambda function in Python
• The Lambda policy attached to the Lambda role
• The Cognito policy attached to the Cognito role
• The GameLift Script in JavaScript
• The Unity package of the game

I also used free Mixamo characters and PolyHeaven-free textures.

The post Building a Real-Time Multiplayer Game With Unity3D and GameLift appeared first on ab.

You may be wondering why we write about Python on the LearnSQL.com blog? Well … They are just a very good couple. Both languages are basic tools in such fields as data science and data visualization. They complement each other perfectly and strengthen their individual capabilities.

That’s why I decided to show you how much you can gain by learning the basics of Python. I know that many people reading this article are still on the path of learning SQL and do not currently have the space to learn another skill. It’s usually best to master one skill – in this case, SQL – and then move on to the next challenge.

However, it is worth planning your development path in advance. That’s why I decided to describe how using SQL with Python will help you spread your wings and boost your career.

The post Unleash the Full Potential of Your Data: Enhancing SQL with Python appeared first on ab.

Most people probably think Python is only for programmers, that this is a skill reserved for a small group. This is not true; Python can be useful to everyone.

In this article, I’ll tell you why you should start using Python for marketing – even if you’ve had nothing to do with coding before.

Let’s Talk About Python

You’ve surely heard about Python, but let’s have a brief recap before we move on to how it can make marketing activities faster and easier.

The post 4 Ways Python Can Boost Your Marketing Activities appeared first on ab.

What Is Python?

There are many benefits to learning Python programming. But what is Python? It is a high-level programming language, designed to be easy to read, write, and understand, with a focus on expressing complex concepts concisely and intuitively.

Python is also a general-purpose programming language. Developers use it for a wide range of applications and tasks, as opposed to domain-specific languages focused on specific use cases or fields. Python is used for web development, software development, machine learning, and scripting, among others.

You may not believe it, but Python is not so young! It was conceived in the late 80s by Guido Van Rossum, a Dutch programmer, who released the first version of Python in 1991. Python was originally used as a scripting language, but it is living a second life with the growth of the artificial intelligence and machine learning fields.

The post The Benefits of Learning Python for Business Professionals appeared first on ab.

Maybe you’ve always loved experimenting with computers and you’d like to learn to write your own computer programs. Or maybe you’ve tried one or two other careers and you’re looking for something that’s interesting, challenging, and in demand. Or perhaps you’re working in another area of IT and you want to add Python to your toolkit. This article will explain how to make the transition to a professional Python developer.

A Brief Overview of Python

Python is a general-purpose programming language: almost all domains and applications can use Python. You can build a website with it, use it to train machine learning models, execute complex financial calculations, or write quick automation scripts with it; there is no limit.

The post Python Developer Career Path appeared first on ab.

Learning almost any new skill requires not only gaining knowledge but experience. And this is what we acquire through practice.

This article will help anyone who has recently started learning Python or who already knows the basics of Python but cannot progress to the next level. Here are the best ways to practice Python.

What Is Python?

Python is a general-purpose programming language which means it is used for a wide range of domains and applications, unlike domain-specific languages that are designed for a specific task or application (i.e. SQL for databases).

The post Different Ways to Practice Python appeared first on ab.

In my last article, I showed how I built a talking chat thanks to ChatGPT, AWS, and Unity3D. We will reuse the main components to create a chat showing images generated by Dall-E, the OpenAI images generation system.

Here is a video of the final result:

General Architecture

This time, it will be much more straightforward. We use API Gateway to expose our endpoint, Cognito for the authentication, and Lambda to perform the OpenAI API requests:

Notes:

• OpenAI stores the image created by Dall-E in its repository and returns the URL.
• The Unity client app reads the image directly from the URL.

AWS Implementation

This is our new Lambda function:

Notes:

• We re-use the openai layer we created in my previous article, and the same environment variable openai_api_key.
• We use the image completion endpoint of OpenAI, as specified in the documentation.
• The default image size generation by the endpoint is 1024×1024. We will show the image inside a chat message; then, we can work with a lower resolution (512×512).

Unity implementation

Compared with my previous article, we use the same application with a few changes.

First, our Lambda function only returns the image URL, so we need to change the result data class:

Then, we need to show the image generated by the OpenAI API, so we replace the message Text component by an empty Image in the message object:

Note that the Image width is 512px, the same size we specify in the Lambda function.

And we retrieve the image thanks to the GetTexture function:

Note: Once we have received the texture, we convert it into a Sprite and assign it to the Image of the new instantiated chat message.

Costs

Let’s check with the AWS Calculator what our system cost would be for a very pessimistic scenario: you love the app and perform 50 daily, 1,500 requests monthly.

• Cognito: This project only has one MAU (monthly active user). Cost: 0.00 USD
• API Gateway: With 1,500 requests to our REST API, the monthly cost is 0,00 USD
• Lambda: With 1,500 requests with an average time of 5 seconds and a 1,024 MB of memory allocation, the monthly cost is 0.00 USD; excellent!
• OpenAI: According to the OpenAI pricing sheet, each image generated costs 0.018 USD for a 512×512 resolution. Then, for 1,500 images generated monthly, we have a cost of 27.00 USD.

Total: The total bill for our system would be 27.00 USD monthly, with a pessimistic scenario. That is expensive, but image generation is a complex process requiring high resources, so it can be understandable.

Closing Thoughts

In this article, we could figure out how to build an entire cloud architecture on AWS and how easy it is to integrate the OpenAI API with a Lambda function. We could also evaluate the cost of the whole system thanks to the AWS Calculator.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

All ids and tokens shown in this article are fake or expired; if you try to use them, you will not be able to establish any connections.

You can download the Unity package of the client app specially designed for this article.

A special thanks to Gianca Chavest for designing the fantastic illustration.

The post Generating AI Images With DALL-E, AWS, and Unity3D appeared first on ab.

Those last weeks, the whole Internet has been upside down. A new actor arrived and shook the AI game: ChatGPT. If you have tried it, you have probably figured out that ChatGPT is incredible: it can give you a detailed answer to almost every question, create poems or jokes, and help developers to program, among others.

After playing with the platform for a while, I remembered that years ago, I built for a client a chatbot that could talk. What about making ChatGPT talk? Is it even possible? Let’s see.

Spoil alert: I could make it! Here is a video showing the final result:

ChatGPT

The ChatGPT interface is fantastic, but even better, OpenAI has an API and an official library for Python—a gold mine for the developers. Once logged, we can figure out the following: OpenAI gives us two months of free trial usage with an $18 credit. Thanks, dudes, that’s cool.

The OpenAI console is straightforward; the only thing you have to do is generate a new API key to allow any external program to connect:

General Architecture

Here is the general architecture of the project:

Notes:

• Do you remember my article about the hotel platform? We will reuse the main components for authentication: we will connect a Unity app to AWS thanks to a login system with Cognito and API Gateway.
• We will create a user pool and a new user with a name and a password in Cognito.
• We will create an endpoint in API Gateway and an associated Authorizer so that only the users of the user pool can consume the API.
• The Lambda function will receive the text from Unity and call the OpenAI API.
• Once the OpenAI API has answered, we call Polly, the text-to-voice converter of AWS, which will convert the answer into a voice stream.
• We keep the audio file in an S3 bucket and generate a pre-signed URL to restrict access to the file.

AWS implementation

S3

In the same way that my previous article, we create a private repository:

Lambda

First, we create a Lambda layer with the OpenAI library. Do you remember my previous article about making a homemade CCTV? I explain there in detail how to create a Lambda layer from a local environment, so we will follow the same method with the OpenAI library:

Now, we create our Lambda function:

Don’t forget to add the openai Layer to the function:

Inside the Lambda function, we define a new environment variable called openai_api_key with the OpenAI API key value.

Inside the function’s role, we create 2 inline policies, one for Polly, and the other for S3.

And here is the function:

Notes:

• We store the OpenAI API key in a Lambda environment variable called openai_api_key, and we call it in Lambda thanks to the os.getenv function.
• We parse the Lambda function’s entry parameters and retrieve the message sent from Unity.
• We call the Create completion function of the OpenAI API with the message sent from Unity. We extract the answer, as specified in the OpenAI documentation, and trim it with the strip function to avoid spaces or line breaks.
• When we call the Create completion function, we concatenate the message with the sentence “Please give me a short answer.” to be sure that the response given by ChatGPT will not be too elaborate.
• We call the synthesize_speech function of Polly with boto3, passing the answer as a parameter. We chose the ogg format, the best choice to work in Unity with, as I demonstrated in this previous article.
• I chose Aria, a friendly New Zealand vocal option of Polly, but it’s up to you to choose your favorite one!
• We keep the audio stream locally as a file thanks to the open and write functions, and we upload it to an S3 bucket thanks to the upload_file function of the boto3 library. After finishing, we remove the local file thanks to the os.remove function.
• We generate a pre-signed URL with 1 minute of time life thanks to the generate_presigned_url function of the boto3 library, so only the user using the Unity app will be able to access the audio file.

Cognito

In the same way that in the hotel platform article, we create a new user pool in Cognito:

In the Pool, we create a new user with a name and a password:

API Gateway

In API Gateway, we create a new REST API with a POST method until our Lambda function, and we deploy it:

And we create an authorizer to allow access to the endpoint only for the Cognito users from the Pool we have created:

Unity3D Implementation

The Audio Component

Our app will be able to talk, so we need an AudioSource component in our project!

Note: We let the AudioClip parameter empty; we will fill it with the audio file from S3.

The UI components

I usually detail little about the UI building of my Unity apps because I’m not a designer nor a front-end developer. Still, in this case, I found it interesting to explain how I built the client app mainly because of the complex layout of the chat.

That’s how I built the client app:

Notes:

• We use a Canvas with a ScrollView (without scrollbars) to show the messages.
• We use a vertical Content Size Fitter to resize the content of the ScrollView automatically, and a Vertical Layout Group to place the messages vertically.
• We use a combination of horizontal and vertical Content Size Fitter and Layout Groups to resize the box containing the message.
• We use a sliced image for the box, so all the messages will always have the same rounder corners, no matter the text size.

The code

Okay, so we have a functional chat in Unity. Let’s connect it with the backend!

First of all, we login to Cognito when the application starts, and we store the token id returned by Cognito in a PlayPrefs parameter:

Please refer to my previous article for an extensive explanation of the above code.

Then, we write the functions to show and hide the user’s device keyboard:

Notes:

• Unity work with the native keyboard of the device where the application is running. That means the keyboard will look different if you run it on iOS or Android.
• We use the class TouchScreenKeyboard as specified in the Unity documentation and the related function Open.

Then, here is the most exciting part: we call our endpoint, and we pass the message written as a parameter:

Well, our endpoint returned a URL of the audio file, so we use it now to retrieve the file and play it:

Notes:

• We use the function GetAudioClip of UnityWebRequestMultimedia to retrieve the audio stream in ogg format.
• We assign the audio stream to the clip parameter of our AudioSource object.

And now, we can add the message to the chat:

Notes:

• We instantiate the user message object or the friend message object according to the needs.
• We use the function ForceRebuildLayoutImmediate to refresh the ScrollView content and avoid graphical bugs.
• We set the verticalNormalizedPosition parameter of the ScrollView to 0, so the scroll position is at the bottom, and we can see the last messages.

Costs

Let’s check with the AWS Calculator what our system cost would be for a very pessimistic scenario: you love the app, and you perform 100 daily, 3,000 requests a month.

• Cognito: We only have one MAU (monthly active user) for this project. Cost: 0.00 USD
• API Gateway: With 3,000 requests to our REST API, the monthly cost is 0,01 USD
• S3: Suppose that 50 KB could be the average size of an audio file; we would have 150 MB stored each month. Additionally, we would have 3,000 put requests and 3,000 get requests, leading to a monthly cost of 0.02 USD.
• Lambda: With 3,000 requests with an average time of 3 seconds and a 1,024 MB of memory allocation, the monthly cost is 0.00 USD; excellent!
• Polly: Polly is undoubtedly the more expensive AWS service here. Let’s suppose chatGPT answers have an average of 100 characters; the monthly bill will be 1.20 USD.
• OpenAI: Based on the OpenAI tokenizer tool, suppose that every question we ask ChatGPT represents 15 tokens, so we use 45,000 tokens monthly. According to the OpenAI pricing, this gives us a total of 0.9 USD monthly.

Total: The total bill for our system would be 2.13 USD monthly. It’s totally affordable, taking into account that this is a very pessimistic scenario.

Closing Thoughts

In this article, we could figure out how to build an entire cloud architecture on AWS and how easy it is to integrate the OpenAI API with a Lambda function. We also had the opportunity to discover Polly, the text-to-voice service of AWS. Furthermore, we could evaluate the cost of the entire system thanks to the AWS Calculator.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

All ids and tokens shown in this article are fake or expired; if you try to use them, you will not be able to establish any connections.

You can download the Unity package of the client app specially designed for this article.

A special thanks to Gianca Chavest for designing the amazing illustration.

The post I made ChatGPT talk using Unity3D and AWS appeared first on ab.

I recently moved into a new building with security guards and a video security system. Nothing strange in Peru; buildings with such a level of security are pretty common since home burglary is exceptionally high in the country. According to the security experts Budget Direct, Peru has the highest home burglary rate in the world, with 2,086 per 100,000 people per year.

Then, an idea came to mind: could I reproduce such a system on my own but for my home? You may remember previous articles of mine where I was playing with camera devices in Unity, so I had some experience dealing with video streams, and the idea looked feasible. Plus, I did not know what to do with the webcam device I bought to get AWS certified, so it was a great opportunity to reuse it.

Furthermore, I thought that it would be awesome to integrate a surveillance component that warns me on my phone when some movement is detected by the cameras. Companies like Xiaomi provide this kind of solution, and I could take it as an inspiration.

Requirements

This may be the hardest part. The idea floating in the air should be converted into something concrete, and a list of reasonable requirements should be defined. The common error is defining too many requirements that lead to an unfeasible project.

Based on my idea, I could define the following requirements:

• I should be able to visualize all the live video streams on my PC monitor.
• The system should work with the old-school webcam device I had to buy to get AWS certified.
• The system should be able to handle a variable number of camera devices.
• Since I can not always watch the monitor, the crucial moments (movements) should be recorded and stored in an online repository.
• After a video is recorded and stored, I should receive a notification on my phone.
• I should be able to download and visualize the videos on my mobile phone.
• Each video recorded should show the name of the camera device and the exact date and time.

Software Architecture

Let’s begin with the software architecture. Based on the requirements, we can define four main components of the system: the Desktop application, the client application, the backend system, and the messaging service.

For the desktop application, I chose to work with Unity3D. It may not be an obvious choice to make such a system, but I have experience with it, and it has been a long time since I wanted to make a direct integration between Unity3D and AWS, so this is a great opportunity!

Unity3D is a more obvious choice for the client application since it can run on multiple platforms and OS (Unity apps can run on Desktop, iOS, Android, WebGL, tvOS, PS4, and PS5).

For the server side, I chose to work with Amazon Web Services (AWS) mainly because of the reliability of its scalable services and my experience with it.

For the notifications, I chose Firebase, which provides the most popular Cloud Messaging platform at a meager price.

Here is the general architecture of the solution:

Notes:

• The desktop application will handle the video streams of the camera devices and show them on screen.
• Both Unity3D applications will connect directly to AWS using the AWS SDK for .NET.
• For this article, I decided to keep it simple: I will use IAM users to connect the applications with AWS securely. If you want to see a great integration with API Gateway and Cognito users, please read the following article: How I Built a Hotel Platform with Unity3D and AWS.
• IAM policies will restrict access to a specific S3 bucket.
• Once a video is uploaded to the bucket, an S3 event is triggered, and a Lambda function is called.
• The Firebase token of the mobile device is kept in a DynamoDB table. Again, I kept it simple: I will manually copy the Firebase token into the DynamoDB table, but we could also write a Lambda function to automatize it.
• The Lambda function calls the Firebase Cloud Messaging service to send a push notification to the registered mobile device.

Unity3D implementation

Showing video streams

Do you remember my article about real-time texture analysis? We will build something similar for the Desktop application. We need the following Unity objects: a Camera, a Plane, and a WebCamTexture.

For each camera device, we will get the stream as a WebCamTexture and render it on a Plane. A Unity Camera will focus on the Plane, and a video will be recorded if a movement is detected.

We will create an object containing the Plane, the Text (camera name and date), and the recording Camera. A script CamController is attached to the parent object and will manage all related to the video stream: opening stream, texture analysis, and recording. For each camera device, we will clone (or instantiate) the model object.

So first, we look for the available webcam devices, and we will create an object for each one:

Notes:

• We use the class WebCamTexture to retrieve all the available video devices.
• We use the Instantiate function to clone the model object.

Then, we will init the cam controller:

Then, we update the text field each second with the current date and time:

Notes:

• We use the InvokeRepeating function to call the update function each second.
• We use the <mark> tag of TextMeshPro, which offers seriously fantastic format options, as mentioned in the documentation.
• We use the .NET DateTime.Now function to retrieve the actual date and time, and the ToString function to format it.

Finally, we can show the video streams:

Notes:

• We create a new WebCamTexture with the exact name of the device and a standard HD resolution.
• We render the stream on the Plane replacing the plane’s texture with the WebCamTexture of the camera device.
• We use the Play function of WebCamTexture to show the video stream.

Here is the final result with two camera devices and my dog Oreo; it looks pretty cool

Movement detection

We will now perform texture analysis in real-time to detect any movement on camera. This should include the following behaviors:

• Objects moving relatively fast: people, animals, or something else walking
• Brusque change of light: light turned on in the room, flashlights, etc.
• Any strong environmental perturbation: earthquake, fire, explosion, lightning, etc.

In the following example, I took two identical pictures consecutively, and I checked the differences with the online tool DiffChecker. Differences are highlighted in pink on the third picture:

Theoretically, there should not be any difference between both pictures, but there are: the camera may have moved a little bit, or it may have been some subtle fluctuations of light that caused this result. So we have to take into account only significant changes and ignore the weak ones during the frame analysis.

In the following example, there was a significant change between the two pictures, and we can conclude that the earphones moved. Everything else can be ignored.

To perform the movement detection, we will retrieve the array of pixels of the current texture frame and analyze it. This consists of two steps:

1. Compare each frame pixel with the previous frame’s relative pixel. If the color is significatively different, then the pixel has changed.
2. Count how many pixels have changed over the total of the frame’s pixels. If the ratio is significant, the frame is different from the previous one, and a movement is detected.

This is a way to perform it in Unity3D:

Notes:

• We use the GetPixel function to retrieve an array of pixels of the current frame.
• Each color channel (R, G, B) is a floating value with a range from 0 to 1.
• I found that under 5% of a color change, a pixel can still be considered unchanged.
• I found that under 5% of a total change, a frame can still be considered unchanged.

Video capture

OK, so once we have realized the movement detection on the camera devices, we need to capture the stream and save it as a video file. I found several ways to do it:

1. Using the VideoCapture class: it looks good, but I’m currently using a Mac. Too bad, I need another option.
2. Working with a third-party asset: It’s a better option, but I’m almost always reluctant with third-party assets. Is the asset really that good? Will it be maintained in the future?
3. Using the Unity package called Recorder. This looks much better than the previous options: multi-platforms are allowed, it is released by Unity itself, and best of all… it’s free!

The Recorder package can be installed through the Package Manager window:

Let’s have a look at the package. The Recorder package provides a graphic interface where video captures from the editor can be performed manually. The fantastic feature of this package is that you can select a specific camera as a target. Good for us; we can record videos from different video streams simultaneously.

This looks good, but we need to automatize it. Here is a way to do it programmatically:

Note: all the objects and classes of the video recorder are described in the documentation.

Connecting Unity3D with AWS

Dealing with the AWS SDK:

I will be honest; I struggled a little bit on this part and will explain why.

Years ago, AWS used to provide a custom SDK for Unity3D, which was very easy to install. It has been deprecated and is now included in the AWS SDK for .NET. It sounds good, but the AWS documentation does not provide further details about using this SDK in a Unity project.

For .NET, Microsoft uses the NuGet mechanism. In Visual Studio, you can open the NuGet Packages window, where you can visualize the .NET packages installed in your current project.

For the current project, we need to realize an integration with S3, so we need the base package AWSSDK.Core and the S3 package AWSSDK.S3.

But hold on! The NuGet window is essentially for .NET projects, so your Unity project won’t be able to recognize the packages if you install them by the NuGet window; you have to do it manually from the NuGet website.

On the NuGet website, you can download the AWS core package and the AWS S3 package. Since the AWS SDK functions use asynchronous tasks, you must also download the AsyncInterfaces package.

Once downloaded, you can unzip the packages and place them in the Plugins folder of your Unity project:

The new packages are now recognized inside the Unity project; we did it well

The functions:

Upload file:

In the Desktop application, we need to upload the video file recorded to our online S3 bucket. Hopefully, the .NET documentation is complete and gives us excellent implementation examples. To Upload a file, we will use the PutObjectAsync method.

Notes:

• We create an S3 client thanks to the AmazonS3Client and AmazonS3Config classes.
• We build the request thanks to the PutObjectRequest class.
• The class Task is a pure .NET mechanism to manage asynchronous processes. You can call it in Unity with the await operator within an async function.

List Buckets:

In the client application, we need to list the content of the S3 bucket. Thanks to the ListObjectsV2 method, we will retrieve a list of file names.

Notes:

• We build the request thanks to the ListObjectV2Request class.
• We store the response in a list of file names.

Download file:

Once we have the list of the files in the bucket, we can download the video of our interest. To achieve it, we will use the GetObjectAsync method.

Notes:

• We build the request thanks to the GetObjectRequest class.
• We use persistentDataPath to store in the device the file we downloaded.

Show videos

We will use the VideoPlayercomponent to show the videos inside the client application and choose “URL” as the source.

Then, for each video selected, we replace the URL parameter with the file path, and we play the video.

Notes:

• We first check if the file is already present in the device thanks to File.Exists method. If not, we download it.

Firebase

In the Firebase console, create a new project and then a new Unity app.

Creating a new Firebase app is straightforward: you must register the app, download the config file to your Unity project, download the Unity SDK, unzip it, and install the desired Unity packages. Every step is well-explained in the console, and the whole process only takes a couple of minutes to achieve.

For this project, we will install the FirebaseAnalytics package (as recommended) and the FirebaseMessaging package.

In the Unity client application, we can use the GetTokenAsync function to retrieve the Firebase device token:

AWS implementation

S3

We create a private repository where all video files will be stored. I will call mine “monitoring-ab”.

IAM

In IAM, we will create two users: one for uploading the video files to S3, and the other one to download them. In the IAM console, we will create two users with programmatic access:

Once created, don’t forget to copy the secret access key or download the CSV file with the credentials. They will no longer be visible!

Those credentials will be used in Unity when the S3 client is created:

Then, we need to create a policy for each user. The policies will allow the user to perform actions on a specific bucket (monitoring-ab in my case). For this project, it is more convenient to create inline policies and not managed policies since they will be an inherent part of each user and will not be reusable. You can check the IAM documentation for more details.

So we will create an inline policy for each user. To achieve it, click on the “Add inline policy” button in the user details screen.

For the download user, we create a policy with the ListBucket and GetObject permissions for the specific bucket monitoring-ab.

For the upload user, we create a policy with the PutObject permission for the specific bucket monitoring-ab.

DynamoDB

In the DynamoDB console, create a new table with appId as the partition key:

Then create an item related to the name of the S3 bucket and an array with the Firebase devices tokens:

Lambda

Lambda layer:

Before writing the Lambda function, we must load the Firebase Python library in a Lambda layer. The library handles Python >=3.7, so we will work with the most recent version of Python supported by Lambda, Python 3.9. We could upload the library along with the code, but by embedding it in a layer, we could reuse the Firebase library in other projects and focus on the code.

A great tip for building a Lambda layer is first to install the required libraries in a new local environment. I recommend using PyCharm Community which integrates local virtual environments. The only thing you have to do is create a new project along with a new associated virtual environment.

So in a new Python 3.9 environment, we will first update PIP, the Python package manager, and then install the Firebase library thanks to the following commands in the PyCharm terminal:

Once installed, you can visualize the Firebase library and all its dependencies in the venv folder of your project:

It looks good! The next step is copying the lib folder contained in the venv folder to a new folder called “python” and, finally, zip it.

We are almost done! Now, we log in to the AWS console, create a new Lambda layer for Python 3.9, and upload the zip file.

Our layer is ready to be used!

Lambda function:

This may be the most exciting part! Our Lambda layer is ready, and we can now send messages to Firebase within a Lambda function.

First of all, we will create a Lambda function for Python 3.9.

In the new Lambda function created, don’t forget to add the firebase layer:

Then, we go back to the S3 console and create an S3 event so that the new Lambda function will be triggered after each file uploads in the bucket. The Lambda function will be called after a “Put” event is detected on the bucket:

Go back to Lambda; we can see that the S3 event has been taken into account, and the layer has been added:

And here is the function:

Notes:

• To retrieve the credentials of your Firebase project, you will first have to create a private key in JSON format. This can easily be done in the project settings as indicated in the firebase-admin documentation. Then, you can load the credentials as a local file in your Lambda function.

• We get the camera and bucket names from the S3 event. You can find the detailed structure of an S3 event and a great example of implementation in Python in the AWS documentation.
• We use the unquote_plus function of the urllib library to parse the camera and bucket names.
• We create a new Firebase message thanks to the MulticastMessage class, and we send the message thanks to the send_multicast function. You can find great examples of the firebase-admin function implementations on Firebase’s GitHub repository.

Final result

Well, we can now visualize the final result! For this test, I placed two cameras on my terrace while my dogs were sleeping in the bedroom, and I called them. I activated the movement detection only for the camera focusing on the dogs.

In the Desktop app, we can see Oreo and Simba coming to me from the bedroom:

Movements have been detected, and a short video has been created (You can observe a red circle on the first cam when the video is being recorded). Once the video was uploaded, I received a notification on my mobile device:

If we check the Firebase console, we can see that the message has been sent successfully:

I can now open the mobile app and watch the video recorded:

Costs

This is an update! After publishing this article, Seth Wells made an excellent observation in the comments section: what about the costs? Let’s do the exercise with the most pessimist hypothesis.

The system I designed has a minimum recording interval. A big interval would make the whole system inefficient, and a too-small interval would make it uncomfortable (receiving a notification every minute can be bothering). So let’s talk about an interval of 5 minutes.

Based on this value, supposing that there always is movement on cameras, we would have 24×60/5 = 288 videos a day, around 9,000 videos a month. The size of a video file can vary depending on the resolution you want. Let’s talk of 1MB videos for 10 seconds recorded. Now let’s check the costs with the AWS Pricing Calculator.

• IAM: IAM is free to use.
• S3: This is the core service of our system. With 9GB of data by month, 9,000 put requests, 1,000 list requests (when a user opens the client app, hypothetical), and 1,000 get requests (when a user downloads a video, hypothetical), we get a bill of 0.26 USD by month, 3.12 USD by year.
• Lambda: With 9,000 requests a month, an average of 3 seconds by request, and 128MB of memory allocated, we have a bill of 0.00 USD, good news!
• DynamoDB: DynamoDB is here underused: we don’t write in any table, and we have a read request every 5 minutes. Cost: 0.00 USD.
• Firebase: Firebase Cloud Messaging is absolutely free.

Total: 0.26 USD by month, 3.12 USD by year with a pessimistic hypothesis. We can safely say that our system is affordable!

Closing thoughts

In this article, we had the opportunity to see how to build an entire cloud architecture and how to call an external messaging service (Firebase) from it. We also could build two Unity3D apps that directly interact with AWS thanks to the .NET SDK. Furthermore, we could evaluate the cost of the entire system thanks to the AWS Calculator.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

All ids and tokens shown in this article are fake or expired; if you try to use them, you will not be able to establish any connections.

You can download the Unity packages of the Desktop app and the client app specially designed for this article.

A special thanks to Gianca Chavest for designing the awesome illustration.

The post I Built a Homemade CCTV Using Unity3D and AWS appeared first on ab.

Packages are basically completed Python code (classes, functions, etc.) that you can use in your projects. They are usually located in a specific directory of your environment. You can create your own custom Python packages or download plenty of fabulous and free Python packages from THE PYPI OFFICIAL REPOSITORY.   

If you’re not already familiar with Python, I suggest you check out our LEARN PROGRAMMING WITH PYTHON track, which introduces you to the fundamentals of programming.

Before going further, it is essential to mention that the difference between a module, package, library, and framework in Python can be quite confusing. If you are interested in knowing the exact terminology, read our article on the DIFFERENCE BETWEEN MODULES, PACKAGES, AND LIBRARIES IN PYTHON. We’ll be using the terms package and library interchangeably in this article.

The post Little-Known But Useful Packages that Come with Python appeared first on ab.