Did you know that our four-legged friends see the world in a different way than us? In this article, we will recreate with Unity3D what would be a dog’s vision and simulate it with a mobile device.

Are you ready?

How do dogs see?

Studies suggest that dogs are dichromats; their vision is very similar to human deuteranopia (red-green color blindness). That means they can only perceive 2 pure spectral colors: blue and yellow and have difficulties distinguishing between red and green colors.

Dogs also have poor visual acuity (they don’t perceive faraway objects with clarity) and awesome moving detection. That can explain some strange behavior, like when your dog does not recognize you until you are near, but can detect a cat walking two blocks away (and eventually get crazy).

In this article, we will focus on the colors singularity of dogs’ vision.

Building the app with Unity

To achieve it, this is pretty simple, we only need 4 main elements: a Camera, a Shader, a Plane, and a WebCamTexture. The Shader will act as a colors filter for the Camera, and the Plane will be a support for the WebCamTexture.

The Shader

First, we write a shader with an RGB color transformation:

According to Martin Krzywinski’s incredible work, this is the RGB transformation matrix of a deuteranopia vision:

OK, so we write a c# script to load the shader we created. We pass the previous matrix values through the OnRenderImage function to render the deuteranopia visual effect:

The Camera

We will use a generic Unity Camera and attach the script we have just created to it. Before starting with the device camera implementation, let’s try our deuteranopia shader with a simple picture (SpriteRenderer).

It works!

Adobe developed a powerful color tool that allows simulating color blindness. Let’s compare our result with Adobe’s tool:

Well, this is a pretty good result! We got a deuteranopia vision very close to Adobe’s tool, ours is maybe a little bit greener.

The Plane

A default Plane in Unity is usually used as a floor component, so we play with the rotation parameters in order to place the Plane in front of the Camera.

Note: if you are going to work with a webcam or a frontal device camera, you will need to invert the x scale.

The WebCamTexture

The WebCamTexture is a special texture that allows capturing a live video from a device camera (or from a webcam if you are working with a laptop) and rendering it in Unity space.

We create a script in order to access the user’s device camera. We have first to request authorization to the user using the RequestUserAuthorization function. Then we create a new WebCamTexture and we apply it to the Plane.

Then we attach the script to the Plane:

Final Result

After building the application on an old LG phone, here is the final result on video:

Closing thoughts

This article showed you how to make a very simple Unity app involving 3D objects, shaders, user requests, and device cameras. We also had the opportunity to learn about dog’s vision and deuteranopia in general and figured out that buying a red or orange ball for our dogs is a very bad idea

If you want to go further with Unity3D and color blindness, you can check the wonderful work of Alan Zucconi.

Every code of this article has been tested using Unity 2020.3.17 and Visual Studio Community 2019. The mobile device I used to run the Unity app is an LG Magna with Android Lollipop.

You can download the full DogVision Unity Package specially designed for this article.

A special thanks to Gianca Chavest for designing the awesome illustration.

The post Building a dog’s vision application with Unity3D appeared first on ab.