This article was initially published on my Medium Page.
Have you ever heard about LayerMasks in Unity? If you have not, be careful: the stuff seems to be easy and well… it’s not.
The truth is, it is sometimes hard to understand how LayerMasks in Unity work. Unity Documentation is kind of evasive and the explanation of how they are built is confusing.
This article will compile several sources to provide an extensive explanation of how LayerMask values are calculated and which operators can be used to perform a perfect LayerMask.
Are you ready?
About Layers
To understand how LayerMasks work, it is necessary to have a brief reminder about Layers.
Layers are used in Unity to group GameObjects with the same behavior. Layers can be used to hide GameObjects from a Camera, determine which GameObjects will be illuminated, or ignore GameObjects from a Raycast for instance.
According to Unity Documentation, there are a total of 32 Layers: 5 Builtin Layers and 27 customizable User Layers. The Builtin Layers are the following: Default, TransparentFX, Ignore Raycast, Water, and UI, and can not be changed. Each Layer is defined by an id (from 0 to 31) and a name. Names of Builtin Layers are locked.
In this article, we will work with a User Layer called Custom Layer. To create a new custom Layer or edit an existing one, open the Tags and Layers window: Edit > Project Settings > Tags and Layers.
Note: Unity discourages using user layer 31 to avoid clashes with Editor’s Preview.
About Physic Casts
Now we need to talk about Physic Casts!
Casts in Unity draw an imaginary shape on space and return possible intersections with colliders. There is a cast function for each basic shape in Unity for 2D and 3D space.
3D cast functions:
Physics.RayCast
Physics.LineCast
Physics.BoxCast
Physics.CapsuleCast
Physics.SphereCast2D cast functions:
Physics2D.RayCast
Physics2D.LineCast
Physics2D.BoxCast
Physics2D.CapsuleCast
Physics2D.CircleCastLayerMasks
LayerMasks are used in the Physics Casts functions to specify which Layers will interact with the cast shape. Only the colliders which belong to the specified Layers will be triggered. In short, LayerMasks act as filters between Layers and Cast functions.
Example of a Raycast function declaration using a LayerMask:
How does it work?
A LayerMask is a bitmask: it is composed of 32 bits, each bit represents a layer. The final result will be an integer.
Example 1: We want to build a LayerMask that allows interactions only with the Water Layer.
Example 2: We want to build a LayerMask that allows interactions only with the Water Layer and the new Custom Layer we just created.
Example 3: We want to build a LayerMask that allows interactions with all Layers except the Water Layer.
Wow… this is a huge number!
Remember that a LayerMask results in an integer value int. According to C# documentation, the int type is a signed 32-bit integer with a range between –2,147,483,648 and 2,147,483,647.
When you go above the maximum value of an int, the result becomes a negative number beginning from the minimum value.
So we have:
max value + 1 <=> min value
And:
max value = 2,147,483,647 = 4,294,967,279–2,147,483,632
min value = –2,147,483,648
We solve it:
2,147,483,647 + 1 <=> –2,147,483,648
4,294,967,279 – 2,147,483,632 + 1 <=> -2,147,483,648
4,294,967,279 <=> -2,147,483,648 – 1 + 2,147,483,632
4,294,967,279 <=> -17
Well, this is a little bit complex… We better use the bitwise complement operator ~ to achieve this LayerMask.
The bitwise complement operator ~ reverses each bit of a bitmask:
1…101111 = ~0…010000 = ~16 = -17
Building LayerMasks in Unity
There are multiple ways to build a LayerMask in Unity, let’s see how it can be done using the previous examples.
1. By name:
The most common way to get a mask from Layers names is using the GetMask function of LayerMask class. Note that the GetMask function accepts one or multiple layer names.
Example 1:
print(waterLayerMask.value); // 16
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, waterLayerMask);
Example 2:
print(layerMask.value); // 80
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, layerMask);
Example 3:
print(exceptWaterLayerMask.value); // -17
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, exceptWaterLayerMask);
2. By id:
If you are not sure of the layer name or you prefer to manage it by layer ID, you can get the mask embedding GetMask and LayerToName functions:
Example 1:
print(waterLayerMask.value); // 16
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, waterLayerMask);
Example 2:
print(layerMask.value); // 80
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, layerMask);
Example 3:
print(exceptWaterLayerMask.value); // -17
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, exceptWaterLayerMask);
3. By shift operator:
An awesome way to get a mask from a specified Layer is using the left-shift operator <<. This operator shifts a sequence of bits by a specified number to the left.
Example 1:
print(waterLayerMask.value); // 16
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, waterLayerMask);
Example 2:
In this example, we combine two bits representation thanks to the logical OR operator |
print(layerMask.value); // 80
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, layerMask);
This example can be optimized:
print(layerMask.value); // 80
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, layerMask);
Example 3:
print(exceptWaterLayerMask.value); // -17
Physics.Raycast(Vector3.zero, Vector3.up, Mathf.Infinity, exceptWaterLayerMask);
4. By integer value:
If you know the integer value of the LayerMask you need, you can pass it directly to the Cast.
Example 1:
Example 2:
Example 3:
5. Using defined values:
Although it sounds weird, it is possible to create a LayerMask composed of no layer with a value = 0. It is also possible to create a LayerMask composed of all existing layers, with a value = -1 (Note that: ~0 = -1).
Some of LayerMask values from builtin layers are available in the Physics class:
- Physics.AllLayers: LayerMask of all Layers. Value: -1 (= ~0)
- Physics.IgnoreRaycastLayer: LayerMask of Ignore Raycast Layer. Value: 4
- Physics.DefaultRaycastLayers: LayerMask of all Layers except Ignore Raycast Layer. Value: -5 (= ~4)
Closing thoughts
Congrats, you read this article entirely, you’re a real Unity nerd!
This article showed you the different ways to work with LayerMasks in Unity, and provided a precise explanation of how LayerMask values are built and calculated.
This article was also a great opportunity to learn how the following low-level c# operators work:
- Bitwise complement operator ~
- Left-shift operator <<
- Logical OR operator |
Every code of this article has been tested using Unity 2020.3.17 and Visual Studio Community 2019.
A special thanks to Xavier Rigoulet for supporting my new adventure on Medium, and Gianca Chavest for designing the awesome illustration.
