Blending (Rendering Partially-Transparent Objects)

Table of Contents

1. Introduction
2. Summary
3. Terminology
4. How transparency works
- 4.1. Alpha testing
- 4.2. Alpha blending
5. How do we determine whether to use alpha testing or alpha blending
6. Sorting (for alpha blending)
- 6.1. Additional deprecated sorting options
7. 2D drawing of primitives and images

1. Introduction

Blending is the technique to render partially transparent objects with real-time graphics. Partially transparent objects mean that the algorithm accounts for transparency values between 0.0 and 1.0, e.g. 0.25 or 0.5.

2. Summary

Things should work out-of-the-box, but if you experience artifacts at rendering multiple partially transparent objects in your game, customize the TCastleViewport.BlendingSort.

You can set it in the editor.

Read on to understand the details.

3. Terminology

Transparency is a value in 0..1 range, where 0 means that object is opaque, and 1 means that object is completely transparent (invisible).

The X3D material nodes have a Transparency property, TPhysicalMaterialNode.Transparency, TMaterialNode.Transparency, TUnlitMaterialNode.Transparency.
Alpha says "how much is something opaque", it is simply "1.0 - transparency". So alpha = 0.0 is something "completely invisible", alpha = 1.0 is "something totally opaque".

When you provide an RGBA color (TCastleColor, which is just equal to TVector4) then the last (4th) component is alpha.

When you provide an RGBA texture, then it includes an alpha channel, i.e. alpha value for each pixel.

When you provide a texture without alpha channel (like RGB or grayscale) then it is always treated like alpha is 1.0 everywhere, i.e. it is fully opaque.

4. How transparency works

4.1. Alpha testing

Engine can handle transparency using alpha-testing, which means that each rendered pixel is either fully opaque or fully transparent.

More precisely: at each pixel, we test the alpha of the "resulting color". "Resulting color" here comes from mixing material color, per-vertex colors, and texture colors. If the color’s alpha is > 0.5 then the pixel is rendered (as if the shape was opaque at this pixel). Otherwise the pixel is not rendered (as if the shape was completely invisible at this pixel).

While alpha-testing is a less capable algorithm than blending (as alpha-testing cannot account for partial transparency), it also avoids some issues unavoidable with blending. E.g. alpha-testing works without any problems with shadow maps.

Alpha testing is sometimes called alpha clipping (e.g. in Blender UI).

4.2. Alpha blending

Engine can handle transparency using blending. This means that all partially-transparent shapes are rendered after all other shapes (that are opaque or use alpha-testing). Moreover,

The partially-transparent shapes are rendered with Z-testing but without Z-writing. This means that partially-transparent shapes that are behind opaque objects are hidden, but partially-transparent shapes behind other partially-transparent shapes are visible. This matches reality, as partially-transparent shapes never fully "obscure" stuff behind them, by definition.
When the partially-transparent pixel is rendered, it is mixed with the existing screen color using the "blending equation". By default is it screen_color.rgb = incoming_color.rgb * incoming_color.a + screen_color.rgb * (1 - incoming_color.a). This equation can be configured using Scene.RenderOptions.BlendingSourceFactor, Scene.RenderOptions.BlendingDestinationFactor and can be overridden per-shape using Appearance.blendMode.
The partially-transparent shapes are rendered in back-to-front order (if sorting is active, which is the default, see below). That is because the default blending equation (see above) assumes such order.

Various shape sorting algorithms are documented at TShapeSort. Basically, common sort3D is intuitive "distance to the middle of the shape bounding box". And common sort2D is intuitive "distance to 2D camera".

Beware: in some cases, partially transparent shapes may be concave and intertwined in various crazy ways in 3D. It may not be possible to strictly say "shape X is in front of shape Y", because they may be sometimes in front, sometimes behind each other, at each pixel of the screen. To have reliable blending results keep partially-transparent shapes simple, preferably convex and using backface culling.

5. How do we determine whether to use alpha testing or alpha blending

Engine makes the decision about blending per-shape.

In most cases, TAppearanceNode.AlphaChannel explicitly says what to do.

If you use glTF model format, this always has explicit information (never acAuto) whether to use alpha testing, blending, or treat shape as opaque.

If you use Blender, you can explicitly specify alpha treatment at Blender material. It will be correctly exported to glTF.

If TAppearanceNode.AlphaChannel is acAuto (this happens e.g. for X3D models):

The engine looks at material transparency field, and the texture’s alpha channel. If transparency is > 0, or if the texture has non-trivial alpha channel (this is detected by analyzing alpha contents, see TEncodedImage.AlphaChannel description), then we use blending.

Note

This auto-detection cannot be always right.

For example, the alpha channel detection at the image (TEncodedImage.AlphaChannel) is a heuristic, with some alpha tolerance. And what happens when multiple textures are used, with different alpha channel? Or if you use GLSL shader code to set/modify alpha value (the engine never parses your GLSL code, so it doesn’t know about it)?

The engine essentially makes an educated guess about the author’s intentions.

To be clear what happens, we advise to always specify the alpha treatment explicitly. One easy way to do this is just to use glTF (in this format, alpha treatment is always specified explicitly). Or set TAppearanceNode.AlphaChannel from Pascal.

6. Sorting (for alpha blending)

The sorting of partially-transparent objects is controlled using TCastleViewport.BlendingSort.

By default it is sortAuto, which performs best sorting for 2D (sort2D) if you have a typical 2D camera (orthographic looking along -Z), and best sorting for 3D (sort3D) otherwise.

All possible sorting options:

sortAuto - Auto-detect and use either sort2D or sort3D.
sort2D - Sort along the Z axis.
sort3D - Sort based on the distance from current camera, looking at bounding box center.
sort3DVerticalBillboards - Sort based on the distance from current camera, projected on the Y=constant plane, looking at bounding box center.

This is especially suitable if your world consists of vertical billboards (like trees) with TCastleBillboard.AxisOfRotation = +Y (0,1,0) and you want to sort them correctly.

See example examples/viewport_and_scenes/billboards_blending_in_3d, screenshot below:
sortCustom - Sort using custom comparer event, see TCastleViewport.OnCustomShapeSort and TShapeSortEvent.
sortNone - Do not sort. This may be reasonable for simple world arrangements, when you manually ordered all shapes and all scenes to follow back-to-front order, or when you use non-standard blending equation that does not require sorting.

6.1. Additional deprecated sorting options

Moreover, the following 2 sorting options are available but deprecated. Most cases when these options were useful in the past should be now covered by sort3D or sort3DVerticalBillboards. Contact us if you have a use-case where these deprecated options still make sense.

sort3DOrigin - Sort based on the distance from current camera, looking at origin point (point (0,0,0) in local coordinates). This may be better than sort3D in some cases because the origin point is sometimes more "persistent", e.g. animations or billboard orientation may change bounding box, but not origin point.
sort3DGround - Sort based on the distance from current camera, looking at origin point (point (0,0,0) in local coordinates) projected on Y=0 plane. This may be better than sort3DOrigin in some cases because it ignores the object height for the sorting purposes.

7. 2D drawing of primitives and images

If you draw using DrawPrimitive2D, DrawRectangle etc. — they automatically use blending when provided color has alpha < 1. They take blending factors (that determine the "blending equation" mentioned above) as explicit arguments.

If you draw using TDrawableImage then it automatically determines alpha treatment looking at image contents and the TDrawableImage.Color. You can override alpha treatment by TDrawableImage.Alpha property, there are also properties to determine blending equation: TDrawableImage.BlendingSourceFactor, TDrawableImage.BlendingDestinationFactor.

The above routines are used by user interface rendering, e.g. by TCastleButton or TCastleImageControl rendering, so they follow the same alpha treatment. The TCastleImageControl.AlphaChannel allows to control blending in case of TCastleImageControl, underneath it uses TDrawableImage to render.

To improve this documentation just edit this page and create a pull request to cge-www repository.