glTF (model format)

Table of Contents

1. Introduction
2. Resources
3. Supported Features
4. Attaching objects to bones
5. Collisions when your glTF mesh uses skinned animation
6. Switching to Phong lighting model (for performance or just different look)
7. Gamma Correction
8. Interoperability with X3D

1. Introduction

glTF is an efficient, modern format for animated 3D and 2D models. Developed by Khronos.

We advise using it in CGE as much as you can. It is supported by a lot of tools. We focus on supporting this format perfectly in Castle Game Engine, with all the features and efficiency.

2. Resources

glTF main page,
glTF specification,
glTF main development repository(lots of useful links in the README).
glTF 2.0 sample models from Khronos, Sketchfab (open them easily with our view3dscene),
Blender includes a full-featured glTF exporter. See Blender manual about glTF exporter and our manual about exporting to Blender.
Many authoring tools and converters to glTF are listed here.

3. Supported Features

Meshes (polygons, lines), transformation hierarchy.
Materials (with physically-based or unlit shading, internally implemented using X3D 4 materials, designed for glTF compatibility), alpha mode, double-sidedness, per-vertex colors.
Texturing (for base color, normal maps, emissive, material-roughness).
Animating transformations (position, rotation, scale) and using skin ("armature" in Blender). Animations can be played using TCastleScene.AutoAnimation or TCastleScene.PlayAnimation. You can also play multiple animations from one model simultaneously using TTimeSensorNode.Start/Stop.
Cameras (perspective and orthogonal).
Camera transformations can be animated too. The advised approach to do this is to place camera as a child of animated bone, exposed using TCastleScene.ExposeTransforms. See Viewports, cameras and navigation in Castle Game Engine movie for a demonstration.
Punctual lights (point, spot, directional lights).
Both .glb and .gltf extensions are supported. Textures can be provided in separate files or embedded inside the GLTF stream.
It is integrated in our engine as X3D nodes graph. This means that you can include a glTF model inside larger X3D file using the Inline node, you can modify the glTF scene graph at runtime (e.g. modify material color, show/hide something etc.) and you can serialize the scene graph to an X3D file.
Collisions automatically work (as with all other scene formats), you only have to initialize Scene.Spatial (see the manual). By default, static meshes have precise collisions (treating them like a set of triangles), while skinned-animated meshes collide as simple bounding boxes (so they don’t cause a slowdown when animating). This can be even customized per-shape by adjusting Shape.collision property.
We use linear color space (gamma correction) automatically on PBR materials. You can request to apply it on all materials (including unlit) to follow glTF spec in 100% easily.
We read glTF "extras" data that can be defined by your authoring tool, for example in Blender this is defined by "Custom properties". This allows to pass any custom information from Blender to the engine, for use in your own applications, or to influence the import — see Custom properties in Blender.
We use PasGLTF, a great open-source library for reading glTF by Benjamin "Bero" Rosseaux.
We support common glTF extensions:
- KHR_materials_unlit - unlit materials.
- KHR_lights_punctual - define lights (point, spot, directional) within glTF file.
- KHR_texture_transform - transform (scale, offset, rotate) texture coordinates.
- KHR_materials_pbrSpecularGlossiness - support for alernative specification of material parameters using specular-glossiness. Only partial support — without textures. Extension considerd deprecated by Khronos in favor of KHR_materials_specular.

TODO: Main missing glTF feature is morph targets. It is coming!

4. Attaching objects to bones

This is available using TCastleScene.ExposeTransforms. You can "expose" a bone transformation as TCastleTransform child and attach there a scene. See https://castle-engine.io/wp/2020/10/09/attach-objects-to-animated-bones-like-weapon-in-characters-hand-by-exposetransforms/ .

5. Collisions when your glTF mesh uses skinned animation

For speed, the shapes animated using skinned animation in glTF uses bounding box for collisions. That’s because the triangles would change every frame and updating the octree would have a significant cost for FPS.

If you need to have better collision detection:

You can use X3D file that uses Inline to include 2 glTF files. One of them would be your animated model, but not collidable. The other would be a static model, invisible, used only for collisions.

This means that your model keeps working fast (as opposed to solution 2 below). And the collisions are resolved with respect to precise triangles. However, the triangles remain static, unaffected by animation.

To do this you would create a file like mycreature.x3dv with content:
```
 #X3D V3.2 utf8
 PROFILE Interchange

 Collision {
   proxy Inline { url "mycreature-collidable-invisible-notanimated.gltf" }
   children Inline { url "mycreature-animated-visible-notcollidable.gltf" }
 }
```
And then in game, you open castle-data:/mycreature.x3dv instead of opening any glTF file directly. Playing all animations on mycreature.x3dv should work exactly as in mycreature-animated-visible-notcollidable.gltf, it exposes the same animations.

If you desperately need precise collisions, and the collision structure has to be updated at runtime, and you can tolerate some performance loss (it may be acceptable for smaller models) then you can find TShapeNode occurrences in the model, and change the TShapeNode.Collision from scBox to scDefault.

Like this:

 procedure TMyState.Load;
 var
   Model: TX3DRootNode;
 begin
   Model := LoadNode('castle-data:/example.gltf');
   Model.EnumarateNodes(TShapeNode, @HandleNode, false);
   Scene.Load(Model, true);
 end;

 procedure TMyState.HandleNode(Node: TX3DNode);
 begin
   (Node as TShapeNode).Collision := scDefault;
 end;

6. Switching to Phong lighting model (for performance or just different look)

glTF models use PhysicalMaterial or UnlitMaterial for their materials.

The PhysicalMaterial node performs physically-based rendering which is very pretty but also comes with some performance cost. It also requires Phong shading (not faster Gouraud shading) to make sense.

If you need maximum speed, you can set global GltfForcePhongMaterials to true. This automatically converts (during load) all PhysicalMaterial nodes into Material nodes (using Phong lighting model, and Gouraud shading by default). Note that it will change the look of your models significantly. So if you want to go this way, you should probably prepare your models from the start testing it.

Of course, remember that you can also use unlit materials in glTF. These always have the best performance :) Blender can export glTF unlit materials.

7. Gamma Correction

The PhysicalMaterial, used by most glTF models, has GammaCorrection turned on by default.

If you need maximum speed, consider disabling gamma correction, by GammaCorrection := gcNone.
If you need maximum glTF compatibility, consider enabling gamma correction for all materials (PhysicalMaterial, UnlitMaterial, Material), by GammaCorrection := gcAlways.

Note that enabling or disabling gamma correction will change the look of your game. So you should make a decision about it early on — and test your models look with the desired setting.

8. Interoperability with X3D

Under the hood, glTF models are loaded as a graph of X3D nodes. This allows to use all X3D nodes functionality with glTF models.

For an introduction to X3D, Physically-Based Rendering (PBR) and glTF see this presentation:

8.1. Process glTF models (e.g. change material color) using Pascal

You can process (modify after loading, e.g. to remove/change some nodes) loaded glTF models using CGE API to handle X3D nodes.

For example glTF material corresponds to the X3D node TAppearanceNode (that in turn has TPhysicalMaterialNode as a child). To modify color of an object loaded from glTF, you can find the TAppearanceNode within the scene, get the TPhysicalMaterialNode child, and modify the properties like TPhysicalMaterialNode.BaseColor as you wish. Like this:

var
  Appearance: TAppearanceNode;
  Material: TPhysicalMaterialNode;
begin
  { We assume that model has "MyMaterialName" material in Blender and glTF. }
  Appearance := MyScene.Node('MyMaterialName') as TAppearanceNode;
  { We assume that material type is PBR, so we can cast to TPhysicalMaterialNode. }
  Material := Appearance.Material as TPhysicalMaterialNode;
  { Set color to yellow. }
  Material.BaseColor := Vector3(1, 1, 0);
end;

See https://github.com/michaliskambi/x3d-tests/wiki/Converting-glTF-to-X3D for details how glTF concepts map to X3D.

8.2. Use `Inline` to import glTF model in an X3D file

You can use Inline X3D node to include glTF model (or part of it) inside a larger X3D model, as many times as you want. This is as simple as just writing

Inline {
  url "some_model.gltf"
}

inside X3D file (with classic X3D encoding).

Moreover, you can use use X3D IMPORT mechanism to access particular parts (like materials or animations or objects) of the glTF model. This way you can control glTF animations from X3D, or reuse glTF materials/objects etc. in X3D.

8.3. Use `Inline` and `IMPORT` to access glTF animations in an X3D file

For example of this technique see https://github.com/castle-engine/demo-models/tree/master/blender/skinned_animation , in particular file skinned_anim_run_animations_from_x3d.x3dv there. The X3D file has this code:

DEF InlinedAnimations Inline {
  url "skinned_anim.glb"
}
IMPORT InlinedAnimations.jump AS jump
IMPORT InlinedAnimations.walk AS walk

and then it can start animations jump, walk. They are just TimeSensor nodes in X3D. We can ROUTE events to them.

8.4. Use `Inline` and `IMPORT` to access glTF materials, meshes, transforms in an X3D file

Demo of this technique is in https://github.com/michaliskambi/x3d-tests/tree/master/gltf/avocado_and_exports . Open the models there with view3dscene or just load them into TCastleScene instance in CGE editor.

It looks like this:

InlinedAvocado Inline {
  url "glTF/Avocado.gltf"
}
IMPORT InlinedAvocado.CastleEncoded_2256_Avocado_d AS AvocadoAppearance

Shape {
  appearance USE AvocadoAppearance
  geometry IndexedFaceSet { ... }
}

You can also hide the inlined glTF model (if you only want to extract subset of it). To do this, place Inline in X3D inside Switch (Switch by default hides the model; you could make it visible by switching Switch.whichChoice). Like this:

Switch {
  children DEF InlinedAvocado Inline {
    url "glTF/Avocado.gltf"
  }
}
IMPORT InlinedAvocado.CastleEncoded_2256_Avocado_d AS AvocadoAppearance

Shape {
  appearance USE AvocadoAppearance
  geometry IndexedFaceSet { ... }
}