Chapter 9. Implementation

First define a function Plug. It is responsible for actual text processing that makes the plug functions correctly called. The argument PlugValue is scanned for all PLUG_xxx function definitions.

The argument CompleteCode is searched for the matching /* PLUG: xxx ... */ comments. The final shader (for given shape) in FinalShader is also searched for the /* PLUG: xxx ... */ comments. In practice, CompleteCode given here is either the FinalShader or the shader for a specific texture or light source.

Appropriate calls and forward declarations are inserted to the CompleteCode. In the process, all handled PLUG_xxx functions inside PlugValue are also renamed to unique names, since a single plug may be overridden by many effects. The modified PlugValue is inserted to the CompleteCode as well. Effectively, the caller can usually “forget” about the PlugValue afterwards — it has been processed, and correctly merged with the CompleteCode.

type
  TShaderType = (vertex, geometry, fragment);
  TShaderSource = array [TShaderType] of a string list;

var
  { shader for the whole shape }
  FinalShader: TShaderSource;

procedure Plug(
  EffectPartType: TShaderType;
  PlugValue: string;
  CompleteCode: TShaderSource);

var
  PlugName, ProcedureName, PlugForwardDeclaration: string;

  { Look for /* PLUG: PlugName (...) */ inside
    given CodeForPlugDeclaration.
    Return if any occurrence found. }
  function LookForPlugDeclaration(
    CodeForPlugDeclaration: string list): boolean;
  begin
    Result := false
    for each S: string in CodeForPlugDeclaration do
    begin
      AnyOccurrencesHere := false
      while we can find an occurrence
        of /* PLUG: PlugName (...) */ inside S do
      begin
        insert into S a call to ProcedureName,
        with parameters specified inside the /* PLUG: PlugName (...) */,
        right before the place where we found /* PLUG: PlugName (...) */

        AnyOccurrencesHere := true
        Result := true
      end

      if AnyOccurrencesHere then
        insert the PlugForwardDeclaration into S,
        at the place of /* PLUG-DECLARATIONS */ inside
        (or at the beginning, if no /* PLUG-DECLARATIONS */)
    end
  end

var
  Code: string list;
begin
  Code := CompleteCode[EffectPartType]

  HasGeometryMain := HasGeometryMain or
    ( EffectPartType = geometry and
      PlugValue contains 'main()' );

  while we can find PLUG_xxx inside PlugValue do
  begin
    PlugName := the plug name we found, the "xxx" inside PLUG_xxx
    PlugDeclaredParameters := parameters declared at PLUG_xxx function

    { Rename found PLUG_xxx to something unique. }
    ProcedureName := generate new unique procedure name,
    for example take 'plugged_' + some unique integer

    replace inside PlugValue all occurrences of 'PLUG_' + PlugName
    with ProcedureName

    PlugForwardDeclaration := 'void ' + ProcedureName +
    PlugDeclaredParameters + ';' + newline

    AnyOccurrences := LookForPlugDeclaration(Code)

    { If the plug declaration is not found in Code, then try to find it
      in the final shader. This happens if Code is special for given
      light/texture effect, but PLUG_xxx is not special to the
      light/texture effect (it is applicable to the whole shape as well).
      For example, using PLUG_vertex_object_space inside
      the X3DTextureNode.effects. }
    if not AnyOccurrences and
       Code <> Source[EffectPartType] then
      AnyOccurrences := LookForPlugDeclaration(Source[EffectPartType])

    if not AnyOccurrences then
      Warning('Plug name ' + PlugName + ' not declared')
  end

  { regardless if any (and how many) plug points were found,
    always insert PlugValue into Code. This way EffectPart with a library
    of utility functions (no PLUG_xxx inside) also works. }
  Code.Add(PlugValue)
end

Using the Plug function, we can create the EnableEffects function. It handles the effects list, correctly processing it and adding to the given CompleteCode..

procedure EnableEffects(
  Effects: list of Effect nodes;
  CompleteCode: TShaderSource);
begin
  for each Effect in Effects do
    if Effect.enabled and
       Effect.language matches renderer shader language then
      for each EffectPart in Effect.parts do
        Plug(EffectPart.type, GetUrl(EffectPart.url), CompleteCode)
end

Using the above functions, we construct the final shader code for given shape. All the effects (including effects specific to lights and textures) are correctly applied by the algorithm below. Specific requirements of the geometry shaders are also taken into account.

FinalShader := new TShaderSource
set FinalShader to basic rendering code
HasGeometryMain := false

At the beginning, FinalShader it set to a simple code that renders 3D object with no lights, no textures and no effects. The code contains magic /* PLUG: xxx ... */ comments, which will allow to enhance it in the following steps.

The real GLSL shader code used by our engine at this step may be found in our engine sources. See http://svn.code.sf.net/p/castle-engine/code/trunk/castle_game_engine/src/x3d/opengl/glsl/template.vs for the vertex shader, http://svn.code.sf.net/p/castle-engine/code/trunk/castle_game_engine/src/x3d/opengl/glsl/template.fs for the fragment shader and http://svn.code.sf.net/p/castle-engine/code/trunk/castle_game_engine/src/x3d/opengl/glsl/template.gs for the geometry shader.

Note that our default geometry shader contains only a set of utility functions, without a main() entry. It will be discarded later if no geometry shader code with a main() definition will be found in the user shaders. That is, if HasGeometryMain will remain false. See Chapter 6, Extensions for geometry shaders for reasons of this behavior.

if a complete custom shader code is provided then
  FinalShader[fragment] := empty
  FinalShader[vertex] := empty
  FinalShader := FinalShader + custom shader code
  HasGeometryMain := custom shader code contains some "GEOMETRY" shader

The X3D file may contain shader code that should replace the default shaders, following [X3D Shaders] specification. For example, a ComposedShader node may be present with a complete GLSL code. We use it at this step.

This step trivially allows the ComposedShader code to also contain plug declarations, like /* PLUG: xxx ... */. The same plug names as our default names may be used (like fragment_modify and so on), in which case the same user effects will be useful with the custom shader. This even allows the browser to add some internal effects (like shadow maps) to the custom shader template.

Alternatively, the ComposedShader may have a completely different approach to rendering. Then it may expose a completely different set of plug names, reflecting a different set of parameters to control.

Note again the special treatment of geometry shaders. Our default geometry shader code (which should contain our utility functions) is always kept. We also update HasGeometryMain.

for each Light in shape.Lights do
  LightShader := new TShaderSource
   set LightShader to basic lighting code (optimized for this Light)
   EnableEffects(Light.Effects, LightShader)
   LightContribution := LightShader.ExtractFirst
   Plug(fragment, LightContribution, FinalShader)
   FinalShader := FinalShader + LightShader

A little care is needed to correctly add light sources. Remember that lights may have user-defined effects that should be applied only to the specific light source. That's why we temporarily keep the shader code specific to a given light source in a separate LightShader variable.

for each Texture in shape.Textures do
  TextureShader := new TShaderSource
  set TextureShader to basic code (optimized for this Texture)
  EnableEffects(Texture.Effects, TextureShader)
  TextureApplication := TextureShader.ExtractFirst
  Plug(fragment, TextureApplication, FinalShader)
  FinalShader := FinalShader + TextureShader

Texture effects require a similar approach as light effects, to correctly catch all the ways how plugs may be used.

We start by creating a default shader code that applies the texture, knowing the texture type (2D, 3D, cube map), texture mode (multiply, add and such) and other properties. It should follow all X3D texturing and multi-texturing requirements. The code should define a function named PLUG_main_texture_apply that can be later connected to the final shader. It should also declare plug named texture_color, that can be used by user effects for this texture.

There are actually more differences between the application of the light and texture effects. Section 9.2, “Correct shadows from multiple light sources” describes one feature that makes their logic slightly more complicated.

EnableEffects(appearance node.Effects, FinalShader)

for each group node containing this shape do
  EnableEffects(group node.Effects, FinalShader)

Effects specific to a given shape, and effects for all groups containing this shape, are applied.

The effects at this point may override also lights and textures plugs, like light_scale. That's simply because we have already added all the lighting and texturing shading code to the FinalShader. Overriding light_scale at this point means that we scale the contribution of every light by the same function.

if HasGeometryMain then
  for each FragmentPart in FinalShader[fragment] do
    FragmentPart := '#define HAS_GEOMETRY_SHADER' + FragmentPart else
  FinalShader[geometry] := empty

Decide if we really want to link geometry shaders, based on whether we have main() for geometry shaders. If yes, then HAS_GEOMETRY_SHADER symbol has to be defined, as it may be useful for fragment shader authors. Otherwise, discard all geometry shader code.

At the end, FinalShader is just a collection of strings forming a shading language source code. For GLSL, each string is naturally a “separate compilation unit”, and can be compiled in isolation. For other shading languages, the parts may be simply concatenated together as necessary.

The full, actual source code of this operation is available in our engine sources, see the unit GLRendererShader. Source code is on http://svn.code.sf.net/p/castle-engine/code/trunk/castle_game_engine/src/x3d/opengl/glrenderershader.pas.