Roadmap

Table of Contents

1. Introduction

Don’t see the feature you need? Tell us on the forum or Discord.

If you would like to see some feature implemented sooner, please support the engine development on Patreon!

2. VR and AR

VR: Michalis has access to Oculus Quest since the beginning of 2022. Port most likely using OpenXR (to include the widest possible range of devices using a single cross-platform API).

Oculus Quest 2: Developer Mode for Castle Game Engine enabled!

AR: most likely using Android / iOS built-in frameworks, as these make AR widely available on modern phones.

3. (planned for 7.0) New material components (allowing also material editing in the editor)

We want to allow you to edit materials in the CGE editor,

  • e.g. to modify (or completely override) materials from TCastleScene (like glTF),

  • to have full control over primitives (like TCastleSphere or TCastleImageTransform look),

  • to enable you to use cool CGE features like mirrors, shader effects easier.

This also means exposing new set of Pascal components to manage materials easier.

4. More Delphi platforms

See about existing Delphi compatibility. We plan to support more platforms with Delphi: Android, Linux, iOS…​

5. Load and animate glTF more efficiently

We should read meshes in a way that keeps binary layout of data in glTF, so that it is ultra-fast to load, and it is loaded almost straight from glTF to GPU.

Some modifications to CGE may be required here (CastleGeometryArrays will have to be made more flexible for this). New X3D nodes should be introduced, like BufferGeometry (same as X3DOM?).

We should also perform skinning fully on GPU, avoiding CoordinateInterpolator done on CPU.

6. More physics

More physics engine integration. The details are listed in the manual about physics, subsection "TODO". In short:

  • TCastleWalkNavigation should support physics out-of-the-box.

  • Older physics based on octrees should be phased out.

  • TCastleSceneCore.ExposeTransforms should work both ways, to allow to attach bodies and colliders to parts of the scene. This will enable e.g. ragdolls.

  • We should add more physics backends, like PhysX or Bullet.

7. (IN PROGRESS) Web platform (rendering using WebGL) support

Using FPC support for the WebAssembly:

Some "glue" may also be done using pas2js. This is a nice solution to expose to our WebAssembly code the JS API we need, like WebGL. Alternatively, we could just do it in plain JS.

This is already in-progress by Trung Le (Kagamma), Andrzej Kilijański and Michalis.

I wrote a longer post about plans and status of this here: https://castle-engine.io/wp/2023/04/08/web-target-progress-and-plans/ .

8. (IN PROGRESS) Particle systems

With a designer. Example usage: blood, snow, rain, fire, smoke…​ 2D, 3D.

This is already in-progress by Trung Le (Kagamma). Use cge-3d-particle-emitter or Effekseer Integration. See CGE components.

9. (IN REVIEW) Steam integration, and engine available as Steam app

Official Steam integration in CGE, see https://trello.com/c/RNDjQhqe/71-steam-headers . We already used it for The Unholy Society, we want to either “just publish that approach” or propose a new, improved (without an intermediate “wrapper” library) approach.

We should also upload CGE as app to Steam.

This is already in-progress by Eugene Loza (branch: https://github.com/castle-engine/castle-engine/commits/castle-steam , PR: https://github.com/castle-engine/castle-engine/pull/471 ), we also have a draft application on Steam for CGE :).

10. Release mobile view3dscene (as Android and iOS application)

Associated with glTF, X3D and other formats that the view3dscene (and Castle Game Engine) handles. Available in the App Store / Google Play Store. For free or for 1 USD (not sure yet; but definitely without ads, I really dislike ads).

11. Terrain designer

Easily design a height map (X3D TElevationGridNode node, with trees, rocks, grass). Saved and loaded as an X3D file.

Implementing this feature will most likely be split into a couple of small releases:

  • Edit the heights.

  • Edit the grass and trees and rocks.

  • Efficiently render huge amounts of grass and trees and rocks.

Implement nice shaders to show it, like this BareGame example.

What we have now: The engine includes a unit CastleTerrain to generate terrains in various ways (most notably, from smoothed random noise). We have examples/terrain/ demo to play around with it. We have the "Wyrd Forest" game that also uses CastleTerrain, and also includes a simple editor of terrain settings.

What we need: Visual, interactive editor for the ElevationGrid (the thing you get from TTerrain.Node call). To make hills, valleys in a comfortable way. And comfortably plant there stuff (rocks, grass, trees…​).

12. Vulkan renderer

Add a Vulkan renderer.

  • API:

    Almost all public CGE API will remain unchanged. For a long time now, we design our API to be independent from OpenGL, and all rendering (e.g. done by TCastleScene, TCastleViewport, TCastleUserInterface, TDrawableImage) is driven by a renderer-neutral API.

    Shaders will continue to be available using GLSL. There will be possibility to precompile them to SPIR-V before usage (for both Vulkan and new OpenGL, as new OpenGL also supports SPIR-V).

    Underneath, we will create a "pluggable" architecture for renderers. In theory we could implement a lot of renderers (e.g. also Metal or Direct3D) but given the limited resources, realistically we will focus on 2 renderers: Vulkan and OpenGL ("OpenGL" here meaning really a family OpenGL + OpenGLES + WebGL).

  • Approach:

    We have a simple example code that shows how you can start a new renderer: see new_renderer_skeleton example. This is great for testing initial implementation, before we have a "pluggable" architecture.

  • Header:

    I was playing with PasVulkan from which we could use a Vulkan API header. See news about PasVulkan.

    We could also get Vulkan header from GLAD 2 if only we port it to Pascal (we plan this already).

  • Platforms:

    Not all platforms support Vulkan, in particular it’s uncertain whether Apple with ever support it (they want to push Apple-specific Metal API). CGE will most likely use MoltenVK to render using Vulkan on Apple, see plans regarding macOS / iOS and OpenGL, Vulkan.

13. (IN REVIEW) Distance field fonts

See about Distance field fonts. See code from Chris Rorden showing how to do it in Lazarus.

14. Mobile

14.1. Wireframe on OpenGLES

Since glPolygonMode is not available at all on OpenGLES, we need to be able to generate wireframe geometry arrays on CPU, adding a "wireframe" option to CastleArraysGenerator that will generate a different (line) primitive for TGeometryArrays. Then the renderer can use such TGeometryArrays for OpenGLES.

This is started now, by having Shape.shading = "WIREFRAME" option, see Shape.shading field. We need to

  • make it available for all geometry shapes

  • make this feature more flexible, so that the renderer can switch between wireframe/non-wireframe rendering without any cost.

  • And then we can implement Attribute.WireframeEffect on OpenGLES correctly,

  • and allow to set Wireframe on specific viewports (so that some viewports may view wireframe, independent of others) on OpenGLES.

14.2. On-screen keyboard on iOS (done on Android)

14.3. TGLImage.GetContents on OpenGLES

TGLImage.GetContents is not available on OpenGLES, as it doesn’t support glGetTexImage. Fixing it is a matter of implementing alternative way that draws texture to FBO and grabs using SaveScreenGL (using glReadPixels internally).

One has to use TGLRenderToTexture to render quad with a texture (making sure the framebuffer size, quad size, and texture size match, and without filtering) and then use glReadPixels to get the pixels from FBO. This will work on mobile (and also on desktops, although it will be less optimal alternative to glGetTexImage).

15. macOS: Package engine as one big application bundle

This will:

  • be simpler to users, as the engine will be "one single application" for macOS users.

  • avoid troubles resulting from Apple translocation. It right now prevents the CGE editor from detecting CGE path.

  • avoid current duplication of castle-editor/data in distributed package, thus making it smaller by ~60 MB.

See macOS.

16. macOS: Have signed app bundle for the editor

This will allow to run CGE editor on macOS by just double-clicking it, out-of-the-box.

See macOS.

We should also enable developers to easily sign their own applications packaged with CGE build tool.

17. (IN PROGRESS) fps_game should be redesigned using editor, CreatureCreatures / CastleResources / CastleLevels / CastleItems / CastlePlayer should be replaced with a better API

In progress. See news post. On branch fps-game-upgrade.

These 5 units (CreatureCreatures / CastleResources / CastleLevels / CastleItems / CastlePlayer) expose a high-level API, that sits on top on existing classes (like TCastleScene and TCastleTransform). But I am not 100% happy with their API. Reasons:

  • The API is too specialized at some points (3D games with creatures / items pickable),

  • It is confusing how it maps to API underneath (e.g. TPlayer somewhat controls the TCastleNavigation).

Gradually I will want to express their features in different ways, in ways that are more closely and obviously connected with TCastleScene / TCastleViewport / TCastleNavigation. Basically, I’m very happy with current API of TCastleScene and TCastleTransform, and I want to make it more obvious how it is related to creatures/placeholders and other concepts introduced in CastleLevels and CastleCreatures and CastkeResources. Currently they use TCastleScene and TCastleTransform inside, but the relationship is non-obvious.

It’s not going to happen any time soon, but it will happen gradually, over the course of next 1 or more years. That is, some of the features of the 5 units mentioned above will become available in other ways too (more flexible, and more obviously connected to TCastleScene).

I know this sounds vague, because I have not yet clarified these plans in my head:) These 5 units are useful, they provide various features on top of TCastleScene. I’m not going to just "ditch" them or even deprecate them, before I made a better API that also has these features. For example:

  • New TCastleScene.Load should be able to take a callback (or some class instance) to easily perform the "placeholders" functionality of TLevel.Load in a flexible manner (user can decide what to replace with what).

  • There will be TCastleSceneView that provides part of the functionality of T3DResource (multiple TCastleSceneView share a single TCastleScene but can show different animation frame of it), but without some often-unnecessary "baggage" from T3DResource (like refcounting of T3DResource, and it’s special Prepare/Release methods).

  • It should be possible to add an A.I. (like TWalkAttackLogic) to any TCastleTransform instance.

I know I want to go in this direction. Based on the questions (including on Discord) I see that the API of these 5 units is not clear to people. It wraps TCastleScene / TCastleViewport / TCastleNavigation, but in ways that are not obvious, and that is something I want to improve.

Again, bear in mind that it will not happen any time soon :) You can safely and happily use these units, probably for a few years to come.

But it is something I think about for the future, and it may explain some of my decisions. E.g. that is why I don’t plan special integration of TCastleCreature with castle-editor. Instead I want to enable you to add future TWalkAttackLogic to any TCastleTransform one day. And thus we will have "easy creatures" in CGE but with more flexible API.

18. Bridge TDrawableImage with TCastleScene, enable drawing UI into TCastleScene, to have UI rotated in 3D with perspective.

19. MDL and/or MaterialX integration

We can make 3D modeling of realistically-looking stuff much easier by reusing libraries of realistic materials.

20. USD – Universal Scene Description support

Explore possibility to support USD – Universal Scene Description format as a first-class citizen, alongside glTF and X3D, in CGE.

21. Integration with Nakama (scalable server for social and real-time games and apps)

Integration with Nakama:

Nakama is an open-source server and client library/API that provide networking capabilities typically needed in games.

  • The server is written in Go (see source code) and can be extended using Lua, JS, or Go modules. See server framework.

  • It supports out-of-the-box stuff like user accounts, matchmaking (connect people into rooms where the game happens), leaderboards, chat, etc. There’s infrastructure to synchronize data between players in a multi-player game.

  • Client library can be used from various game engines and languages so we’re confident it can be used from CGE too.

  • You can host the server yourself or pay to get ready hosting.

  • As you can see in above links, it seems really well documented, and actively maintained. These are naturally critical qualities for our dependencies.

I think Nakama may give CGE networking capabilities comparable with some popular solutions known from other game engines :)

Allowing to trivially get multi-player functionality in your games.

22. Project settings editable using GUI

We will expose both current CastleEngineManifest.xml and data/CastleSettings.xml as something you can edit yourself with GUI.

But we don’t want to create and maintain a custom GUI for them, instead the plan is to:

  • Create Pascal components with published properties representing their data.

  • Make the project settings files just "JSON design files" for the editor.

So you’ll have a file like project.castle-project and using editor menu item like "Project Build Settings" will really just open a design project.castle-project (JSON) in the editor, and it will be just a serialized TCastleProject component. Same for "Project Runtime Settings" which edits data/settings.castle-project-settings.

Bonus: we’ll associate *.castle-project with the system, to open when you double-click it e.g. from Windows installer.

Bonus: you can add non-visual components to data/settings.castle-project-settings and load them using Container.DesignedComponent('xxx'). This is great e.g. to add sounds or fonts or other data that you want to later load in CGE, from any view.

There are a few details that have to be ironed out here:

  • how should the new equivalent of CastleEngineManifest.xml and data/CastleSettings.xml interact. We will likely keep 2 files, project.castle-project and data/settings.castle-project-settings, as

    • Some information doesn’t need to be packed into data, part of it could even be considered "somewhat secret" (e.g. SDK IDs for Android analytics should not be present anywhere in the non-Android builds).

    • Some information should be packed into data automatically, by default, everything. To enable that Bonus: you can add non-visual components feature.

  • most intuitive names for all the things above (menu item names, component names, extensions etc.) :)

23. Creating animations in CGE editor, for UI and transforms

Allow designing and testing animations in CGE editor.

The features cover animating typical UI/transform things that you see in CGE editor directly: e.g. UI stuff (like TCastleUserIterface.Translation, TCastleImageControl.Color), 3D stuff (TCastleTransform.Translation, TCastleTransform.Rotation etc.).

Note
Animating everything in CGE is already possible right now using code. You can update any values you want in Update method of your view and change any value you need (like MyControl.Translation) using routines like Lerp or SmoothStep.

The point of this feature is to allow to design and see simple animations from editor. Code will then be able to just play them (e.g. using simple method MyAnimation.Play) whenever needed.

Features:

  • Make it possible to animate (at least!) any property of TCastleUserInterface and TCastleTransform. Access the property using RTTI, and at design-time in editor allow to choose TargetObject and TargetProperty with suitable type.

    Bonus: allow to animate any property of any TComponent.

  • It should allow to animate at least any Single/Double, TVector2/3/4 properties (this includes also colors, as TCastleColor[RGB] is just alias for TVector3/4; this includes also rotations, that is just TVector4 but can use slerp for interpolation).

    Bonus: allow to animate other property types. E.g. animating discrete (like boolean) actually makes sense — to hide stuff by animation, by changing Exists at specific point in time.

  • It should use TCastleBehaviorBase that will allow to attach it to any TCastleUserInterface and TCastleTransform. See talk about it in https://github.com/castle-engine/castle-engine/issues/473 .

  • Each such animation should define:

  • Bonus: Group animations, so that from code you can fire like MyUserInterfaceAppear.Play and it runs a number of animations at once — e.g. something "slides in" (translation changes), something appears (Exists changes), something fades in (Color.alpha changes)

24. Screen effects - expose as nicer components

Address a few shortcomings of our (see examples/screen_effects_demo/, screen effects):

  • Change terminology. "Screen effect" is non-standard, "post-processing effects" are better.

  • Expose as components some standard effects, like blur, changing colors in HSV, grayscale. They should have easy Pascal properties, serializable to JSON, and wrap the related X3D nodes underneath.

  • Expose as component a customizable shader effect allowing you to provide GLSL code in a text box (TStringList editor) in CGE editor.

  • This depends on making a more generic TCastleComponentEditorList to expose TCastleScreenEffects.ScreenEffects list nicely.

25. Export any TCastleTransform hierarchy to X3D

Currently you can only export TCastleScene to X3D.

26. Export TCastleScene, or any TCastleTransform hierarchy, to glTF

Currently you can only export TCastleScene to X3D.

This will likely also imply rename of tovrmlx3d to castle-convert.

27. Allow animating selected node graph properties, without changing the graph, to be able to reuse graphs

Right now our X3D graph usage is similar to HTML DOM usage from typical web applications: we modify the graph at runtime, everything that changes stuff (like animations) just changes the graph. This is simple to use, it nicely allows to animate everything in X3D and gives developers an easy way to modify everything at runtime.

The downside is that we waste significant memory, and loading time, in some cases. If you want to play different animations on each TCastleScene, then they need to have a different graph even though they are loaded from the same URL. Hence using e.g. TCastleScene.Clone or TCastleSceneCore.Cache really internally makes multiple copies of the graph. This could be more optimal in the future. E.g. assuming your model only uses glTF skinned animation or animates simple transformation position/rotation/scale, we could make the graph read-only, and only maintain small "diff" information at each instance.

28. FMOD Studio support

Right now our FMOD backend uses only "FMOD Core API".

We want to enable using higher-level FMOD events that drive the sound banks exported from "FMOD Studio".

29. Wwise backend

Allow using AudioKinetic’s Wwise as sound backend, and as a middleware to design sounds.

30. Asynchronous loading of assets

As described in threads, we want to allow loading assets asynchronously. This means that you use CGE API from one thread, but some operations are not "blocking" and CGE can internally use threads to finish their work, while your code continues to calculate, CGE events like Update continue to be processed etc.

For example:

TCastleScene.LoadAsync(const Url: String; const OnFinish: TNotifyEvent);

TCastleScene.PrepareResourcesAsynchronously;

TCastleTerrain.UpdateDataAsync(const NewData: TCastleTerrainData; const OnFinish: TNotifyEvent);

We already use such approach e.g. with TCastleDownload. Internally it can download in a thread, but you don’t need to worry about inherent "hard" issues with threads (synchronizing communication between threads, being careful who/when touches the shared variables). This approach results in:

  • Really comfortable API. You handle everything from one thread, you get simple callbacks when things are done, you can simply watch (e.g. in some Update) status of something, display progress bars, interrupt the operation etc.

  • Flexible implementation underneath. It can use Pascal threads underneath, or it can delegate work to another asynchronous API (e.g. to download using Java classes on Android), it can even fallback to synchronous if threads are not possible on some platform.

And it is possible to load OpenGL things in a thread: by having separate OpenGL context for the background thread, that shares resources (like VBO).

The ultimate demo is to implement exploring a huge city and loading neighboring pieces of the city asynchronously.

I mentioned this in the bottom part of Slides, movies and thoughts from my GIC 2022 presentation .

31. Merge TCastleImagePersistent and TDrawableImage

One less image class (we have too many image classes, see https://castle-engine.io/using_images ).

And it will avoid current counterintuive fact:

ImageControl.DrawableImage.CustomShader := Xxx;

The above is ignored, because ImageControl.CustomShader (equal to ImageControl.Content.CustomShader) overrides TDrawableImage.CustomShader. Similarly a few other drawable image properties, as TCastleImagePersistent.Draw does now:

 { All the DrawableImage properties must be set here,
    in case muliple TCastleImagePersistent refer to the same
    DrawableImage instance through the cache.
    Fortunately, almost all TDrawableImage properties have zero cost
    of change. }
  DrawableImage.SmoothScaling := FSmoothScaling;
  DrawableImage.Clip := FClip;
  DrawableImage.ClipLine := FClipLine;
  DrawableImage.Rotation := FRotation;
  DrawableImage.RotationCenter := FRotationCenter;
  DrawableImage.Alpha := FAlphaChannel;
  DrawableImage.Color := FColor;
  DrawableImage.CustomShader := FCustomShader;

32. (IN PROGRESS) Gamepads (joysticks) support

Goal: Support modern gamepads (joysticks) with a nice API.

Note: We have support for gamepads (joysticks) already, see CastleJoysticks and examples/joysticks/ . But it is not easy to use — I’d like to improve the API, and improve it to handle various gamepad types in a more "seamless" fashion.

Features:

  • It should support at least the most popular controller, which X-box compatible controller, out-of-the-box.

    It seems we can easily support much more, using joysticks database: https://trello.com/c/483EsTNX/12-use-database-of-joysticks-to-better-understand-their-buttons-axis . But I make it explicit: the critical feature is support for the most common X-box compatible controller. Support for, potentially, every gamepad that exists in the world is secondary :)

  • Should support at least Windows, Linux, and Nintendo Switch. Switch support in this case is justified — because we already support Switch gamepads, but we’ll want to upgrade it to new API.

    Of course support for more platforms is desired. macOS, and maybe even mobile (it does seem one can connect gamepad to Android through at least Bluetooth, it actually works). But these are secondary. The platforms mentioned above are primary for this feature.

  • Should support multiple gamepads.

  • API should be nice. Current CastleJoysticks is complicated. This means:

    • TInputPressRelease structure, as send to TCastleUserInterface.Press (so in particular, TCastleView.Press) should have a type like GamePadButton (alternative to key on keyboard, or mouse botton) that occurs when user presses / releases the button. It should say which gamepad was used to press it (to support multiple joysticks connected to system). And it should report a button.

      Buttons should have agnostic names, like North, East etc. Avoid using names like ABXY that are in different places on different gamepad models (Xbox, PS, Nintendo…​). Maybe we can also have names like Accept, Cancel to follow conventions (e.g. on Xbox controller, A corresponds to "Accept" by convention, B corresponds to "Cancel").

    • TCastleInputAxis (that is also already in progress) should allow to observe chosen gamepad axis (based on gamepad index, and choice horizontal/vertical, also choice of gamepad axis — typical gamepads now have 2 axis). This is already a nice API to observe input that we want to have.

33. Smart Tasks

Allow to define tasks that should occur regularly, but do not have to be done every frame. The engine should then automatically execute the tasks based on their need and current workload.

E.g. animation task may say "I want to update the model every frame, but actually I don’t have to — if under heavy workload it’s enough to only update 30 times per second or such". This would be a better (more general, smarter — activated automatically) way of optimizing it than current TCastleScene.AnimationSkipTicks.

Note

The API (terminology) and implementation have to play nicely with Delphi existing concept of "tasks" from Delphi Parallel Programming Library. Delphi already has a "task" meaning "a unit of work that can be executed in auto-created thread (using pool of threads underneath)", see

So:

  1. Our tasks, that are mainly to execute tasks in the main thread but with some delay ("execute this when convenient, I don’t need it done right now") should be clearly something different. To avoid confusing API for people that know Delphi tasks.

  2. When using threads, then we should use Delphi tasks (on Delphi), where possible. Only for FPC we’ll need to duplicate subset of it (unless FPC will get the compatible implementation).

34. Improve automatic reloading of files (in editor) when they changed

Automatic reloading (CastleInternalFileMonitor) now:

  • Doesn’t capture changes to secondary files, e.g. to textures used in glTF file. We only watch the file indicated by TCastleScene.Url now.

  • When TCastleScene.Cache, the file will not be reloaded from disk correctly.

  • Not all components implement necessary watching logic. For now only TCastleSceneCore. All components should eventually do it, e.g. TCastleTiledMap, TCastleImageTransform, TCastleImageControl.


To improve this documentation just edit this page and create a pull request to cge-www repository.