URL improvements: castle-config:/ protocol, use UriExists and FindFiles on any URL (including e.g. zip), TCastleMemoryFileSystem

Posted on

Scene inspired by Metal Gear Solid from Sketchfab https://sketchfab.com/3d-models/chibi-gear-solid-a4ca0e3864e143af9bf1dffc7fc8029b

We present a number of improvements to our URL system, which is the basis for all engine resource loading and saving:

  1. We introduced castle-config:/ URL protocol, to save files in a system-specific place (directory) that is “persistent” for given user.
    • It points to a specific directory that is by convention writeable and should be used to save user data. For example, something like C:/Users/<username>/AppData/Local/<application_name>/ on Windows or /home/<username>/.config/<application_name>/ on Linux and FreeBSD. We follow system-specific conventions and APIs.
    • This replaces previous ApplicationConfig routine.
    • It is consistent with castle-data:/ which is read-only location with your application data.
  2. We added TCastleMemoryFileSystem – useful to create a temporary filesystem, living only in memory, registered at given URL.

    It is, for now, our implementation of castle-config:/ on the web — so the data is not persistent if you reload the page, but at least it makes sense within a single session. This made our portable editor possible. Of course it is a TODO to actually persist the data using WWW browser mechanisms.

    That said, TCastleMemoryFileSystem may be useful for your own purposes too, if you want a temporary virtual filesystem in your application.

  3. RegisterUrlProtocol exposes now a class that allows to configure URL behavior. (So we can extend the functionality offered by custom URLs, aka virtual file systems, without adding more and more parameters to RegisterUrlProtocol.)

  4. Many built-in URLs, and URLs pointing to ZIP (using TCastleZip), and URLs pointing to TCastleMemoryFileSystem, use above, so their resources can be tested using UriExists and searched using FindFiles.

  5. Queries like UriExists('castle-data:/my_texture.png') on the web are also useful now, as we use castle-data:/auto_generated/CastleDataInformation.xml to test data files existence.

What’s the image shown at this news post? Chibi Gear Solid by glenatron on Sketchfab, cool 3D scene rendered using our Castle Model Viewer. It has no relation to this news post (except that’s it’s rendered using our engine and you can download it too and turn into a game right now!), just something pretty I wanted to show 🙂

Notable Replies

  1. Would the TCastleMemoryFileSystem work between different programs running on the same machine?

  2. It uses memory stream to hold the data so you would implement memory sharing between your app and the server. However, as the path can be either a real file system on a disk, or an URL, I think you could connect the client to your terrain server simply by HTTP request - you implement kind of “localhost://query” (registered as “my_terrain:/” protocol) that handles GET to get the chunk of terrain and POST if you need to save the changes made by a player. Good thing about it is that you don’t need to develop own sockets anymore, it’d be just a web page… I may be wrong tho :slight_smile:

  3. I already have socket mechanism for terrain which seems to work pretty well. I was thinking if shared memory file between applications running on same machine could be simpler way for handling things like images and models.

  4. Indeed the storage of TCastleMemoryFileSystem is just a TMemoryStream – so it’s just something that lives in the memory of “this process”, and only this process, it’s not shared. You could try to fork and modify the TCastleMemoryFileSystem mechanism to synchronize the underlying TMemoryStream in some way, but it’s likely both inefficient and more work than just inventing your own URL :slight_smile:

    If you want to use the URLs for synchronizing, you should invent and implement your own URL protocol, it’s starts easy – just use RegisterUrlProtocol :slight_smile:

Continue the discussion at Castle Game Engine Forum

Participants

Avatar for DiggiDoggi Avatar for edj Avatar for michalis