Data directory and castle-data:/ URL protocol

Table of Contents

1. Introduction
2. Features
3. Example things to put in the data subdirectory
4. Data should be considered read-only
- 4.1. Use ApplicationConfig instead to write stuff
5. Advanced: How do we determine the data directory?

1. Introduction

The data subdirectory of a Castle Game Engine project is special. This is where you should put all the files that your application will load during runtime. This includes designs (usually created using editor), 3D models, images, sounds, and any other data files that you need.

Load these files using a special URL protocol castle-data:/….

2. Features

This directory is automatically correctly packaged by the build tool and editor. E.g. it will be correctly added to the Android apk file, iOS or Nintendo Switch application.
It is detected in a smart way. E.g. it allows to place your data files in a system-wide location on Unix. The details are below.
The detection can be customized using the ApplicationDataOverride global variable. Though we recommend to not touch it — the default algorithm was designed to account for a lot of normal use-cases and has been adjusted to account for specifics of every platform.

Note	You are not limited at runtime to reading only files from this `data` directory, of course. You can read any file on disk (using regular filenames or `file:/…` URLs) at runtime. See network docs.

3. Example things to put in the `data` subdirectory

Designs made by editor. A typical view has this in constructor:

constructor TViewMain.Create(AOwner: TComponent);
begin
  inherited;
  DesignUrl := 'castle-data:/gameviewmain.castle-user-interface';
end;

Game 3D and 2D models, loaded e.g. by
```
MyScene.Load('castle-data:/my_model.x3d');
```
See loading 3D and 2D models in viewports.

2D images, loaded e.g. by

MyImageControl.Url := 'castle-data:/my_image.png';

See displaying images.

Sounds, loaded e.g. by

MySound.Url := 'castle-data:/my_sound.wav';

See loading sounds.

… and really anything else you plan to load during the game. Your custom files can be loaded using
```
MyStream := Download('castle-data:/my_binary_file');
```
or
```
MyTextReader := TTextReader.Create('castle-data:/my_text_file.txt');
```
See loading from URLs.

4. Data should be considered read-only

The resources accessed using the castle-data:/ should be treated as read-only. That is, do not attempt to modify them.

While sometimes you can actually modify them (in particular, they are just regular files, owned by the current user, when you distribute your application as a simple .zip on Windows or Linux) but for portability we advise to treat them as read-only. This means that your application will continue to work on Android, iOS, or when users install it "system-wide" on desktops e.g. to c:/program files…/ on Windows. In all these scenarios, the castle-data:/ resources will be strictly read-only.

4.1. Use ApplicationConfig instead to write stuff

If you need to write some data, use the ApplicationConfig to get URL for writing configuration files. For example

var
  MyUrl: String;
  MyStream: TStream;
begin
  MyUrl := ApplicationConfig('my_data.txt');
  MyStream := UrlSaveStream(MyUrl);
  try
    // TODO: write to MyStream whatever you need
  finally
    FreeAndNil(MyStream);
  end;
end;

More ApplicationConfig notes:

The URLs you get from ApplicationConfig point to writeable resources.

If case of regular desktops, these are just files and opening them with UrlSaveStream does underneath just TFileStream.Create(FileName, fmCreate).
The contents of streams saved to ApplicationConfig are persistent across program runs. Use it for savegames, databases (e.g. using SQLite), user preferences, etc.

Our UserConfig is also saved there.

The ApplicationConfig('') is a directory that is initially (when user first runs the application) empty. But you can create as many files there as you want.

There is no special way to initialize it with some files before your first application run. If you want to initialize something in your config based on your data, do it explicitly — e.g. like this:

uses SysUtils, CastleClassUtils;

procedure MakeSureMyStuffExistsInConfig;
var
  OutputStream, InputStream: TStream;
begin
  if not UriExists(ApplicationConfig('my_stuff.data')) then
  begin
    OutputStream := UrlSaveStream(ApplicationConfig('my_stuff.data'));
    try
      InputStream := Download('castle-data:/my_stuff_initial.data');
      try
        ReadGrowingStream(InputStream, OutputStream, true);
      finally FreeAndNil(InputStream) end;
    finally FreeAndNil(OutputStream) end;
  end;
end;

The subdirectories inside ApplicationConfig('') are automatically created as you save files using UrlSaveStream, so saving e.g.MyStream := UrlSaveStream(ApplicationConfig('my_subdirectory/my_data.txt')); just works.

5. Advanced: How do we determine the data directory?

The algorithm to find base data directory is OS-specific. It searches a couple of common locations, using the first location that exists. We look inside standard user-specific directories, then inside standard system-wide directories, then we look for the data subdirectory in the current exe directory (under Windows) or in the current working directory (under other OSes).

Warning

The algorithm below is complicated. Don’t read this! Instead follow the short explanation: just place the files inside the data subdirectory of your project, and everything will work out-of-the-box.

The details how we currently choose the data directory:

Windows

data subdirectory inside our exe directory, if exists.
../../data relative to our exe location, if it exists and exe seems to be inside a subdirectory <platform>/<config>/.

Where <platform> matches current Delphi <platform> name (like Win32 or Win64 — this is combined OS and CPU) and <config> matches Debug or Release. This is deliberately adjusted to Delphi / C++ Builder default project settings, so that we detect data automatically when exe location follows Delphi conventions. This deliberately checks whether subdirectory names match <platform>/<config>/, to avoid picking up a random data subdirectory for unrelated project.
Otherwise: just our exe directory.

Warning
Don’t depend on this "last resort" fallback in your applications. Instead, place the data inside the data subdirectory.

macOS

Contents/Resources/data subdirectory inside our bundle directory, if we are inside a bundle and such subdirectory exists.
Otherwise, fallback on generic Unix detection, see below.

iOS

data subdirectory inside our bundle directory, if we are inside a bundle and such subdirectory exists.
Otherwise, fallback on generic Unix detection, see below.

Android

Always use Android assets packaged in APK. This is a special location on Android where application should store it’s assets.

Nintendo Switch

Always use special location on Nintendo Switch where application should store it’s data.

Unix (Linux, FreeBSD, macOS…)

~/.local/share/<ApplicationName>.

This is user-specific data directory, following the default dictated by basedir spec. If such directory exists, it is returned.

This is checked first, to allow user to always override system-wide installation of a program with his/her own installation. E.g. consider the situation when an old version of a program is installed system-wide in /usr/local/share/my_program/ , but some user (with no access to root account) wants to install a newer version of it for himself. This is possible, because ~/.local/share/my_program/ is checked before /usr/local/share/my_program/.
/usr/local/share/<ApplicationName>. If such directory exists, it is returned.

This is for system-wide installations without package manager.
/usr/share/<ApplicationName>. If such directory exists, it is returned.

This is for system-wide installations with package manager.
data subdirectory of the current directory, if exists.

This is easiest and comfortable for development, just keep the data subdirectory alongside the executable binary.

This is searched after system-wide specific dirs above, to avoid accidentally picking unrelated data in current directory instead of system-wide data.
Otherwise: As a last resort, we just return the current directory.

Warning
Don’t depend on this "last resort" fallback in your applications. Instead, place the data inside the data subdirectory.

To improve this documentation just edit this page and create a pull request to cge-www repository.