Configuration and Data Sources

Engine-owned documentation. This page explains reusable configuration parsing, runtime settings, mounted data sources, file lookup, and cache storage. Project-specific config values and content folder policy belong to the embedding project.

Purpose

Use this page when changing how the engine reads .fomain/config data, applies command-line or sub-config overrides, mounts resource directories/packs, reads files, or stores cached resource data.

For the executable project-authoring route, use Configure a Game Project. This page remains the reusable implementation/reference owner.

Read this together with:

• Build Workflow for configure/build entry points.
• Baking Pipeline for resource-pack production.
• Text and Localization for Baking.BakeLanguages, Client.Language, text-pack filename selection, and bake-time normalization.
• Font Formats And Text Layout for font descriptor raw-copy selection, referenced image lookup, client binding, and project-owned slot policy.
• Generated API and Metadata for generated settings and metadata inputs.
• Client Runtime, Server Runtime, and Tools for runtime/tool consumers.

Source paths inspected

• Source/Common/ConfigFile.h
• Source/Common/ConfigFile.cpp
• Source/Common/Settings.h
• Source/Common/Settings.cpp
• Source/Common/Settings.inc
• Source/Common/DataSource.h
• Source/Common/DataSource.cpp
• Source/Common/FileSystem.h
• Source/Common/FileSystem.cpp
• Source/Common/CacheStorage.h
• Source/Common/CacheStorage.cpp
• Source/Common/SettingsStorage.h
• Source/Common/SettingsStorage.cpp
• Source/Essentials/DiskFileSystem.h
• Source/Essentials/DiskFileSystem.cpp
• Source/Essentials/Platform.h
• Source/Essentials/Platform.cpp
• Source/Frontend/ApplicationInit.cpp
• Source/Client/Client.cpp
• Source/Client/Updater.cpp
• Source/Client/ResourceManager.h
• Source/Client/ResourceManager.cpp
• Source/Tools/Baker.h
• Source/Tools/Baker.cpp
• Source/Tools/ConfigBaker.h
• Source/Tools/ConfigBaker.cpp
• BuildTools/cmake/stages/Codegen.cmake
• BuildTools/cmake/stages/ScriptsAndBaking.cmake
• BuildTools/cmake/stages/Packages.cmake
• related tests under Source/Tests/

Layer map

1. Config text parser — ConfigFile parses sections, keys, values, repeated sections, optional collected content, and first-section reads.
2. Settings model — Settings.inc declares setting groups; Settings.* turns config files, command-line overrides, internal config, defaults, auto-settings, sub-configs, and resource-pack declarations into GlobalSettings.
3. Data-source abstraction — DataSource mounts disk directories and pack files behind a uniform file-list/open interface.
4. File-system view — FileSystem combines mounted data sources, exposes FileHeader, File, FileReader, and FileCollection, and resolves file reads by path/name.
5. Cache storage — CacheStorage persists named string/data entries for reusable cache consumers.
6. Settings store — SettingsStorage persists per-user tool/editor preferences (registry on Windows, file store elsewhere), scoped by application name.
7. Low-level disk access — DiskFileSystem performs direct disk operations below mounted engine resources.

Config parsing

Source/Common/ConfigFile.* owns syntax-level parsing. ConfigFileOption controls optional behavior:

• CollectContent preserves section content for consumers that need raw block text.
• SkipNestedSections parses only anchor sections and skips nested (/-addressed) section bodies — cheap header enumeration on files with large nested payloads (map files).
• Nested sections: a section name containing / is nested. ConfigFile recognizes only the syntax - names are stored verbatim and no prefix is ever resolved, so what a prefix means belongs to the consuming format. GetOrderedSections() exposes sections in file order, which is what a consumer needs to bind a nested section to the section it follows (the by-name multimap cannot express that, since repeated names collapse). SkipNestedSections parses only non-nested sections and skips nested bodies.
• ConfigFile takes only the content: no file identity, no parse callbacks, no format tokens. For map files, MapLoader owns the interpretation - [ProtoMap] declares a map named by its $Name or by the file, and a nested $Name/<Type> prefix binds content to the anchor above it.

The parser stores owned strings internally and returns string_view values from parsed sections. Consumers must not assume those views outlive the ConfigFile instance.

Runtime settings

The secure network channel adds ServerNetwork.ChannelSecretKey (the required static server secret) and ClientNetwork.ChannelServerKeys (client public-key pins). Both are startup settings, not writable script state. The server setting contains the 64-hex-digit key value, not a file path: $TARGET_FILE{...} reads a protected file at target runtime without embedding its contents during baking. An unset or malformed server key blocks server startup, even in an interthread-only profile; an empty or invalid client pin list blocks connection. See Networking and Security and Secrets.

Source/Common/Settings.inc is the central generated-like declaration file for setting groups and individual settings. Every built-in setting is addressed by its full Group.Name, is mutable only while GlobalSettings constructs the startup snapshot, and is exposed as const afterwards. Live state belongs to its runtime owner rather than being written back into settings. Settings.h exposes:

• ResourcePackInfo — name, input directories/files, include/exclude glob patterns, side flags, and baker list.
• SubConfigInfo — named config overlays and setting maps.
• GlobalSettings — combined client/server/baking/base settings with apply/save/custom-setting operations.

GlobalSettings applies input through:

• ApplyConfigAtPath() and ApplyConfigFile() for config files;
• ApplyCommandLine() for runtime/build-tool overrides;
• ApplyInternalConfig() for generated internal config;
• ApplySubConfigSection() for named overlays;
• ApplyDefaultSettings() and ApplyAutoSettings() for engine defaults/derived values.

Ordinary application startup creates non-baking GlobalSettings and applies Engine defaults before reading project input. The effective runtime order is: defaults, project config (or packaged internal config), selected sub-configs, the writable local-config cache, command-line overrides, then derived auto settings. A project .fomain therefore records deliberate authored choices; an omitted setting receives its declared Engine default rather than a zero-initialized value. Source/Tests/Test_Settings.cpp protects both the default baseline and the fact that a project override still wins.

ConfigBaker (Source/Tools/ConfigBaker.cpp) re-derives every sub-config from the root. Metadata stores each game setting’s configured root value and is the runtime baseline: BaseEngine applies it only when config, sub-config, local config, or command line did not set that name. The side-specific internal config therefore carries only game-setting sub-config deltas; false and empty deltas are written too, because they must override a non-empty metadata baseline. Built-in server/client settings retain the existing non-empty compact-config rules. MetadataBaker includes applied config timestamps in freshness and rejects a game-setting declaration that has no configured value.

That metadata baseline is not applied until BaseEngine exists. A game setting read by ApplicationInitHook or any earlier startup path must therefore travel in the binary config itself. Baking.BootstrapGameSettings lists exactly those exceptions: ConfigBaker writes each listed setting in full for every baked sub-config instead of reducing it to a delta. Every name must resolve to a declared game setting or baking fails. Keep this list narrow so the fixed 20000-byte internal-config patch remains bootstrap data rather than a second copy of the metadata baseline.

GlobalSettings::Save() still emits only settings present in _appliedSettings, which is populated from applied config keys plus the baking-mode auto-settings allow-list. Runtime-only settings (platform/build flags, monitor size, command-line/git/compatibility values, and the resolved Common.UserWritablePath) must remain in that allow-list. Settings consumed only by BuildTools/package.py are validated as ordinary settings. Built-in lookup accepts only dotted Group.Name spelling; an unqualified old name is a distinct custom setting and cannot mutate the built-in value. Script SetRuntimeSetting likewise rejects writes to built-ins as read-only while remaining available for project-owned custom settings.

Managed numeric, boolean, and enum setting getters use the typed indexed ABI. A built-in entry reads its immutable GlobalSettings field; a project ///@ Setting entry reads the custom map. A custom value is parsed once into the ABI entry’s typed cell and reused while GlobalSettings::GetCustomSettingsGeneration() is unchanged. Every custom-map writer (SetRuntimeSetting, SetCustomSetting, SetValue, and CopyFrom) increments that generation. String and list settings keep their name-based helpers, and both paths verify the complete setting name after hash dispatch so a colliding custom hash cannot shadow a built-in.

Custom settings have two read shapes. Use FindCustomSetting() when missing keys are normal and should stay in the nullable pointer vocabulary. Use GetCustomSetting() only for compatibility with the historical non-null sentinel behavior: it returns the stored value when present and _emptySetting when absent.

Do not document one embedding project’s .fomain contents as universal engine behavior. Use project docs for concrete values; use this page for the engine mechanics that consume them.

Baking.BakeLanguages is an ordered content contract, not an unordered locale allowlist: text baking uses the first value as the normalization base. Client.Language selects the initial client text pack. Exact .fotxt, $Text, fallback, and runtime lookup behavior belongs to Text and Localization.

Startup has one extra resolution step before the log, config, or local-config cache is opened. ResolveWritableRoot(args) in Source/Frontend/ApplicationInit.cpp derives the read-only Common.UserWritablePath without consulting settings: --UserWritablePath <path> (the dotted --Common.UserWritablePath spelling is accepted too) wins, an INSTALLED marker beside the executable selects the per-OS user-data directory plus FO_NICE_NAME, and otherwise the empty value keeps the portable working-directory layout. The special CLI value * requests the same per-user directory as the marker. Android obtains its base directory from SDL internal storage; the other platforms use Platform::GetUserDataBase(). A config file cannot set this path because the config itself may live beneath it. The resolved value is applied to the settings snapshot before the local cache is read, and the required cache/resource subdirectories are then created; failure logs a warning and falls back to the working directory.

The ordinary command line is still applied to live settings exactly once, after config, sub-config and local config, so it has final precedence and +-append overrides (-Setting +value) do not accumulate twice. That pass logs each Set <name> to <value> override. Settings whose names contain a token from Common.SecretSettingTokens (case-insensitive substrings, default secret token password apikey) are logged as Set <name> to ***. This is not general credential protection: raw process arguments and setting values remain available, and other logs, settings UI, crash output, baked configs, and project code have independent exposure paths. Never pass credentials on the command line; use target-time provisioning and follow Security and Secrets.

CommandLineArgs::IsOption() recognizes a leading dash as an option only when the next character is a letter. Thus --Render.Sleep -1 and other negative values remain values, not new option names. String settings preserve text as written, including Windows backslashes and quote characters; string-list settings split only at spaces and tabs. Numeric, boolean, and enum settings still parse their text. The AnyData escaped-string grammar belongs to properties, not settings (Source/Common/Settings.cpp, Source/Tests/Test_Settings.cpp).

Resource packs and data sources

ResourcePackInfo describes resource-pack inputs that bakers and runtimes consume. The bake side uses BakingContext / BakerDataSource in Source/Tools/Baker.*; the runtime side uses mounted DataSource and FileSystem abstractions.

Resource-pack input directories are mounted recursively. IncludePatterns and ExcludePatterns are optional space-separated glob lists applied to normalized resource-relative paths before any baker runs. An empty include list accepts every path; exclusion is evaluated after inclusion and wins. Patterns are case-sensitive and support:

• * — zero or more characters other than /;
• ? — exactly one character other than /;
• ** — zero or more characters including /; **/ also matches zero directory levels.

Both / and \ are accepted as pattern separators and normalized to /. For example, IncludePatterns = **/*.fomap selects maps at any depth, while ExcludePatterns = **/_*.fomap removes scratch maps such as Generated/_compose.fomap. Multiple packs may mount the same InputDirs and select disjoint resources with different pattern lists. Use IncludePatterns = * to reproduce the former top-level-only input behavior.

DataSource provides two built-in mount shapes:

• MountDir(dir, recursive, non_cached, maybe_not_available) for disk directory resources;
• MountPack(dir, name, maybe_not_available) for packed resource data.

MountPack probes .fores first, then ZIP-compatible .zip/.bos, then Fallout .dat. Packaging now writes one <Name>.fores per logical resource pack; only executable-embedded resources remain ZIP. A .fores pack is one full base plus at most one writable .patch.fores with a complete current catalog, not an override directory or patch chain. A corrupt claimed .fores is an error, never a reason to fall back to an older sibling ZIP. The byte layout, integrity hashes, recovery rules, and .foindex cache are specified in Resource Pack Format.

FileSystem then combines sources and offers:

• AddDirSource(), AddPackSource(), AddPacksSource(), and AddCustomSource();
• FilterFiles(), GetAllFiles(), and existence checks;
• ReadFile(), ReadFileText(), and ReadFileHeader();
• FileReader helpers for endian-aware binary reads.

Cached directory mounts snapshot their file index when mounted. Long-running tools can call FileSystem::ReindexDataSources() to ask every mounted source to refresh that snapshot; the method returns true when indexed paths, sizes, or write times changed. Sources that do not cache disk state keep the default no-op behavior. Custom sources can override DataSource::Reindex(); BakerDataSource uses it to rebuild input mounts and bake newly added or changed resources on demand.

Mount order matters for lookup behavior. When changing it, verify the runtime/tool path that owns the resource pack, not only the parser.

Shared index over mounted sources

Point lookups (IsFileExists(), ReadFile(), and ReadFileHeader()) use one shared index when every mounted source can provide a complete DataSource::GetIndexSnapshot(). ResourcePackSource, ZipFile, EmbeddedFile, and FalloutDat provide snapshots; the empty DummySpace used for an absent optional pack provides an empty snapshot. Directory sources, including CachedDir, keep the default nullopt, so adding any directory or other live source disables the index for that FileSystem and preserves ordered probing. Pure pack-backed filesystems used by packaged runtimes therefore index their resources, while unpacked development mounts, baker input directories, and mixed updater filesystems keep live lookup behavior.

The snapshot for a newly mounted source is taken before that source is published. A later mount has higher priority and replaces every indexed path it claims, matching the existing reverse-mount probe order. ReindexDataSources() rebuilds a replacement index separately and swaps it only after every snapshot succeeds; CleanDataSources() clears both sources and index. Point reads remain lock-free after setup. If an indexed source owns a path but cannot open it, ReadFile() returns no file instead of probing a lower-priority duplicate.

FilterFiles() and GetAllFiles() continue to enumerate sources in order even when point lookup is indexed. Script module load order, prototype registration, and other consumers depend on that source-by-source order, which an unordered index does not preserve.

Common.Packaged is a fixed auto-setting populated from the executable’s packaged marker by GlobalSettings::ApplyAutoSettings(). After settings are loaded, runtime policy must read that snapshot (settings.Packaged) so copied or injected settings stay internally consistent and testable. Direct IsPackaged() checks are reserved for pre-settings bootstrap decisions and FileSystem::AddPackSource(), where the physical executable marker deliberately selects archive-versus-directory mounting; tests may also inspect that marker when choosing compatible fixtures.

Packaged clients select one effective source per configured logical pack: a writable replacement Pack.fores when present, otherwise the installed base, paired with a writable Pack.patch.fores only when it binds to that exact physical base. Its complete catalog determines present and deleted paths; later configured packs retain precedence. GetClientResources() assembles that same view for gameplay and the updater’s metadata check. The suffix after the last Embedded entry can use a disposable Resources.foindex cache; stale or damaged cache data falls back to authoritative pair mounts, while damaged authoritative pack data remains an error. An absolute installed or APK root puts writable replacements under <UserWritablePath>/Resources. See Resource Pack Format and Client Runtime Split and Updater.

ResourceIndexSource and ResourcePackSource move the decoded byte vector into the file buffer holder. Its allocation remains owned until the caller releases the File: handing over a large stored resource no longer creates another full-size copy. Deflated entries still hold compressed and decoded buffers concurrently during decompression; this is reduced peak memory, not streaming or a new pack format. FileBufferHolderMovesVectorStorage in Test_ResourceIndex.cpp checks pointer preservation and empty buffers.

Low-level disk access

Source/Essentials/DiskFileSystem.* performs direct disk operations below mounted engine resources. fs_write_file() writes content at the requested path but, on a case-insensitive filesystem, an existing entry that differs only by letter case is reused with its old spelling. Callers that rewrite an exact-name tree they own must reconcile such entries explicitly. The baker does this once per pass; see Output names are reconciled with the names bakers addressed.

Cache storage

Source/Common/CacheStorage.* stores named binary/string cache entries behind HasEntry(), GetString(), GetData(), SetString(), SetData(), and RemoveEntry(). Bounded consumers use GetDataBounded(name, max_size), which checks the file size before allocating and distinguishes Success, Missing, TooLarge, and Failed, plus SetDataChecked(...), which reports whether the complete write succeeded. The underlying disk helper fs_read_file_bounded() applies the same pre-allocation cap and returns no data for an oversized file instead of reading it. It is separate from resource packs: cache entries are mutable runtime/tool artifacts, while baked resources are generated from configured inputs. Client-side cache consumers resolve relative cache paths through fs_make_writable_path(UserWritablePath, CacheResources), so portable clients keep cache next to the executable and installed clients write under the per-user root.

An entry is stored as one plain file named after the entry, with path separators folded to _, so the cache directory stays readable and inspectable. Two entry names that differ only in those separators therefore map to the same file — acceptable because an entry is only ever a cache, where a miss is always recoverable, but it means a caller that needs distinct entries must not rely on directory structure alone to separate them. The cache is not a confidentiality boundary: anything that must not be readable at rest has to be protected by its owner before it is handed over (the embedding project’s secure-storage bridge does exactly that).

Settings store

Source/Common/SettingsStorage.* persists small per-user tool/editor preferences (ImGui window layout, view options, last selection) behind GetString()/SetString(), typed GetInt/SetInt, GetBool/SetBool, GetFloat/SetFloat, HasKey(), and Remove(). It is scoped by an application name passed to the constructor so different tools never collide. The backend is platform-specific through a pimpl: on FO_WINDOWS the values are REG_SZ entries under HKCU\Software\FOnline\<app_name> (Win32 headers are confined to the .cpp behind WIN32_LEAN_AND_MEAN + WinApiUndef.inc, using the explicit *A registry entry points); on other platforms it is a per-application CacheStorage under Platform::GetUserDataBase()/FOnline/<app_name>. Every value is stored as a string (the typed accessors serialize through it), so both backends behave identically, and the multi-line ImGui imgui.ini blob round-trips verbatim. Persistence is best-effort: a backend failure is logged, never thrown, so a tool never dies because its settings could not be written. It differs from CacheStorage in intent (durable user preferences vs. regenerable cache artifacts) and, on Windows, in medium (registry vs. files).

Only the GUI tools reference it (Mapper MapperEngine::_uiSettings, migrated from the resource Cache; standalone AnimationViewer / ParticleViewer, each loading in its constructor and saving on shutdown). It lives in CommonLib for simplicity, but because the client and server reference no SettingsStorage symbol, the linker (/OPT:REF plus on-demand static-library inclusion) drops the object from the shipped client/server binaries — so the Windows registry calls never land where antivirus heuristics might flag them. ImGui’s own imgui.ini autosave stays disabled (Application.cpp), so all layout persistence flows through this store.

Build and package routing

• BuildTools/cmake/stages/Codegen.cmake generates internal config inputs used by runtime settings.
• BuildTools/cmake/stages/ScriptsAndBaking.cmake wires resource baking/script compilation that consume ResourcePackInfo and baking settings.
• BuildTools/cmake/stages/Packages.cmake packages resources for runtime targets.
• Source/Tools/ConfigBaker.* bakes config resources; full bake orchestration is in Baking Pipeline.

Tests to inspect

Focused tests for this area:

• Source/Tests/Test_CacheStorage.cpp
• Source/Tests/Test_SettingsStorage.cpp
• Source/Tests/Test_ConfigFile.cpp
• Source/Tests/Test_DataSource.cpp
• Source/Tests/Test_DiskFileSystem.cpp
• Source/Tests/Test_FileSystem.cpp
• Source/Tests/Test_Settings.cpp
• Source/Tests/Test_ConfigBaker.cpp

Related consumers are covered by resource, client, server, script, and baker tests listed in Testing.

Change routing

• Config grammar and parsed section/key behavior: Source/Common/ConfigFile.*.
• Setting groups, defaults, command-line/config/sub-config application: Source/Common/Settings.* and Settings.inc.
• Installed-client writable-root resolution: Source/Frontend/ApplicationInit.cpp, Source/Essentials/Platform.*, and Source/Essentials/DiskFileSystem.*.
• Mounted resource lookup: Source/Common/DataSource.* and FileSystem.*.
• Raw disk operations: Source/Essentials/DiskFileSystem.*.
• Runtime resource consumption: Source/Client/ResourceManager.* plus owning runtime docs.
• Particle source selection, .spark/.efkproj compilation, and .spk/.efk runtime consumption: Source/Tools/ParticleBaker.*, Source/Client/ParticleRuntime.*, Source/Client/VisualParticles.*, and Particle Format And Runtime.
• Font descriptor raw-copy selection and runtime consumption: Baking.RawCopyFileExtensions, Source/Tools/RawCopyBaker.*, Source/Client/FontManager.*, and Font Formats And Text Layout.
• Audio baking, sound indexing, Vorbis decoding, and client playback: Baking.AudioVorbisQuality, Audio.SoundFileExtensions, Audio.*, Source/Tools/AudioBaker.*, Source/Client/AudioManager.*, and Audio.
• Video raw-copy selection, exact-path loading, Ogg/Theora decode, fullscreen/embedded client playback, and memory ownership: Baking.RawCopyFileExtensions, Source/Tools/RawCopyBaker.*, Source/Client/VideoClip.*, Source/Client/Client.*, and Video.
• Resource-pack generation: Baking Pipeline and Source/Tools/*Baker.*.

Validation checklist

1. Run the focused parser/settings/filesystem/cache tests for the changed area.
2. If resource-pack shape or mount order changes, run at least one bake path and one runtime/tool consumer path.
3. If command-line or sub-config behavior changes, verify the embedding project config that exercises it, but keep project-specific values in project docs.
4. If packaging/resource staging changes, re-check Web Build, Packaging, and Browser Debugging, Android Build, Packaging, and Device Debugging, and Client Runtime Split and Updater as applicable.
5. Update Baking Pipeline or BuildTools Pipeline when build-stage ownership changes.