View on GitHub

FOnline Engine

Flexible cross-platform isometric game engine

Testing

Engine-owned documentation. This page maps the current engine test executable, generated test targets, coverage targets, and every Source/Tests/Test_*.cpp suite currently present in the checkout.

Purpose

Use this page when choosing validation for an engine change or when adding/removing tests. The source-tree README at ../Source/Tests/README.md is a short entry point; this page is the maintained full test map.

Source paths inspected

Test runner model

Source/Applications/TestingApp.cpp is the test application entry point. It requires FO_TESTING_APP, initializes the application layer with InitApp(-1, nullptr), marks IsTestingInProgress, and delegates execution to Catch::Session().run(argc, argv).

BuildTools/cmake/stages/EngineSources.cmake owns FO_TESTS_SOURCE, the explicit list of test source files compiled into test builds. BuildTools/cmake/stages/Applications.cmake builds test executables through SetupTestBuild(name):

BuildTools/check_windows7_imports.py <binary> [...] is a standalone PE-level regression check for Windows 7 artifacts. It rejects the reported CreateFile2 import; embedding-project CI should run it after linking and before packaging.

For an embedding project with dev name LF, the standard generated names are LF_UnitTests, RunUnitTests, LF_CodeCoverage, RunCodeCoverage, GenerateCodeCoverageReport, and AnalyzeCodeCoverage. Treat the prefix as project-generated, not universal.

Running tests

Preferred local baseline from a configured build:

cmake --build . --config RelWithDebInfo --target RunUnitTests

With FO_EFFEKSEER_PARTICLES enabled, the focused [particle] Catch2 cases invoke the published helper through the production ParticleBaker path. They cover text compilation, dependency invalidation, malformed XML, and rejection of cooked files presented as authored inputs.

The executable target can also be invoked directly when you need Catch2 arguments. In Last Frontier-style layouts, test binaries are emitted under Binaries/Tests-*, for example Binaries/Tests-Windows-win64/LF_UnitTests.exe or Binaries/Tests-Linux-x64/LF_UnitTests.

With Visual Studio/MSBuild generators, RunUnitTests writes the test process output to <build-dir>/<ProjectDevName>_UnitTests.log and uses the test process exit code as the pass/fail signal. This keeps expected negative-case diagnostics such as compiler error lines from being reclassified as MSBuild errors. When the run fails, the helper also echoes the captured output before failing, so a failure is diagnosable from the build output alone — on CI the log file never leaves the runner, and the exit code by itself does not say which test or assertion broke.

For broad validation scenarios, the BuildTools validators can run selected scenarios:

Engine/BuildTools/validate.sh unit-tests
Engine/BuildTools/validate.sh android-arm64-client linux-client linux-server

Use the smallest focused tests first, then the broader run target when the change crosses subsystem boundaries.

Unit tests under sanitizers

The unit tests also run under Clang sanitizers via dedicated validators, which select the matching San_* build type and run RunUnitTests instrumented:

Engine/BuildTools/validate.sh unit-tests-san-address    # AddressSanitizer (+LeakSanitizer)
Engine/BuildTools/validate.sh unit-tests-san-memory     # MemorySanitizer (requires Workspace/msan-libcxx)
Engine/BuildTools/validate.sh unit-tests-san-undefined  # UndefinedBehaviorSanitizer
Engine/BuildTools/validate.sh unit-tests-san-thread     # ThreadSanitizer

The validate.yml workflow runs these as a unit-tests-sanitizers matrix job. ASan/MSan/UBSan/TSan are blocking legs. The unit-tests-san-memory validator prepares Workspace/msan-libcxx by building LLVM’s libc++, libc++abi, and libunwind with MSan instrumentation, then configures San_Memory with FO_MSAN_LIBCXX_ROOT. The runtime build applies a narrow libunwind ignorelist so C++ exception and sanitizer-report unwinding do not self-report on ABI register snapshots. Engine native stack capture and the backward-cpp signal handler are disabled under FO_MEMORY_SANITIZER so MSan owns fatal reports. unit-tests-san-memory-with-origins is available locally as the slower diagnostic variant when a future MSan finding needs origin tracking. San_DataFlow remains intentionally unwired: DataFlowSanitizer is a taint-tracking framework, not a defect detector.

Applications that load BakerLib while running under a sanitizer must use a baker built with the same San_* configuration. Hiding the plugin’s ELF exports prevents direct symbol interposition, but calls implemented inside the shared C++ runtime may still allocate through the host and return to an inline deallocator in the plugin. Matching configurations keep the sanitizer runtime and allocator contract identical on both sides of that module boundary.

On MSVC, the San_Address/Debug_San_Address configs additionally link executables with /STACK:8388608 (AddExecutableApplication in BuildTools/cmake/helpers/Build.cmake): ASan’s stack-frame inflation overflows the 1 MiB Windows executable default on recursion depths that fit every production configuration, so sanitizer runs get the same 8 MiB reserve that Linux runs already have from the default rlimit. Production configs keep the 1 MiB default.

Vendored third-party libraries are excluded from UBSan’s -fsanitize=function and -fsanitize=alignment checks (the rest of -fsanitize=undefined still applies to them). DisableLibWarnings adds -fno-sanitize=function,alignment on the San_Undefined/San_Address_Undefined configs because several vendored libraries trip those two checks by design:

Both are third-party idioms, not undefined behaviour in engine code, so they must not fail the UBSan leg (which CI runs with halt_on_error=1). First-party engine code keeps both checks fully active.

LeakSanitizer runs as part of the address-sanitizer leg (CI sets ASAN_OPTIONS=detect_leaks=1). It runs with no suppression list — every leak it can report is fixed at the source rather than masked. Notable cases:

Code coverage

When FO_CODE_COVERAGE is enabled, BuildTools/cmake/stages/Init.cmake selects the backend from the compiler:

BuildTools/cmake/stages/Applications.cmake wires coverage command targets through BuildTools/codecoverage.py:

Coverage output is rooted under CodeCoverage/<Toolchain>/<Platform-Config>/. BuildTools/codecoverage.py reports first-party production engine sources under Engine/Source/; it excludes Source/Tests/, ThirdParty/, GeneratedSource/, and Applications/ from the denominator. See ../Source/Tests/README.md for current local task notes.

Coverage is a per-platform, per-environment measurement, and the denominator reflects that in two different ways:

The summary prints the scoped headline, a combined all-sources figure, and the excluded bucket file-by-file with reasons, so the split stays auditable. Adding an entry there is a routing decision, not a write-off: it must be covered by the layer that can run it — a windowed/rendering run on the owning platform, or an integration suite with real endpoints.

Covering ImGui diagnostic panels

DrawGui() implementations normally only run inside the windowed application, but they are reachable from unit tests through a backend-less ImGui context: no renderer is attached and the draw data is discarded, while every panel builder runs for real. Test_ServerEngine.cpp shows the pattern. These details matter:

Pressing a widget so the branch behind it runs

Drawing a panel covers its layout, not its behaviour: the body of every button, checkbox, selectable and tree node stays unreachable because nothing is ever clicked. Test_ImGuiHarness.h closes that gap, and Test_ImGui.cpp pins the harness itself against a window the test owns. The rules that matter:

What an inbound remote call can reach

The handler is entered with the calling player covered, plus - transitively - the critter it controls. Everything else needs an explicit Game.Sync(...), and some native paths reach further than any cover a script can prepare. Reachable on a second critter after Game.Sync(npc): Map.AddCritter, Critter.SetDir, Critter.Action and synchronized property writes. Not reachable on a critter the caller does not control: TransferToHex, SetCondition, DestroyItem, AttachToCritter and Game.DestroyCritter, plus moving a map item into an inventory and reusing one location across two logins.

A script-level catch around a failing call does not contain the damage: the sync violation still tears the session down, so the next remote call never arrives. A test that probes these operations behind try/catch therefore reports “the last step was never sent” rather than the operation that actually failed - drive only what is reachable.

Covering the crash reporter

ExceptionHandling.cpp publishes SetCrashStackTrace, SetCrashSignalInfo, SetCrashSehInfo, SetCrashTerminationInfo and GetCrashStream to backward.hpp only — they carry no engine namespace and appear in no engine header, so a test declares them exactly as that header does. The report is emitted through the base log on the first write to the crash stream, so point LogToFile at a private file, write one line into GetCrashStream() and read the report back instead of letting “FATAL ERROR!” leak into the test console. Restore the log with LogToFile("/dev/null") ("NUL" on Windows); there is no “stop logging to a file” call. ReportExceptionAndExit and ReportStrongAssertAndExit kill the process and stay uncovered by design.

Covering the text formatter without a real font asset

FontManager refuses to answer any metric for an unbound slot, so text measurement, wrapping and drawing are unreachable until a font exists. Both loader formats can be synthesized in-memory, which is cheaper and more stable than shipping a binary asset:

Bind with a scale in (0..1] — larger scales are rejected on purpose, because the intended fix for bigger text is a bigger font asset.

SplitLines paginates into rect-sized pages rather than into individual lines: it emits an entry only once the text overflows the rect height, so a test that wants several entries needs a short rect, not merely embedded newlines.

Driving a logged-in client↔server session

A connected client is not a logged-in one: the pre-login session accepts only a remote call, so login is script-driven from the client. The pieces that have to line up:

Reaching the world-reload path

Server tests default to the in-memory database, which means the branch a real server takes on every restart — “Restore world” and EntityManager::LoadEntities — never runs. Point the settings at the file-backed JSON storage instead (DbStorage = "JSON <dir>"), let one server write the world and shut down, then start a second server on the same directory. Two constraints:

Instantiating a 3D model headlessly

The Null renderer serves the whole model path, so a ModelInstance can be created, posed and drawn without a GPU. The fixture chain is what makes it work:

Get the manager from the live client with client->SprMngr.GetSpriteFactory(typeid(ModelSpriteFactory)).dyn_cast<ModelSpriteFactory>()->GetModelMngr().

Authoring static map content for a server fixture

A .fomap-bin-server blob is the hash table, then the critter records, then the item records. Each record is ident (int64), the prototype hash (uint64) and a properties blob preceded by its uint32 size. Writing a zero size fails with “Unexpected end of buffer” — a default-constructed Properties still serializes to a non-empty payload, so produce it with props.StoreAllData(...) rather than assuming empty means zero bytes. With content present, map creation runs the content generator instead of skipping it.

The client-side .fomap-bin-client blob is a different, shorter layout (hash table plus static items only).

Writing into a real Maps root from the mapper

SaveMap / SaveMapToDir resolve the on-disk Maps root from an existing map container, so a memory-only fixture cannot reach them. Point a resource pack at a temp directory with InputDirs = <dir> (the plural key — the singular one is silently ignored), drop a reference .fomap there, and set ProtoFileExtensions to include fomap so the container is recognised. The same fixture gives DrawMapListWindowImGui real entries to enumerate.

Prefer SaveMapToDir in tests: plain SaveMap falls back to the first source file’s directory when the map has no container of its own, which in a test process is the working directory — it will write into the repository.

Current test inventory

Current count: 100 Test_*.cpp suites.

Essentials and low-level utilities

Configuration, data sources, files, and caches

Common runtime model

Networking and server/client integration

Scripting and script-visible APIs

Bakers and tools

The model-animation tests divide the production contract explicitly. Test_ModelMeshData.cpp exercises the mandatory LFMODMSH schema-1 mesh-only header and complete recursive payload codec. It covers geometry, skin palettes, children, structural validation, trailing data, every truncated header size, rejection of old headerless data, and exact byte compatibility with the original schema-1 writer layout. Test_ClientEngine.cpp also bakes a position-only OBJ through ModelMeshBaker and preloads the resulting bytes through the real ModelManager parser. This crosses the BakerLib/ClientLib boundary and catches payload-layout drift that a second test-only parser could reproduce instead of detecting. Test_ModelSourceLoader.cpp covers complete source validation, real minimal OBJ/ASCII-FBX extraction, per-call cache single-flight behavior, shared results, exception fan-out, and missing inputs. Test_ModelAnimationData.cpp exercises the little-endian archive, joint-remap, and rig-manifest contracts, including truncation, count/length bombs, ordering, metadata mismatches, and bindings. Test_ModelAnimationConverter.cpp covers canonical conversion and the per-instance runtime pose: unaligned/owned loading, body blending, movement replacement, reverse and nearest sampling, stable storage, canonical resolution, and numeric limits. Test_ModelAnimationPoseProcedural.cpp covers bounded procedural pre-rotations and exact world-matrix overrides; Test_ModelAnimationRuntime.cpp covers the validated direct-model rest path, canonical contributed-joint lookup, and cross-model joint-link resolution without physical bones. Test_ModelBaker.cpp covers source-backed model-info generation, dependency-mtime invalidation, exact animation-geometry exceptions, Base, reverse, case-insensitive lookup, and clip deduplication. Test_ModelAnimation.cpp is the timeline/binding behavior gate: controller copies own mutable event state while sharing only immutable Ozz clip metadata.

After source-loader, mesh-wire, or converter changes, ForceBakeResources is the positive real-content gate: it must parse the project’s actual selected FBX sources and extract their animations successfully. Run ordinary BakeResources afterward to check that the dependency-mtime contract leaves an unchanged tree incremental-clean.

Rendering/frontend smoke tests

Validation routing by change type

Adding or removing tests

  1. Add the new Source/Tests/Test_*.cpp file with deterministic Catch2 tests.
  2. Add it to FO_TESTS_SOURCE in BuildTools/cmake/stages/EngineSources.cmake.
  3. Update this page and ../Source/Tests/README.md so the inventory stays complete.
  4. Run the focused test binary and, when practical, RunUnitTests.
  5. If coverage behavior changed, verify the relevant coverage target.

Validation checklist

  1. Every current Source/Tests/Test_*.cpp file should appear in this page.
  2. No deleted/nonexistent test file should be listed.
  3. Target names should be described as generated from FO_DEV_NAME, not hard-coded as universal engine names.
  4. If TestingApp.cpp, FO_TESTS_SOURCE, or coverage target wiring changes, update this page in the same change.