View on GitHub

FOnline Engine

Flexible cross-platform isometric game engine

Generated API and Metadata

This document explains the engine code-generation and metadata-registration flow. Use it when changing generated source, metadata annotations, property definitions, or script-visible API contracts.

Ownership model

The engine owns the reusable metadata/codegen machinery. An embedding project supplies project configuration, extra metadata sources, common headers, and script/content inputs through CMake options and project files.

Generated files are build artifacts. Document the source annotations, templates, generator inputs, and validation flow; do not treat generated output as hand-authored engine source.

Source paths inspected

CMake codegen stage

BuildTools/cmake/stages/Codegen.cmake constructs the generator command and output list.

Important command arguments include:

The stage creates normal and forced code-generation command targets and appends CodeGeneration to FO_GEN_DEPENDENCIES. The normal target depends on codegen-args.txt as well as metadata sources and the generator. Reconfiguring after a revision or engine configuration change therefore refreshes generated build/version macros. An unchanged argument list preserves the file timestamp and does not trigger another generation.

Generated outputs

Codegen.cmake declares generated outputs under GeneratedSource/, including:

These file names are useful for understanding build flow, but changes should usually be made in templates, annotations, metadata sources, or generator scripts rather than in generated output.

InternalConfig.gen.inc reserves an engine-owned fixed 10000-byte patch area. Embedding projects cannot resize it; project-only script settings are shipped through metadata instead of this bootstrap config.

Metadata registration entry points

Hand-authored declarations live in Source/Common/MetadataRegistration.h:

Source/Common/MetadataRegistration.template.cpp is the template used to generate side-specific registration files. It contains code-generation markers such as ///@ CodeGen RegisterHelpers and ///@ CodeGen Register.

Source/Common/GenericCode.template.cpp is the template for generated common code.

GenericCode-Common.cpp also emits per-target GetServerSettingsTyped(), GetClientSettingsTyped(), and GetMapperSettingsTyped() lists for engine ExportSettings. ManagedScriptBaker consumes those lists, together with the game settings from metadata, to generate C# accessors for scalar settings and supported vector<T> settings as List<T>, while dictionary-shaped settings remain outside this generated managed surface. A setting is read by its own dotted name: Domain.Name becomes the property Name of the nested static class Settings.Domain, so scripts write Settings.Audio.MusicVolume. The baker rejects a setting name that is not exactly Domain.Name. Mapper generated settings include the Client/Common engine ExportSettings surface because mapper AngelScript runs with the same visible engine settings groups for editor rendering and input helpers. Managed methods returning non-generic Task register as native void script functions: callback invocation observes an incomplete task and returns without blocking the script pump, so ScriptTask.Delay can resume it and deferred faults remain accounted. Task<T> registers as T and is awaited synchronously because the native caller requires the result.

Engine hook tags

Project/native extension code can mark selected C++ functions with ///@ EngineHook. BuildTools/codegen.py validates hook names and emits no-op stubs for hooks that the embedding project does not implement. Current hook names recognized by the generator are:

ClientStartupSettingsHook is called by app entry points immediately before constructing a client engine. Use it for project-owned startup setting adjustments; do not use it as a gameplay authority bypass.

ApplicationShutdownHook is a native lifecycle hook for project-owned process integrations that must be stopped before a client runtime DLL is unloaded. It is intentionally not part of the compatibility hash because it does not change script metadata, saved data, or the network contract.

Script Entity promotion

The AngelScript Entity type has no single native counterpart. get_entity_from_target in BuildTools/codegen.py promotes every argument, array element and return that spells a script Entity to the entity class of the target: ServerEntity* on the server, ClientEntity* on client and mapper, and the base Entity* elsewhere. Generated entity methods are registered on each concrete script entity type for which they are available; the base script Entity exposes only its common built-in operations, so a prototype cannot reach a server-only method through an abstract Entity receiver.

The promotion is a claim about the caller, not something AngelScript checked. register_entity_protos and register_entity_abstract in Source/Scripting/AngelScript/AngelScriptEntity.cpp register an implicit cast from every Proto* and Abstract* type to Entity, so a prototype is an Entity to a script — but ProtoEntity and ServerEntity are siblings under a single Entity base, so the promotion is false for one. Under single inheritance the pointer value survives it, which means an unchecked promotion does not fail at the call; it fails later, inside the callee, on a member the object does not have.

NativeDataCaller::ConvertArg in Source/Common/ScriptSystem.h therefore reads every entity slot as the base and narrows it to the declared type, for scalar arguments and array elements alike, throwing ThrowScriptEntityTypeMismatch with the type and name that were actually passed. Array elements deliberately keep no destroyed-entity check: an array argument may carry entities that died since the caller built it, and its callees drop them.

A dict value carrying an entity handle is narrowed the same way. Two static_asserts bound what is not: an entity handle as a dict key, because a key is a script value type rather than a handle slot, and an array of handles nested inside a dict value, because that slot holds a whole array object.

Dynamic metadata

Source/Common/MetadataRegistration.cpp implements RegisterDynamicMetadata(). It reads binary metadata sections and dispatches them into typed registration steps such as:

This is the runtime side of metadata that can be loaded from generated/baked data rather than compiled static registration alone.

Value types are plain data

IsStruct is an invariant, not a hint: a value type (///@ ExportValueType in the engine, ///@ ValueType in a project) is plain data that every consumer moves with memcpy - the managed bridge, the property store, remote-call buffers, native assignment. EngineMetadata::RegisterValueTypeLayout is where it is enforced:

Generated C# value types are [StructLayout(Sequential)], and the managed backend checks mono_class_value_size against the layout wherever it boxes or unboxes one, so a mismatch is an error rather than a partial copy. Data that is not plain - strings, lists, nested records, anything with identity or ownership - belongs in a ///@ RefType. hstring is plain in this sense: its object representation is the intern entry pointer on both sides. The one place its bytes differ is property storage, which keeps the 64-bit hash; the managed bridge swaps hash and handle in place there (PropertyDataToValue / ValueToPropertyData). Test_EngineMetadata pins the rejected shapes (ValueTypeIsAlwaysPlainPackedData).

Metadata version

Invariant: a server and every client connected to it run on metadata produced by one bake. This is not a preference — the property index space that entity data travels by is the registration order of that metadata, so two sides holding different bakes silently address different properties. A divergence is a defect in how the build or the deploy was done; it is detected and refused, never tolerated or worked around.

Why the compatibility version does not cover it: BuildTools/codegen.py only receives the engine and embedding- project C++ meta sources, so FO_COMPATIBILITY_VERSION changes with the binaries. Project ///@ Property declarations live in scripts and are registered at runtime from the baked metadata (RegisterDynamicMetadataProperties), which means the property layout is a property of the resources, not of the executable.

MetadataBaker therefore derives a metadata version from every codegen tag it parsed, in a deterministic order. The input is the raw tag stream as read from the sources — before any target filtering — so client, server and mapper of one bake always derive the same value even though their section bodies differ (Entity, Event, Setting, RemoteCall are filtered per target on the way out). Hashing the finished file instead would not work for exactly that reason; hashing the raw tags has no such limit, so no kind of divergence stays invisible: a property insertion that shifts every reg index below it, a changed struct layout, a renamed enum entry, a new remote call — all of them change the version, and all of them mean the two sides came from different bakes.

Every Metadata.fometa-* therefore opens with a fixed header, ahead of the section table:

Field Type Purpose
magic uint32 METADATA_FILE_MAGIC — a foreign or truncated file is rejected at the first bytes
file version uint16 METADATA_FILE_VERSION — bumped when this file layout changes; a mismatch means “rebake”
metadata version uint16 length + bytes the value above

A change to the token layout of any section is a change to this file layout, so it bumps the file version. The metadata version cannot stand in for it: that hash is derived from the codegen tags, which do not move when the baker starts writing another token, so an unbumped pack from the previous engine passes the header and is then read record by record under the new layout — the failure surfaces as a section-level VerificationException deep inside registration instead of the “rebake” verdict. MakeMetadataHeader() writes the header and ReadMetadataHeader() reads it, both in MetadataRegistration.cpp, so the format lives in one place. RegisterDynamicMetadata() reads the header before any section and hands the version to EngineMetadata::RegisterMetadataVersion(); ReadMetadataVersion() reads only the header, which is what the updater and the server startup check use — neither walks the sections to answer “which bake is this”. The value is read back through EngineMetadata::GetMetadataVersion() — it is computed, not configured, so it is deliberately not a setting (Network.ForceMetadataVersion exists only to simulate a divergence in tests).

Four layers keep the invariant, in the order they apply:

  1. One bake produces both sides. Baking.ServerResources and Baking.ClientResources must be deployed together; refreshing one of them is the classic way to break this.
  2. The server refuses to distribute foreign resources. UpdaterBackend::LoadFromClientResources reads the layout version out of the client packs it is about to hand out and fails startup (UpdaterException) when it differs from the one the server itself loaded.
  3. The updater syncs before a client exists. Updater::FinishResourcesUpdate re-reads the version from the local packs after the sync and reports UpdaterResult::MetadataMismatch unless it equals the server’s, so a ClientEngine is never constructed against data the server cannot talk to.
  4. The handshake is the last line. The client sends its version, the server compares and answers with a verdict plus its own version; see ClientUpdater.md.

Deserialization is guarded independently of all four: Properties::VerifyRestoredPropertyData() checks every property write coming from a serialized payload (target enabled, non-virtual, plain size matches) and throws VerificationException instead of reaching the strong assert inside SetRawData — a mismatch has to be diagnosable, not a process termination inside a memcpy.

When a divergence is reported, find the cause — do not silence the check. The useful facts are in the logs: the server prints Metadata version: at startup and names both versions when it rejects a client; the updater prints the local version, the server version, and the resource directory it read. From there the question is always the same: which of the two resource directories came from a different bake, and why.

Tests: Test_MetadataBaker.cpp (one version shared by every target, changed by a property insertion; a pack written in an older file layout is refused), Test_Properties.cpp (PropertiesRestoreRejectsForeignMetadata), Test_ClientServerIntegration.cpp (ServerReportsMetadataMismatchInHandshake).

Properties and generated contracts

Source/Common/Properties.h and Source/Common/Properties.cpp define the property runtime model used by entities and metadata. Key concepts include:

Fixed value-type layouts are shared by native C++, AngelScript registration, and metadata field traversal. hstring therefore has an explicit ABI invariant: sizeof(hstring) == sizeof(hstring::hash_t) == 8 on every supported target. The live handle is the intern-entry pointer; empty is nullptr (a zero-filled slot). On 32-bit targets the pointer-backed handle carries trailing padding to preserve that width and keep composite offsets (for example TextPackKey) platform-independent. The padding is not wire data: RPC/property serializers still convert the handle through as_hash() and resolve the received hash through the target engine’s hash resolver.

When property metadata changes, inspect both the property runtime and the generator inputs/templates. Script-visible nullability or API changes should also update Scripting.md, ScriptMethodsMap.md, and Nullability.md as applicable.

Managed property access completes native exception unwinding before returning through Mono’s internal-call ABI. Typed property wrappers and the generated Entity.GetAsInt / SetAsInt / GetAsAny / SetAsAny helpers therefore surface access, destroyed-entity, property-index, type, disabled-property, and mutability violations as catchable managed InvalidOperationException instances; no C++ exception crosses the ABI. Typed int8, uint8, int16, and uint16 property accessors use the integer property-index bridge directly, preserving their narrow C# types without relying on Mono’s boxed-object scalar ABI.

When Managed scripting is enabled, ManagedScriptBaker consumes the same property metadata to emit typed C# property enums and entity wrappers. The generated base Entity includes generic GetAsInt<TProp>(), SetAsInt<TProp>(), GetAsAny<TProp>(), and SetAsAny<TProp>() helpers that pass property enum values through Native.EnumToInt32 into native property-index bridges; the any pair reads and writes the managed any type (Scripting.md). Metadata any is generated as any everywhere, and each value struct gets the operators that convert it to and from any. A wrapper type belongs to its backend’s load context, so a wrapper only ever reaches its own engine; once that engine is torn down, Native.IsBackendAlive turns false and access through the wrapper fails with ObjectDisposedException before the entity pointer can reach released engine metadata. The strong native reference a wrapper holds is taken on the entity in the wrapper’s constructor and given back in its finalizer, and backend teardown runs the collector and waits for the finalizer queue to drain so those references come back while the engine is still alive; see Scripting.md and ServerRuntime.md. Entity equality and hashing use the native entity pointer, so separately boxed wrappers for the same entity compare equal; wrappers of two engine instances are instances of different types and never compare equal. The generated C# Game.GetPropertyInfo(<Type>Property, out ...) overloads are metadata-only mirrors of the AngelScript property-info API for entity and fixed-type properties, so changes to property flags, base type, sync state, or enum names are reflected by a managed rebake. Cross-backend managed callbacks marshal fixed types and entity-proto values as proto-entity handles, matching AngelScript’s handle representation while preserving their metadata type identity for function lookup. Managed callbacks may also return metadata-defined dynamic ref types: the bridge transfers one native reference to an AngelScript result or retains it through managed boxing, so the reconstructed object remains valid for the complete handoff. Scalar AngelScript object handles returned through managed ScriptFunc.Invoke<TResult> stay owned by the call frame until boxing completes, then release the temporary script reference; dynamic ref types are copied into managed values during that window. The baker and native backend share one ABI description in ManagedInteropAbi: dense method, event, setting, and inner-entry ids plus a content hash. Generated *Abi.gen.cs bind stubs call Native.BindAbi from Initializator.InitializeEarly; a hash or count mismatch is a load error before script execution. Scalar, enum, and value-type methods fill a caller-owned frame and dispatch with CallMethodIndexed, and so do methods whose arguments or result are entities, protos, fixed types or native ref-type arguments: those travel as pointer slots (ManagedAbiValueKind::Handle), with slot nullability part of the manifest hash; remaining signatures use CallMethodBoxed with the same id. Array properties of fixed values read and write through GetPropertyList<T> / SetPropertyList<T> as raw bytes, and every other non-scalar property reaches the converting bridge by registrar index instead of by name. Numeric/bool settings read through indexed GetSettingValue<T>; the generated settings surface stays get-only. Inner-entity getters snapshot through FillInnerEntities instead of Count + n×At. Non-nullable value-type properties use GetPropertyValue<T> / SetPropertyValue<T> the same way as primitives. hstring is the interned entry pointer, the same 8 bytes in managed and native code, so frames and value types copy it as is; property storage alone holds the hash, and the bridge swaps hash and handle in place at that boundary, inside value types too. Generated event surfaces provide synchronous, Task-returning, and explicit-EventResult handler overloads. Events whose arguments are fixed values or by-value handles subscribe and fire by ABI event id (FireEventIndexed / a generated AdaptInvoke); signatures with strings, collections or by-ref handles still box. Native-to-managed callbacks with entity/ref-type handles and fixed values dispatch through generated CallbackAdapters.Adapt_<key> frame adapters resolved once per registration; the same bind stub registers Native.RegisterWrapperFactory<T> for every generated wrapper class, so wrapping a native pointer needs no reflection (see Scripting.md). Native-to-managed event dispatch runs through the engine’s script-context boundary, giving each server callback its own nested sync cover just like AngelScript execution. A Task handler runs synchronously until its first incomplete await; the event chain then continues without blocking the script dispatcher, while the managed bridge observes deferred faults. Explicit EventResult handlers remain synchronous. Generated Game.AddPropertySetter(...) overloads are also metadata-shaped: entity-only post-set callbacks receive Action<TEntity>, simple setters receive (entity, ref value), and property-group setters can receive (entity, <Type>Property, ref value) through PropertySetterWithProperty<TEntity,TProperty,TValue>.

Public API relationship

../PUBLIC_API.md documents public build/API knobs such as build toggles and project helper functions like resource/package additions. Keep public API docs high-level and stable; put generator internals here.

Metadata and baker relationship

Metadata generation and metadata baking are related but not identical:

For resource baking details, see BakingPipeline.md.

Tests to inspect

Relevant tests include:

If a generated script API change is involved, inspect AngelScript-related tests as well.

Change routing

Validation checklist

  1. Configure from an embedding project root so project metadata sources are available.
  2. Run normal code generation and verify generated files are updated as expected.
  3. Run forced code generation when generator caching/dependency behavior changes.
  4. Build the smallest target that compiles the generated files.
  5. Run metadata/property tests relevant to the change.
  6. If metadata is baked, run the relevant baker test and bake target.
  7. Update docs that expose changed public contracts, especially Scripting.md, ScriptMethodsMap.md, Nullability.md, and ../PUBLIC_API.md when applicable.