Change-based validation
Tools/SelectValidation.py reads which files a change touches and runs only the builds, tests and acceptance scripts that cover them, with a small smoke set every time and everything when shared code changes.
Plan the checks for a change
Run the tool from the repository root. Without arguments it diffs the working tree against the merge base with your branch's upstream (or HEAD~1 when the branch has none), including uncommitted and untracked files, and prints the plan.
A plan lists, in order: a configure step when needed, one build of the affected targets, the CTest runs (by label and by name), the acceptance scripts and the documentation check. --config Release plans a Release build instead of Debug.
PowerShell
python Tools/SelectValidation.py # this branch's changes python Tools/SelectValidation.py --base main # everything since main python Tools/SelectValidation.py --files KroysEngine/Editor/Panels/VfxStudioPanel.cpp python Tools/SelectValidation.py --json # the plan as JSON
What a change selects
| Change | Checks |
|---|---|
| Documentation or roadmap only | python Tools/BuildDocsSite.py --check and nothing else |
| Any other change | The smoke set: catalog and map drift, layer includes, core task and log cases |
A module (KroysEngine/<Module>) | Its layer's doctest programs and its CTest labels, for example KroysVFX or KroysNetworking |
| A test source | The program(s) that compile it, read from KroysEngine/Tests/CMakeLists.txt |
Path handling (Core/PathUtils, Core/Paths, asset registry and cooker) | The same cases again with KROYS_TEST_SHORT_PATHS=1 |
| Renderer, shaders, materials, lighting, scenes | Tools/RunRenderRegression.ps1 |
| An editor tool (VFX, materials, UI documents, placement, play mode, templates, tasks, content browser) | That tool's Tools/Run*Acceptance.ps1 |
| Packaging, assets, cooking, server, player, networking | Tools/RunPackageAcceptance.ps1; server and network code also the headless server check |
KroysEngine/Streaming | Tools/RunStreamingWorldAcceptance.ps1 |
Docs/Site, managed APIs, Plugin SDK headers | python Tools/BuildDocsSite.py --check |
Shared core headers, top-level CMake or cmake/, third-party code, scene, asset, project or save-game formats, test infrastructure | Full validation |
A file the map does not know selects full validation, so a gap in the map never skips checks.
Full validation
--full plans everything CI runs and more: configure, the whole build matrix, every CTest case with KROYS_TEST_SHORT_PATHS=1, render regression, every acceptance script, package acceptance, the headless server and the documentation check. Use it before a push and for release candidates (Release gates).
PowerShell
python Tools/SelectValidation.py --full
Run the plan
--run executes the plan one step at a time: one build owner, never two builds at once. A failed build skips the remaining steps; failed tests or scripts are reported and the rest still run. Every step runs with KROYS_NO_DIALOGS=1, KROYS_NET_BIND=127.0.0.1 and a fresh KROYS_USER_SETTINGS_DIR, so nothing opens a dialog, a port to the network or touches your own editor settings.
Logs and summary.json go to Artifacts/Validation/<timestamp>/. The tool prints PASS or FAIL per step and exits non-zero when any step failed, so it can run in the background and be checked when it finishes.
PowerShell
python Tools/SelectValidation.py --run
The map
Config/ValidationMap.json holds the rules. A changed file is matched against fullTriggers first, then against areas in order; the first matching area wins, so specific areas (an editor tool) come before general ones (the whole editor). An area names its build targets, CTest labels (each case carries Kroys<Subsystem> from Config/RegressionFixtures.json), an optional tests name pattern, scripts, and flags for the short-path run and the documentation check.
Config/ValidationMap.json
{ "name": "streaming", "paths": ["KroysEngine/Streaming/**"],
"targets": ["KroysRuntimeTests", "KroysEditorTests"], "labels": ["KroysStreaming"],
"scripts": ["StreamingWorldAcceptance"] }Keeping the map complete
Kroys.ValidationMapSync (part of the smoke set) runs python Tools/SelectValidation.py --verify-map and fails when:
- a CTest case is selected by no area and not by the smoke set,
- a
Tools/Run*Acceptance.ps1script has no entry, is not part of full validation or is selected by no area, - a tracked source file, and so a top-level source directory, matches no rule,
- an area's labels select cases of a program the area does not build, or a label or pattern selects nothing,
- a target named in the map does not exist in the build.
When you add a test, an acceptance script or a module, add it to the map in the same change.
KroysEngine 0.1.0 documentation, built from the engine repository by Tools/BuildDocsSite.py.