Skip to content

Latest commit

 

History

History
248 lines (175 loc) · 31.6 KB

CODEBASE_OVERVIEW.md

File metadata and controls

248 lines (175 loc) · 31.6 KB

Codebase Overview

Where things live in the GdPlanningAI repository, for contributors: the folder layout, the addon's files, how the Rust engine and the GDScript code work together, the classes, what happens at runtime, and how to build and test.

For the search itself, see ALGORITHM.md; for class diagrams, ARCHITECTURE.md; for authoring, AUTHORING_GUIDE.md.


1. Repository layout

The top-level of the repo:

Path Purpose
addons/ The plugin itself: addons/GdPlanningAI/ is the installable addon.
docs/ Project documentation (see section 9).
examples/ Demo scenes (2D hunger and campfire, 3D campfire, character showcase).
script_templates/ Godot script templates for subclassing Action, Goal, GdPAIObjectData, and others.
scripts/ Developer scripts (necro checks, docs generation, addon packaging, release tooling, Godot/addon install, compat smoke test).
test/ GUT GDScript test suite (test/unit/, test/integration/).
media/ The README's illustrations and the brand logos.

Other files of note at the root: Makefile (see section 6), project.godot (the dev project, on the pinned Godot 4.7; the addon floor is 4.2, see section 2.1), addons.jsonc (third-party addon manifest, gut v9.6.0 + GdTimeMachine tracking main), and .pre-commit-config.yaml (hooks config, run with prek).


2. Addon files

addons/GdPlanningAI/ is the Godot addon that gets copied into a user's project. It is also a full Godot project in its own right when opened at the repo root. The key files:

addons/GdPlanningAI/
├── plugin.gd                 # EditorPlugin; registers the autoload singleton
├── plugin.cfg                # Plugin metadata (name, version, log_level)
├── gdpai_autoload.gd         # Autoload singleton GdPAIAutoload
├── gdpai_utils.gd            # GdPAIUTILS helpers (get_child_of_type, etc.)
├── gdplanningai.gdextension  # GDExtension config; points at bin/<platform> libs
├── bin/                      # Built Rust shared libraries (linux/macos/windows)
├── rust/                     # Rust planning engine crate (see section 3)
└── scripts/
    ├── gdpai_rust_bridge.gd      # GdPAIRustBridge serialization layer
    ├── gdpai_blackboard_plan.gd  # GdPAIBlackboardPlan resource
    ├── nodes/                    # GdPAIAgent, GdPAIWorldNode, GdPAIObjectData,
    │                             #   GdPAILocationData, GdPAIInteractable
    ├── refcounteds/              # Action, Goal, Precondition, specs, PropertyUpdater
    └── resources/                # GdPAIAgentConfig, GdPAIBehaviorConfig
  • plugin.gd is @tool extends EditorPlugin. Its only job is to register the GdPAIAutoload autoload in _enter_tree.
  • gdpai_autoload.gd creates the GdPAIPlanScheduler in _ready and calls _scheduler.process_callbacks() every frame in _process. This call passes results and callback requests between the background planner threads and the main thread. It also applies the log level from plugin.cfg.
  • gdplanningai.gdextension declares the entry symbol gdext_rust_init, compatibility_minimum (the oldest supported Godot, equal to the godot crate's api-4-N feature; make test-compat runs on it), and loads the platform shared library from addons/GdPlanningAI/bin/{linux,macos,windows}/.
  • The Rust-exposed classes (GdPAIBlackboard, SimObjectProxy, GdPAIPlanScheduler) are registered via the .gdextension; they have no .gd file.

2.1 Godot version floor

The addon's supported floor is Godot 4.2. addons/GdPlanningAI/rust/Cargo.toml builds against the api-4-2 feature and gdplanningai.gdextension sets compatibility_minimum = "4.2"; both are set to 4.2. make test-compat and the CI compat job run test/compat/smoke.gd on that 4.2 editor, so the shipped library and the GDScript it calls are exercised at the floor.

The repository around the addon is not 4.2. global.json pins the development and test engine (4.7.2); project.godot carries its 4.7 features (and 4.4+ UID paths), GUT and GdTimeMachine need a newer editor, and the 2D example prefabs (examples/source_2d/prefabs/scenery.tscn and huge_scenery.tscn) use TileMapLayer (Godot 4.3+) in format-4 scenes (4.4+). Those are dev-only: the addon copied into a user's project, and script_templates/, keep to the 4.2 API. The godot CI matrix runs the pinned 4.7 on Linux, Windows and macOS; the compat job is the 4.2 leg.


3. Rust and GDScript

The planner is written in Rust; everything users write is GDScript.

3.1 GDScript API (what users write)

Everything under scripts/ is the user-facing API. Users subclass Action, Goal, GdPAIObjectData, Precondition, RequirementSpec, ProvisionSpec, PropertyUpdater, GdPAIBehaviorConfig, and GdPAIAgentConfig to describe what their agents can do. GdPAIAgent and GdPAIWorldNode are the runtime nodes. GdPAIRustBridge converts GDScript objects into the data the Rust engine reads. See section 4 for the full class map.

3.2 Rust planning engine

The Rust crate lives in addons/GdPlanningAI/rust/. Crate gdplanningai-rust, edition 2024, crate-type = ["cdylib", "lib"]. It depends on godot (gdext), built against the API of the oldest supported Godot through an api-4-N feature, and rayon for its thread pool. The GDExtension entry point GdPlanningAIExt calls logger::init_log_channel() at InitLevel::Servers.

Module map of rust/src/:

Module Role
lib.rs Crate root, module declarations, GDExtension entry.
planner/mod.rs Public surface (PlannerEngine, EngineInputs, EngineConfig, TerminationStrategy); the Dijkstra priority.
planner/engine.rs PlannerEngine, the resumable search loop (step_search, process_simulation).
planner/expander.rs Candidate discovery (find_candidates, lookup_or_start_discovery).
planner/simulation.rs StepResult/SimResult, eval_precondition, simulate_action.
planner/types.rs PlanBranch, SearchNode, PriorityNode, SearchContext, insertion logic.
plan_types.rs ActionSpec, GoalSpec, PreconditionSpec, callback channel messages.
requirement.rs RequirementSpec/ProvisionSpec enums, matching and holding predicates.
scheduler.rs GdPAIPlanScheduler: job lifecycle, callable registry, dispatch.
snapshot.rs VariantSnapshot, SimObjectData, BlackboardSnapshot (send-safe mirrors; main-thread conversion to and from Godot types).
gdpai_blackboard.rs GdPAIBlackboard Godot class (agent/world state).
precondition.rs PreconditionEvaluator (main-thread reconstruction for the pre-filter).
plan_tree.rs PlanResult, the plan handed back to GDScript.
sim_object_proxy.rs SimObjectProxy, a world-object snapshot.
debug_tree.rs TreeDump, search-tree dump structures backing the get_debug_tree text output.
logger.rs Tiered logging macros; messages from planner threads travel over a global channel and are printed by the scheduler on the main thread.

3.3 How the two languages communicate

GdPAIRustBridge.serialize_actions() and serialize_goals() turn GDScript actions and goals into dictionaries that embed their Callables; GdPAIPlanScheduler.submit_plan registers those callables, snapshots the blackboards and runs the search on a Rayon worker. Each frame, GdPAIAutoload._process() calls scheduler.process_callbacks(), which answers the worker's callback requests on the main thread and delivers finished plans to agent._on_plan_ready(dict). ALGORITHM.md sections 6 and 9 cover the protocol, snapshots and threading.


4. Class hierarchy overview

The GDScript classes in scripts/, grouped by base type:

Node classes (scene nodes)

Class File Notes
GdPAIAgent scripts/nodes/gdpai_agent.gd The agent node. Two blackboards (self + world), goals, actions, plan execution.
GdPAIWorldNode scripts/nodes/gdpai_world_node.gd Sources world state; rescans the scene tree for GdPAIObjectData nodes.
GdPAIObjectData scripts/nodes/gdpai_object_data.gd Base for world objects; broadcasts provided actions.
GdPAILocationData scripts/nodes/gdpai_location_data.gd Extends GdPAIObjectData; 2D/3D position and rotation.
GdPAIInteractable scripts/nodes/gdpai_interactable.gd Extends GdPAIObjectData; interaction distance and drift limits.

RefCounted classes you extend

Class File Notes
Action scripts/refcounteds/action.gd Planning-time overrides (get_validity_checks, get_action_cost, get_preconditions, get_requirements, get_provisions, simulate_effect) and execution-time overrides (pre/perform/post_perform_action).
GoToAction scripts/refcounteds/goto_action.gd Extends Action. Wildcard at_target provision; handles navigation via inject_binding.
Goal scripts/refcounteds/goal.gd compute_reward, get_desired_state.
PropertyUpdater scripts/refcounteds/property_updater.gd update_properties every frame, initialize once.
Precondition scripts/refcounteds/precondition.gd Base plus static factories (agent, world state, world object proxy, custom).
PreconditionBuiltin scripts/refcounteds/precondition_builtin.gd Evaluated natively in Rust; no callback.
PreconditionCustom scripts/refcounteds/precondition_custom.gd Evaluated via a main-thread callback.
PreconditionCustomWithDeps scripts/refcounteds/precondition_custom_with_deps.gd Adds dependent_object_ids for Rust liveness validation.
RequirementSpec + subclasses scripts/refcounteds/requirement_spec*.gd binding_exists, binding_equals, binding_in_set, fact.
ProvisionSpec + subclasses scripts/refcounteds/provision_spec*.gd binding, fact, fact_wildcard (the wildcard is how GoToAction satisfies any location requirement).

Resource classes (config)

Class File Notes
GdPAIAgentConfig scripts/resources/gdpai_agent_config.gd Planning strategy (CONTINUOUS, ON_INTERVAL, ON_DEMAND, ON_INTERVAL_FORCED), failure backoff, max recursion, iteration budget, blackboard plan, behavior configs.
GdPAIBehaviorConfig scripts/resources/gdpai_behavior_config.gd Groups goals, self actions, and property updaters into reusable behavior modules.
GdPAIBlackboardPlan scripts/gdpai_blackboard_plan.gd Template for agent blackboard properties; generate_blackboard().

Rust-exposed classes (no .gd file)

  • GdPAIBlackboard: set/get/erase/has_property, set_dict, get_object_for (returns a SimObjectProxy snapshot), group queries. The reserved GDPAI_OBJECTS key holds arrays of GdPAIObjectData nodes.
  • SimObjectProxy: a send-safe world-object snapshot (uid, groups, properties).
  • GdPAIPlanScheduler: the planning job manager and the callable dispatcher.

Internal (do not use)

Action.set_state/get_state/erase_state/has_state (namespaced state on the agent blackboard), all GdPAIAgent underscore members, GdPAIRustBridge (the agent owns it), and the base to_bridge_dict implementations.


5. Runtime flow

The agent lifecycle, from configuration to execution:

  1. Config to blackboard: GdPAIAgent._ready() builds the agent blackboard from config.blackboard_plan, sets entity and the GDPAI_OBJECTS entry (children in the GdPAIObjectData group under the entity), applies each behavior config via apply_to_agent (which appends goals/self actions and calls initialize on property updaters), finds the world node, and creates the Rust bridge.
  2. Per-frame updates: _process(delta) runs property updaters (hunger decay, stamina regen), checks the planning strategy, and drives plan execution.
  3. Plan submit: _start_plan_async() refreshes GDPAI_OBJECTS, collects self_actions + world-provided actions, bumps a generation counter, and calls scheduler.submit_plan(...) with the serialized actions and goals and the limits from GdPAIAgentConfig (max_recursion, iteration_budget, time_slice_ms, max_effect_fallbacks). The search runs on a background thread.
  4. Plan ready: _on_plan_ready(result) is generation-guarded so stale results from a superseded submission are discarded. It deserializes the plan into {action_chain, bindings_by_position}, sets the current goal from result.goal_index, and clones the actions via clone_for_plan().
  5. Execution: plans run in three phases. Pre: every action's pre_perform_action runs; any FAILURE aborts the plan. Action: the current action's perform_action(agent, delta) returns SUCCESS (advance), RUNNING (stay), or FAILURE (abort). Post: every action's post_perform_action runs as guaranteed cleanup.

Planning strategies: CONTINUOUS replans each frame once the current plan is done; ON_INTERVAL replans on a timer; ON_INTERVAL_FORCED replans every tick regardless of plan state; ON_DEMAND only plans when manually_start_plan() is called. failure_backoff delays the automatic strategies after a failed plan.


6. Build system

Prefer make targets over ad hoc commands. Two Makefiles exist.

Root Makefile

Target Purpose
test, test-rust, test-godot Run all / Rust / Godot suites. test runs both and exits nonzero if any sub-suite fails. test-godot exits 0 with a GUT-SUITE-OK/GUT-SUITE-FAILED verdict line (checks load/parse error markers, gut exit status, and that tests ran); test-rust propagates cargo test's exit code with a CARGO-TEST-OK/CARGO-TEST-FAILED verdict.
test-godot-pipe-output Full GUT log written to test_output.log.
test-e2e End-to-end example-scene cases (e2e=true in test/scenarios/scenes.cfg), one after another; a minute each at 60 fps.
test-compat Smoke-test the built library on the oldest supported Godot (Linux; downloads that editor).
check Run every hook in .pre-commit-config.yaml on every file, as CI's precommit job does (formatters may rewrite, then it fails).
clean Remove intermediates (media/captures, logs, instrumented profiles); bins, caches and addons stay.
necro, necro-fix Check every file against necro.yaml (house rules and packs: rustfmt, clippy, gdformat and gdlint among them) / necro --fix (rustfmt, gdformat, GDScript spacing).
check-docs, sync-docs Diff/sync README + LICENSE between the repo root and the addon copy.
launch-editor, addons-install, godot-pin, godot-install-pinned Editor and Godot version management (GodotEnv). Recording is via the GdTimeMachine addon.

addons/GdPlanningAI/rust/Makefile

Target Purpose
build-release cargo build --release for this OS (macOS: universal via lipo), then copies the library to addons/GdPlanningAI/bin/{linux,macos,windows}/.
build-debug, clean-binaries Debug build and cleanup.
test cargo test.
clean, doc Cargo clean and cargo doc.

Every target sets RUSTFLAGS with --remap-path-prefix, so the built binaries carry no local paths. Build through these targets; a plain cargo call without the flags rebuilds everything.

Build artifact flow

make build-release produces the shared library for the current OS and copies it into addons/GdPlanningAI/bin/<platform>/. gdplanningai.gdextension loads the platform-appropriate file at runtime (.so on Linux, .dll on Windows, .dylib on macOS). Built libraries are never committed: bin/ ignores them, a hook in .pre-commit-config.yaml rejects them, and CI runs the hooks over every file.

Release binaries are built by CI (build.yml): Linux and macOS (universal, lipo of aarch64 and x86_64) on their runners, Windows natively with MSVC. The release workflow packages those fresh builds into GdPlanningAI-X.Y.Z.zip; nothing under bin/ is committed or shipped from git, so there is no committed copy to lag the Rust sources.

Local development: there is no committed library, so build it for your platform with make -C addons/GdPlanningAI/rust build-release before running the editor or the GDScript tests, and again after any Rust change or pull. A mismatched library shows up as a startup warning and as a failing test_engine_version. Do not commit the result; CI builds the release binaries.


7. Testing

Two independent suites, plus a smoke test on the oldest supported Godot. Each test proves a behavior listed in BEHAVIORS.md.

make test-compat (Linux; the compat job in CI) installs the Godot in compatibility_minimum, lays the addon out as the release zip does in a throwaway project, and runs test/compat/smoke.gd: one agent plans and runs a two-step chain. GUT is not used there, since it needs a newer Godot.

GDScript suite (GUT)

Configured by .gutconfig.json (dirs ["res://test"], include_subdirs true). Run with make test-godot (headless GUT via addons/gut/gut_cmdln.gd).

  • test/unit/: blackboard and SimObjectProxy property operations; agent config (planning strategy switching, failure backoff, log level); and single example classes (GoToAction, pickup, hunger and fire goals).
  • test/integration/: async planner behavior (empty plans, satisfied goals, cheapest plan selection, cancellation of stale submissions); requirements/provisions chaining (Pickup to Eat, GoTo wildcard chaining, binding sets, cost comparisons); and example scenes as data: test_scene_scenarios.gd runs each case in test/scenarios/scenes.cfg (every scene loads and settles; hunger_basic_2d and campfire_2d run sped up until each loop action has completed), and test_example_scenes.gd holds the checks that look inside a scene (camera rig, guide, home-fire pairs). End-to-end multi-agent cases (e2e=true) run only in make test-e2e (CI runs it on the Linux leg of the godot job): a minute each at 60 fps, then every agent must have finished plans, nobody starved and every fire kept burning.

Rust suite

In addons/GdPlanningAI/rust/tests/ (plus common/mod.rs helpers), run with make test-rust. Tests build engines through PlannerEngine::new, as the scheduler does, and answer callbacks from common's responder thread, so no Godot runtime is needed. Planner cases are data: tests/scenarios/<slug>.toml, one per behavior in BEHAVIORS.md, run by tests/scenarios.rs (format in tests/scenarios/README.md); the other test files cover what fixtures can't express (time slices, budgets, cancellation, callback counts, logging, the debug tree). The few states PlannerEngine::new can't reach are white-box #[cfg(test)] modules beside the code.

Coverage: make test-rust-coverage measures src/ under the Rust tests, and make -C addons/GdPlanningAI/rust coverage-behavioral (Linux) under the GUT and e2e suites against an instrumented library. CI uploads both to Codecov (flags rust-tests, behavioral) as a signal, never a gate.


8. Developer tooling

  • Formatting and linting: make necro-fix (necro --fix: rustfmt, gdformat, GDScript spacing), make necro (house rules plus the rust, rust-style, gdscript and gdscript-style packs: rustfmt, clippy with warnings failing, and for GDScript a := ban, borders, docstrings, spacing, gdformat and gdlint).
  • scripts/: check_effect_spec.py, check_behaviors.py, check_godot_floor.py, check_godot_comments.py, check_readme_safety.py (house rules necro runs).
  • Hooks (.pre-commit-config.yaml, run with prek at commit; tests are run by hand): the standard pre-commit hooks, gitleaks, mdformat (--wrap=no), ruff, shellcheck, necro, check-docs (README/LICENSE drift), no-built-binaries and no-ai-attribution (commit-msg).
  • Third-party addons (addons.jsonc): gut v9.6.0 (bitwes/Gut) + GdTimeMachine main (WahahaYes/GdTimeMachine), installed via GodotEnv, gitignored, cached under .addons/. All three editor plugins (GdPlanningAI, gut, GdTimeMachine) are enabled.
  • project.godot: the dev project, on the pinned Godot 4.7 (section 2.1), Forward Plus renderer, no main scene (run scenes directly). Autoloads: GdPAIAutoload, GdTimeMachineGracefulStop. Its [dotnet] section is written by the .NET build of the editor; the project has no C#.
  • CI (.github/workflows/): build.yml runs on pushes to main and pull requests, path-filtered into rust/godot legs. A precommit job runs the prek hooks over all files plus make docs-check, gitleaks over the pushed commits, and the commit-message attribution check; a rust job runs make test-rust-coverage (coverage uploads to Codecov as a signal, never a gate); a godot job with a linux/windows/macos matrix builds the per-OS library and runs make test-godot-verbose (the Linux leg also runs make test-e2e and reports behavioral coverage); and a compat job runs make test-compat, a smoke test on the oldest supported Godot. release.yml calls build.yml on vX.Y.Z tags and publishes GdPlanningAI-X.Y.Z.zip with the freshly built libraries; pr-cache-cleanup.yml drops a closed PR's caches.

9. Documentation map

Doc Purpose
README.md What GdPlanningAI is, installation, examples, credits.
docs/AUTHORING_GUIDE.md How to write actions, goals, preconditions and object data.
docs/EXAMPLES.md What each demo scene shows and which scripts to read.
docs/ALGORITHM.md How the planner searches, with pseudocode of the engine.
docs/ARCHITECTURE.md Class diagrams, generated from docs/generated/ (make docs).
docs/BEHAVIORS.md What the addon promises, each behavior with the tests that prove it.
docs/CODEBASE_OVERVIEW.md This file: where things live, building and testing.
docs/DOCUMENTATION_GUIDELINES.md Comment and docstring style.

A good reading order for a new contributor: this file, then ALGORITHM.md, then EXAMPLES.md for a concrete end-to-end picture.