Skip to content

Latest commit

 

History

History
834 lines (645 loc) · 46.7 KB

README.md

File metadata and controls

834 lines (645 loc) · 46.7 KB

Flutter Scene VR fork

Experimental VR fork of bdero/flutter_scene, maintained by Adrian Moisa. This repository works with the companion flutter_vr engine fork to demonstrate Flutter Scene on the web and in native Meta Quest VR. The original Flutter Scene README is preserved below this addendum.

Demo setup snapshot: 31 August 2026.

Why we forked Flutter and Flutter Scene

While adding VR to Visual Space, we wanted to keep one 3D engine across the ordinary application and the headset. Maintaining Flutter Scene on web/desktop and a second native world renderer on Quest would duplicate scene features, materials, lighting, shadows, and performance work.

The first Flutter Scene VR path rendered stereo images through Flutter's compositor into an Android surface, then copied them into the headset's eye images. Flutter Scene was already drawing real GPU geometry; the extra cost was in presentation, buffering, copies, and synchronization.

These two forks let Flutter Scene render directly into OpenXR-owned eye targets. The Flutter fork supplies generic external GPU surfaces and Android GLES interoperation. This fork supplies render-to-target support, stereo frame coordination, an OpenXR plugin, and the demo. OpenXR owns tracking, controllers, swapchains, and headset presentation. Flutter still supplies the widget panels.

Start the example on… Scene rendering and presentation
Chrome / web Flutter Scene's WebGL2 backend inside the ordinary Flutter gallery. No headset or native engine compilation is needed for this mode.
Meta Quest VR Flutter Scene → Flutter GPU/Impeller GLES → borrowed OpenXR eye images → native stereo presentation. Widgets use separate floating UI surfaces. Requires the custom compiled Flutter engine.

The native Quest world is not a web page or a flat screenshot placed in front of the user. Conversely, opening Chrome does not start a native OpenXR session or promise WebXR support. The shared renderer and examples have different platform presentation paths.

1. Get the forks and select Flutter

The documented engine-build host is macOS on Apple Silicon, targeting an ARM64 Meta Quest 3. Start with the Flutter VR setup guide: it covers SDK installation, PATH/IDE selection, engine dependencies, and the memory-conscious Android/host build. Keep stock Flutter installed separately. For a web-only first look, you can stop before compiling the native engine, but still use the fork SDK and matching Dart GPU source override below.

Use the same directory layout as that guide:

~/Projects/flutter_vr/         Flutter SDK and engine source
~/Projects/depot_tools/        Engine dependency/build tools
~/Projects/flutter_scene_vr/   This workspace and examples
export FLUTTER_VR_ROOT="$HOME/Projects/flutter_vr"
export PATH="$FLUTTER_VR_ROOT/bin:$PATH"
git clone --branch vr https://github.com/adrian-moisa/flutter_scene_vr.git "$HOME/Projects/flutter_scene_vr"
cd "$HOME/Projects/flutter_scene_vr"
git remote add upstream https://github.com/bdero/flutter_scene.git

Both forks keep their maintained VR changes on vr. Commit and push fork work only to origin/vr; leave master untouched. Explicitly select vr when cloning either fork, even if GitHub opens a different default branch. Maintainers should select vr as each fork's default branch in GitHub repository settings so repository visitors see the maintained code too.

For an existing clone, first commit or otherwise preserve any local work, then fetch origin. If local vr does not exist, create it with git switch --create vr --track origin/vr. If it exists, use git switch vr and git branch --set-upstream-to=origin/vr vr. Do not reclone or discard local changes. If origin/vr is missing, the maintainer must publish it first.

Use the fork revisions containing the VR changes. A stock Flutter installation meeting the upstream README's version requirement does not include the external surface API needed here. Do not switch channels or run flutter upgrade to resolve a missing VR API.

Select the matching flutter_gpu Dart sources

At this workspace's root, create a local pubspec_overrides.yaml containing:

dependency_overrides:
  flutter_gpu:
    path: /absolute/path/to/flutter_vr/engine/src/flutter/lib/gpu

Replace that path with the absolute path to your Flutter fork. YAML does not expand $HOME, $FLUTTER_VR_ROOT, or ~. If the file already exists, merge the flutter_gpu entry into its existing dependency_overrides; do not replace other overrides. This machine-specific file is ignored by Git.

flutter pub get

This resolves the monorepo's local workspace packages. The native deployment helper also creates this override when absent, but does not overwrite or repair an existing file. Set it up here so the web gallery works before native deployment. --local-engine selects native binaries; it does not select this Dart package.

The existing build hooks compile engine/gallery shaders, models, textures, and materials during the app build. You do not need to run a separate shader script or flutter_scene:init in these already configured examples. On the native local-engine path, the host output must contain impellerc and its sibling shader_lib directory. Do not copy in a compiler from an unrelated SDK. Rapier normally downloads a prebuilt physics library; a source fallback needs Rust/Cargo. Some gallery entries fetch external assets or require network access.

2. Start the main gallery on web

examples/flutter_app is the main example. Its platform scaffolding is generated locally; create the web files once in a fresh clone:

cd "$HOME/Projects/flutter_scene_vr/examples/flutter_app"
flutter create . --platforms=web
flutter run -d chrome

No local-engine flags are needed for Chrome. Select VR in the gallery to open the original VR demonstration scene as an ordinary interactive web scene; the gallery also contains the upstream material, lighting, model, and effect examples. Click the scene to focus it, drag to look around, use WASD or arrow keys to move, Shift to move faster, the wheel to dolly, and R to restore the authored camera. Overlays and settings keep their own input.

For Android Studio, open examples/flutter_app, set its Flutter SDK to the flutter_vr root, and use lib/main.dart as the entrypoint. If Dart is not detected, select <flutter_vr>/bin/cache/dart-sdk. Choose Chrome in the toolbar device selector and leave additional run arguments empty. Opening the whole Flutter engine source tree is unnecessary for running the gallery and can trigger expensive IDE indexing.

3. Connect a Meta Quest

Install Android SDK Platform Tools and put adb on PATH. The current Android plugin uses SDK 36, NDK 28.2.13676358, CMake 3.22.1 or newer, and ARM64; follow the companion Flutter guide for the host/toolchain setup.

  1. Complete Meta's developer account/device setup and enable Developer Mode for the headset in the Meta Horizon app. Follow Meta's device setup instructions for the current account requirements and menus.
  2. Connect Quest with a USB data cable, wake/unlock it, and accept the USB debugging authorization inside the headset. File-transfer permission is not the same authorization.
  3. Check the connection:
adb devices -l

Your headset must be listed as device, not unauthorized or offline. For unauthorized, put the headset on and approve debugging. If nothing appears, check Developer Mode, the cable/USB port, and whether another ADB installation is conflicting. Select a particular headset when several are connected:

export ANDROID_SERIAL='YOUR_HEADSET_SERIAL'
adb -s "$ANDROID_SERIAL" shell getprop ro.product.model

USB is the simplest first-run connection. For wireless development, connect and authorize over USB first, put the headset and computer on the same trusted network, then enable ADB over Wi-Fi in Meta Quest Developer Hub's Device Manager. Use the IP:port entry from adb devices -l as ANDROID_SERIAL, since the helper's automatic discovery looks for USB devices. Reconnect USB if the wireless session disappears after a restart. See Meta's ADB guide and MQDH device management.

4. Build, install, and enter native VR

First finish both engine builds in the Flutter fork guide. The native host is examples/openxr_quest; it reuses the main gallery's registry and examples. Its Android project is already checked in, so do not run flutter create there.

From this repository root:

cd "$HOME/Projects/flutter_scene_vr"
export QUEST_FLUTTER="$FLUTTER_VR_ROOT/bin/flutter"
export QUEST_LOCAL_ENGINE_SRC_PATH="$FLUTTER_VR_ROOT/engine/src"
export QUEST_LOCAL_ENGINE=android_profile_arm64
export QUEST_LOCAL_ENGINE_HOST=host_profile_arm64
export QUEST_BUILD_MODE=profile

./examples/openxr_quest/tool/quest-build-and-deploy-local-forks.sh

Set these paths explicitly: the helper's fallback paths refer to the maintainer's machine. It checks the engine artifacts, resolves dependencies, builds the ARM64 profile APK, stops the previous demo, installs the new APK, and launches the immersive OpenXR activity. It does not compile Flutter's engine for you. Keep the headset awake and its controllers available while starting.

The APK is written to examples/openxr_quest/build/app/outputs/flutter-apk/app-profile.apk. It is a development demo, not a store-ready distribution. Use the local-forks helper; the other deployment helper is for an environment where the required engine APIs are already integrated.

The same Android app also has a flat gallery and an Enter VR action. Switching between Flat mode and VR retains the selected example, but restarts its authored settings: tuned slider state does not transfer between the separate Android processes.

Inside the headset

Control What it does
Examples Select a scene without leaving the immersive session. Choose VR for the original demonstration.
Controls Show that example's Flutter controls; VR → Controls → Enable shadows toggles shadows live.
Settings Open shared graphics and resolution controls.
Aim + trigger Click panel buttons, checkboxes, and other controls.
Aim at a panel + that hand's stick Scroll the panel. Point away from panels to navigate the scene.
Left stick / right stick Orbit the scene target / move the camera rig and target. Physical head tracking remains independent.
Aim + hold grip Move a floating panel; use the same hand's stick to push it away or pull it closer, then release to leave it there.
Recenter / panel-reset icon Reset the camera / restore the floating-panel layout.
Small native FPS / UI ON button Hide/suspend Flutter widget panels; trigger again to restore them. Scene rendering and statistics continue.

Not every gallery example or interaction has a native VR mapping. The current registry keeps Multiplayer, DICOM Volume, External Texture, and Split Screen flat-only; other examples can have interaction or device-service limitations. The gallery report separates adapted examples from physically verified ones.

5. Show the demo on a laptop

Use Meta Quest Developer Hub (MQDH) to connect the headset and cast/record the running VR experience on your computer. Its Device Manager provides casting, screenshots, and video capture; see Meta's MQDH debugging tools. If MQDH and the terminal disagree about connected devices, configure MQDH to use the same Android SDK adb executable.

Casting is a view of the application running on Quest; this workflow does not require Quest Link or PC VR streaming. Operate VR with the headset controllers. For a laptop-only interactive presentation, run the Chrome gallery and select the same example. The browser and headset are separate app sessions, not automatically synchronized controls. A cast is useful for explaining the demo, but headset comfort, stereo depth, sharpness, and tracking must be judged while wearing the headset. Keep the headset's safety boundary enabled when using VR.

Settings and performance

This is a working rendering demo, not yet a highly optimized VR engine. Some demanding gallery scenes remain slow. In the maintainer's original VR demo, enabling shadows reduced the observed frame rate from roughly 70 FPS to 40 FPS. This is a scene/device/settings-specific observation, not a controlled benchmark or a promise for other machines and examples.

Start with VR, compare shadows off/on, then explore heavier examples. Shadows currently start enabled. To build with the original VR entry's shadows initially disabled (other entries retain their authored defaults):

QUEST_SHADOWS=0 ./examples/openxr_quest/tool/quest-build-and-deploy-local-forks.sh

You can also use its live checkbox without rebuilding. Settings exposes 50%, 67%, 75%, and 100% resolution, plus Low / Medium / High / Ultra / Custom graphics presets. On Quest those percentages scale the runtime-recommended eye dimensions; on web they scale the current physical viewport render size. They do not resize Flutter's UI to the same percentage. Half the width and height means about one quarter of the scene pixels.

Presets change resolution and effects together. For an isolated comparison, hold resolution and all other settings fixed while changing one effect. Manual changes select Custom, saved per example for the current app session; restarts and flat/VR process transitions do not preserve it. Ultra is a quality comparison, not a recommended Quest performance baseline. See the Quest gallery guide for the current preset values.

The performance panel reports frame rates, timing, and actual render/atlas sizes. The web FPS button is beside Settings. Distinguish scene submissions, completed stereo frames, Flutter widget frames, runtime refresh rate, and GPU time: a 72 Hz OpenXR loop alone does not prove 72 fresh displayed scene frames. Runtime-recommended eye dimensions are not the headset's physical panel size. Compare the same scene, viewpoint, resolution, effects, build mode, and warmed-up headset conditions; exclude scene-switch and resolution-transition samples.

The native FPS / UI button helps isolate widget/panel cost by suspending framework frame callbacks and panel texture work. It keeps the Flutter engine, Dart isolate, Flutter GPU scene rendering, simulation, and stats alive. It does not measure a renderer with Flutter removed. Widget-texture examples retain their last captured widget image while UI is suspended.

Work already present in this fork includes:

  • Direct eye output, removing the old full-size stereo Flutter surface and native eye-copy stage while keeping widget surfaces separate.
  • One scene update for both eye views, bounded diagnostic history, and the ability to suspend UI work for comparisons.
  • Conditional sharing of compatible shadow work between eyes, static shadow caster hints, and a bounded shadow setup for the original VR scene.
  • Reduced intermediate work through single-channel depth passes, retained per-eye scene-color history, and shader/material changes that avoid some redundant texture sampling and depth-of-field calculations.

These are implemented changes, not a claimed measured speedup for every scene. Sharing shadows has already been attempted and exists for compatible views, but it does not remove shadow receiver sampling, filtering, or all update costs. Shadows remain a major area for investigation. Next steps include profiling individual GPU passes, improving shadow reuse/filtering, reducing intermediate buffer bandwidth, tuning expensive effects for mobile GPUs, and investigating multiview/foveation and less blocking synchronization. Those are ongoing or future directions, not completed performance guarantees.

Troubleshooting and keeping the forks current

Symptom Check first
Missing Surface, SurfaceFrame, or openExternalSurface The local flutter_gpu override, selected Flutter SDK, and compatible fork revisions; then rerun flutter pub get.
Helper reports missing engine artifacts Build both full engine targets; confirm source path, output names, and profile/debug/release agreement.
Shader compiler or asset build fails Read the first hook error. Keep the matching host impellerc and shader_lib together; let the existing package hooks own generated assets.
No authorized Quest Developer Mode, the in-headset debugging prompt, USB data cable, adb devices -l, and ANDROID_SERIAL.
App opens only as a flat Android window Enter VR or use the local-forks helper's immersive launch. A normal Android launch alone is not an OpenXR session.
Poor FPS Begin with the VR entry at lower resolution/effects, compare shadows and UI independently, and inspect timing rather than refresh rate alone.

The intent is to contribute both forks upstream and periodically rebase them while that work proceeds. The Flutter engine API requires broader design, lifecycle, backend, and test review; the Scene changes are more contained but depend on that engine work. Neither upstream acceptance nor timing is promised. Performance and interaction validation remain part of the demo's development.

For a reproducible showcase, record both fork commit IDs. Preserve local work before updating, follow compatible fork revisions, and rebuild both engine outputs after engine/dependency changes. Refresh Scene dependencies and rebuild the app so its hooks regenerate the appropriate assets. Do not publish local SDK paths or assume an arbitrary upstream rebase remains compatible.

Update or publish vr

For developers consuming the published fork, run from this checkout:

git switch vr
git pull --ff-only origin vr
flutter pub get

Maintainers commit changes on vr and publish ordinary new commits with git push -u origin vr. To incorporate official changes, start from a clean working tree, fetch both remotes, review any remote-only commits, then rebase:

git switch vr
git fetch origin
git log --oneline --left-right vr...origin/vr
git fetch upstream
git rebase upstream/master

Use upstream for https://github.com/bdero/flutter_scene.git; add that remote if an older clone lacks it. Reconcile unexpected divergence before rebasing. After checking compatibility with the Flutter vr checkout, publish a rewritten history with git push --force-with-lease origin vr. Never push to master, main, or upstream. A failed lease requires reviewing the remote changes. Consumers whose fast-forward pull fails after a rebase or amend must reconcile with the published vr history; switching to master is not a fix.

For implementation details, see the direct-rendering design/history. Its dated measurements and earlier defaults are historical; use this addendum, the current gallery guide, and the current helper for setup and launch.

Original Flutter Scene README

The material below is preserved from upstream. Its published-package setup describes ordinary Flutter Scene use; use the fork instructions above for this VR demo.


Flutter Scene

Flutter Scene

A complete 3D game engine for Flutter, built on Flutter GPU

Rendering, physics, audio, animation, a scene editor with an MCP server, and a full asset pipeline, on every platform Flutter runs on, web included. Built and maintained by the author of Flutter GPU.

Covered by Argos Visual Testing

Website · Docs · Examples App · Games · FAQ

Physically based rendering of the glTF DamagedHelmet sample with image-based lighting in Flutter Scene

Dashsurfers, an endless runner game built with Flutter Scene

The Flutter Scene Editor, driven by a coding agent over MCP

At a glance

  • Rendering. Physically based materials, image-based lighting, directional, point, spot, and area lights with shadows, probe-based global illumination, and a full post-processing stack (bloom, depth of field, fog, god rays, screen-space reflections, color grading, SMAA and TAA). Decals, Gaussian splatting, instancing, LODs, particles, and sky materials.
  • Games. Physics (Rapier and box3d backends), positional audio (SoLoud and FMOD), multiplayer over dashwire, skeletal and morph animation with blending, camera controllers, a gameplay kit, raycast picking, and interactive Flutter widgets on 3D surfaces.
  • Tooling. A desktop scene editor with an MCP server for agent-driven editing, agent skills for coding assistants, a build-time asset pipeline, and hot reload for models, shaders, textures, and environments.
  • Formats. glTF import, the .fscene scene document, .fmat custom materials in GLSL, KTX2 compressed textures, and HDR/EXR environments.
  • Platforms. iOS, Android, macOS, Windows, Linux, and the web, where a built-in WebGL2 backend stands in for Impeller.

Built with Scene

  • Dashsurfers, an endless runner with skinned characters, physics, and particles.
  • Dashmap, a live 3D world map with streaming terrain, OSM buildings, and a physical sky.
  • The examples app, 43 runnable feature examples.

Why Scene exists

Scene began its life inside the Flutter Engine, as a C++ module for Impeller with a declarative widget interface exposed through a 3D API in the Flutter SDK. Flutter GPU was built initially to set this project free, providing the low-level GPU access needed to develop Scene outside the engine, as an ordinary ecosystem of Dart packages.

That origin shapes the project's philosophy. Scene is a full 3D game engine and toolkit that makes Flutter GPU practical to build on, spanning rendering, physics, audio, editor tooling, and asset support across every target Flutter runs on. And the relationship flows both ways by design. Scene's cross-backend render tests are where Flutter GPU regressions get caught, so Impeller and Flutter GPU get better because Scene exists.

The goal has always been to pave the way for advanced graphics in Flutter. In line with the goals set out in Flutter GPU's original design doc, Scene aims to de-fracture, unite, and elevate Flutter's graphics ecosystem, so that building incredible 3D experiences with Flutter and Dart no longer requires engine forks, complicated external renderer integrations, or giving up platforms to get advanced features.

Scene is built by the author of Flutter GPU, a former core Flutter engine team member who spent four years on Flutter, most of it building Impeller.

Getting started

flutter pub add flutter_scene
dart run flutter_scene:init

init sets up the asset pipeline, which is the recommended way to use Scene. Drop sources under assets/, load them by their source path, and render:

final level = await loadScene('assets/level.glb');
scene.add(level);
// ...
SceneView(scene, cameraBuilder: (elapsed) => PerspectiveCamera(...));

The same code runs on web, and the engine's shaders are compiled for you during the build by flutter_scene's own build hook. Rendering goes through Flutter GPU, which is off by default, so enable it once per platform (below). The web needs nothing.

Impeller, which Flutter GPU builds on, is the default renderer on every native platform as of 3.47, so there is nothing to do for it.

Coding agents

This package ships a set of agent skills so a coding assistant writes idiomatic Scene instead of guessing: correct usage and traps, the run-settle-capture verification loop, copy-paste look presets, procedural content, and performance. dart run flutter_scene:init offers to install them, and dart run flutter_scene:skills installs, updates, or checks them on their own without touching your build hook. Upgrading Scene can carry newer revisions; dart run flutter_scene:skills --check reports whether any are available.

Enable Flutter GPU

While developing, pass the flags on the command line:

flutter run --enable-flutter-gpu

To turn it on permanently, for every run and for the app you ship, edit the platform file:

Platform File Add
iOS ios/Runner/Info.plist <key>FLTEnableFlutterGPU</key><true/>
Android android/app/src/main/AndroidManifest.xml, in <application> <meta-data android:name="io.flutter.embedding.android.EnableFlutterGPU" android:value="true" />
macOS macos/Runner/Info.plist <key>FLTEnableFlutterGPU</key><true/>
Web nothing

Windows and Linux set it on the DartProject their runner builds. This needs Flutter 3.47.1.

// linux/runner/my_application.cc
g_autoptr(FlDartProject) project = fl_dart_project_new();
fl_dart_project_set_enable_flutter_gpu(project, TRUE);
// windows/runner/main.cpp
flutter::DartProject project(L"data");
project.set_enable_flutter_gpu(true);

On 3.47.0 there is no such setting, so desktop takes the command-line flags per run, and release builds compile the engine's environment switches out, meaning a shipped Windows or Linux release needs 3.47.1.

A scene with no assets

The built-in geometry needs no asset pipeline, so a cube renders straight after flutter pub add flutter_scene:

import 'package:flutter/material.dart';
import 'package:flutter_scene/scene.dart';
import 'package:vector_math/vector_math.dart' as vm;

void main() => runApp(const MaterialApp(home: CubeView()));

class CubeView extends StatefulWidget {
  const CubeView({super.key});

  @override
  State<CubeView> createState() => _CubeViewState();
}

class _CubeViewState extends State<CubeView> {
  final Scene scene = Scene();
  bool ready = false;

  @override
  void initState() {
    super.initState();
    // Geometry and materials touch the shader bundle, so build them once the
    // engine's static resources are up.
    Scene.initializeStaticResources().then((_) {
      scene.add(
        Node(
          mesh: Mesh(
            CuboidGeometry(vm.Vector3(1, 1, 1)),
            PhysicallyBasedMaterial(),
          ),
        ),
      );
      if (mounted) setState(() => ready = true);
    });
  }

  @override
  Widget build(BuildContext context) {
    if (!ready) return const SizedBox.expand();
    return SceneView(
      scene,
      camera: PerspectiveCamera(position: vm.Vector3(2, 2, -4)),
    );
  }
}

The scene's default studio environment lights it, so there is nothing else to set up.

Moving a node

Every Node carries a transform relative to its parent. Read and write it one component at a time, or as a whole matrix through node.localTransform.

node.position = vm.Vector3(0, 1, 0);
node.rotation = vm.Quaternion.axisAngle(vm.Vector3(0, 1, 0), 0.5);
node.scale = vm.Vector3.all(2);
node.position += vm.Vector3(0, 0.1, 0);

Each getter returns a copy, so node.position.y = 1 moves nothing. Assign the value back instead. Debug builds throw on an edit that cannot reach the node, whether it is to a returned copy or to localTransform in place.

Built-in geometry

Every class below builds vertex data for you and drops into a Mesh the same way CuboidGeometry does above.

  • Primitives: CuboidGeometry, SphereGeometry, IcosphereGeometry, CapsuleGeometry, CylinderGeometry, TorusGeometry, PlaneGeometry, DiscGeometry, RingGeometry, WedgeGeometry.
  • Swept along a path or profile: ExtrudeGeometry, TubeGeometry, RibbonGeometry.
  • Lines and camera-facing quads: PolylineGeometry, LineSegmentsGeometry, BillboardGeometry.
  • Your own vertex data: MeshGeometry and GeometryBuilder.

The asset pipeline

init writes a hook/build.dart that converts your assets at build time, creates flutter_scene_generated/ with a .gitignore for its outputs, and adds that one directory to flutter.assets in your pubspec.yaml. It is safe to run again, and it will not overwrite a hook/build.dart you wrote yourself. It prints a block to paste into your existing build() callback instead.

The hook it writes discovers .glb and .fscene models, .fmat materials, and loose images under assets/, and converts each one:

// hook/build.dart
import 'package:flutter_scene/build_hooks.dart';
import 'package:hooks/hooks.dart';

void main(List<String> args) async {
  await build(args, (input, output) async {
    buildScenes(buildInput: input, buildOutput: output);
    await buildMaterials(buildInput: input, buildOutput: output);
  });
}

The one line it adds to your pubspec.yaml:

flutter:
  assets:
    - flutter_scene_generated/

Upgrading from an earlier version, run it again. Generated assets now go into that directory on every Flutter release, so re-running init migrates the hook it wrote and adds the pubspec entry. A hook still asking for a removed asset mode fails the build and names its replacement.

From then on, drop sources under assets/ and load them by source path:

final level = await loadScene('assets/level.glb');
final toon = await loadFmatMaterial('assets/toon.fmat');
final ground = await loadTexture('assets/ground.png');

A .glb has to be parsed and unpacked into GPU-ready form every time the app loads it. The pipeline does that work once, at build time, into the .fsceneb format the engine reads directly, so loading a model at runtime costs far less. Prefer it for anything that ships with your app. It is also how .fmat custom materials and block-compressed textures with full mip chains reach you, and editing any source reconverts just that source and hot reloads it.

Loose images cook to the engine's own .fstex by default. For a smaller download, cook color textures to standard ETC1S KTX2 instead, several times smaller at some cost in quality:

buildTextures(
  buildInput: input,
  buildOutput: output,
  textures: ['assets/ground.png', 'assets/ground_normal.png'],
  encoding: const TextureEncoding.etc1s(quality: 128),
  // Normal maps keep more detail in the default encoding.
  encodings: {'assets/ground_normal.png': TextureEncoding.universal},
);

Keep your sources in version control. The generated directory holds compiled output tied to the Flutter engine that built it, which is why the hook manages its .gitignore for you.

For a model that only exists once the app is running, because you download it or the user supplies it, import the .glb directly with Node.fromGlbAsset('assets/model.glb'). That needs no hook, and it parses the glTF on every load, so prefer the pipeline whenever the model ships with you.

Where to go next

fscene.dev carries the full documentation. Your first scene renders something on screen from here, and the guides cover each subsystem in depth with live demos, including assets and loading, materials, lighting and environment, animation, and cameras. The API reference documents every public symbol.

Requirements

Flutter Scene is pre-1.0 and evolving quickly. Minor releases can carry breaking changes, and every change is documented in the CHANGELOG.

  • Flutter 3.47 (stable) or newer. Rendering is built on Flutter GPU, which every platform except the web needs turned on once (see Enable Flutter GPU).
  • On native platforms rendering runs on Impeller, Flutter's default renderer on every native platform as of 3.47. The web has no Impeller, so the package ships its own WebGL2 backend and runs there without flags.

Features

Rendering

  • Physically based materials with clearcoat, sheen, anisotropy, and transmission, lit by image-based lighting (prefiltered radiance plus spherical-harmonic diffuse), with a built-in procedural studio environment so an imported model looks good with zero lighting setup.
  • Sky materials with live IBL rebaking, HDR/EXR environment import, smooth environment cross-fades, and parallax-corrected reflection probes.
  • Directional, point, spot, and rect area lights, shaded through per-view froxel clustering so a scene scales to many lights, with light channels to control what each light touches.
  • Shadows from directional, spot, and point lights. Cascaded directional shadows with percentage-closer soft shadows and screen-space contact shadows, cube-face point shadows, cached shadow tiles for static geometry, and alpha-masked casters.
  • Global illumination from a world-space irradiance probe field with probe visibility and offline or progressive baking, plus screen-space indirect light and ground-truth ambient occlusion.
  • A full post-processing stack, with HDR tone mapping, physical camera exposure, automatic eye adaptation, bloom with lens flares, fog, god rays, screen-space reflections, depth of field with bokeh, film-look color grading from .cube LUTs, chromatic aberration, and radial screen distortion.
  • Anti-aliasing with FXAA, SMAA, and temporal anti-aliasing, geometric specular anti-aliasing, and resolution scaling.
  • Projected box decals, 3D Gaussian splatting from .ply and .splat captures, instanced rendering, automatic geometry LODs, mesh chunking at import, and an allocation-light frame loop.
  • A particle system driven by configurable emitter and behavior modules.

Materials and shaders

  • A custom-material workflow (.fmat) covering fragment, vertex, and sky stages, with per-instance attributes, engine inputs, and shader hot reload.
  • Per-frame scene inputs for custom shaders, including scene depth, world position, and shadow data, plus a depth-aware and shadow-aware custom post-pass API.
  • Barycentric wireframe helpers and a noise library with matched CPU and GPU implementations.

Assets and animation

  • glTF (.glb and multi-buffer .gltf) import at runtime, or pre-converted at build time into the engine's .fsceneb format through build hooks, loaded by source path. Sparse accessors, KHR_materials_variants with instant switching, and KHR_texture_basisu.
  • The .fscene/.fsceneb scene description format, human-readable as text and fast to load as binary, with prefabs and declarative components.
  • KTX2 compressed textures with full mip chains, .fstex texture builds, standard ETC1S (Basis Universal) texture builds and transcoding, and HDR/EXR environment decoding.
  • Skinned meshes, morph targets, and a blended animation system with declarative per-clip playback control.
  • Hot reload for models, shaders, textures, environments, and scene documents.

App integration

  • A SceneView widget with both an imperative scene-graph API and a fully declarative widget API (SceneNode, SceneMesh, and SceneModel with async loading placeholders).
  • Orbit, fly, and follow camera controllers, raycast picking, and interactive Flutter widgets embedded on 3D surfaces.
  • A gameplay kit (package:flutter_scene/kit.dart) with a camera boom, character controller, day/night cycle, water surface, audio, pooling, and debug components.
  • Screen-reader accessibility, exposing scene content through Flutter semantics.
  • Render-target control, split-screen and multi-view layouts, synchronous frame capture as ui.Image, and platform textures (video, camera preview) as material inputs.
  • Geometry readback, procedural geometry builders with derivation operations, and a GPU memory report with explicit release.
  • Agent skills that teach coding assistants idiomatic Scene, installed by dart run flutter_scene:init.

Ecosystem

Gallery

A 3D platformer game running on Flutter Scene

Depth of field and bloom post-processing on a glTF model

Rigid-body physics through flutter_scene_rapier

Instanced rendering of thousands of animated meshes

A custom sky material written in GLSL through the .fmat format

Skinned, animated glTF characters cloned across a scene

3D Gaussian splatting rendered as a scene node

Dashmap, a streaming 3D world map built on Flutter Scene

FAQ

Q: What platforms does this package support?

On native platforms flutter_scene runs anywhere Impeller does. On the web it runs on a built-in WebGL2 backend.

Every native platform needs Flutter GPU turned on. Impeller, which it builds on, is already the default everywhere. Enable Flutter GPU has the file and the key for each.

On the web, no flags are needed; it works under both the CanvasKit and Skwasm renderers.

Platform Status
iOS 🟢 Supported
Android 🟢 Supported
Web 🟢 Supported
MacOS 🟢 Supported
Windows 🟢 Supported (3.47.1 to ship a release)
Linux 🟢 Supported (3.47.1 to ship a release)
Custom embedders 🟢 Supported

Q: How does web support work?

Impeller and Flutter GPU aren't available on the web, so flutter_scene ships a built-in WebGL2 backend (a drop-in for flutter_gpu) and renders through it there. It works under both the CanvasKit and Skwasm web renderers, with no extra flags or configuration.

Sponsors

Scene's development is funded by its sponsors, listed in full at fscene.dev/sponsors. Anyone can sponsor through GitHub Sponsors, from $5 a month. Company tiers are at fscene.dev/sponsors.

Silver

Primio

Infrastructure

  • Codemagic (macOS CI on Apple silicon)
  • Argos (visual regression testing)

To sponsor Scene, write to [email protected].

Repository

This repository is a pub workspace containing the engine, its companion packages, and the example apps:

Path Description
packages/flutter_scene The 3D engine, including the glTF importer, the .fscene format, and the web (WebGL2) backend. Published to pub.dev as flutter_scene.
packages/flutter_scene_rapier Rapier physics backend, shipping prebuilt native binaries and a wasm module. Published to pub.dev as flutter_scene_rapier.
packages/flutter_scene_box3d box3d physics backend. Published to pub.dev as flutter_scene_box3d.
packages/flutter_scene_soloud SoLoud audio backend. Published to pub.dev as flutter_scene_soloud.
packages/flutter_scene_fmod FMOD Studio audio backend. Published to pub.dev as flutter_scene_fmod.
packages/flutter_scene_net Multiplayer on dashwire, binding replicated state to scene nodes with interpolated transforms and in-app hosting. Published to pub.dev as flutter_scene_net.
packages/flutter_scene_input Rebindable input actions for keyboard, mouse, gamepads, and pointer lock, with contexts, rebinding, and scene drivers. Published to pub.dev as flutter_scene_input.
packages/scene The engine-agnostic .fscene document core and the physics contract the backends implement. Published to pub.dev as scene.
packages/flutter_scene_codegen Static extraction and code generation for annotated components. A development-time tool, not published.
packages/flutter_scene_editor_core, packages/flutter_scene_editor, packages/flutter_scene_mcp The Flutter Scene Editor stack (headless command core, Flutter UI, and MCP tool surface). Shipped as the desktop app under apps/, not as pub.dev libraries. In active development.
apps/flutter_scene_editor_app The standalone Flutter Scene Editor desktop app.
examples/flutter_app Runnable example app with 43 feature examples.

The remaining examples/ folders are dev-only test harnesses (the web-backend smoke test, deterministic smoke renders, and a CPU stress bench).

To run the example app from a fresh clone:

flutter pub get                                             # resolves the workspace

cd examples/flutter_app
flutter create . --platforms=macos,ios,android,linux,windows,web  # generate gitignored platform stubs
flutter run --enable-flutter-gpu                              # native; add `-d <device>` if needed
flutter run -d chrome                                         # web

Pass --dart-define=FLUTTER_SCENE_PROFILE=true to print 120-frame render graph, culling, encoding, instance packing, binding, byte, draw, and instance summaries. Multiple active RenderViews share the counters.