webeditor3d/roadmap.md

# roadmap.md

Use this as a direction/scope doc, not as proof that every earlier slice is still unimplemented.

Do not read this file end-to-end by default.
Read only the relevant milestone/slice headings after checking the current code.

Completed or partially completed historical slices may still be documented here for context.

## Overview

This roadmap is organized as vertical slices.
Each slice must deliver an end-to-end usable capability across:

- document model
- commands
- viewport/editor behavior
- UI
- persistence
- runner behavior where relevant
- tests
- manual QA notes

We optimize for:

- proving the product loop early
- preserving architecture
- shipping coherent slices
- avoiding speculative complexity

If a roadmap item is too large for one implementation pass, split it into smaller end-to-end sub-slices rather than landing half-systems.

---

## Product north star

A browser-based engine/editor with a built-in runner, enabling users to quickly create and share lightweight interactive 3D spaces with:

- intuitive whiteboxing / level blocking
- material/texture workflows
- imported GLB/GLTF assets
- spatial audio
- typed entities and simple interactions
- optional navigation modes

---

## Early project decisions

These are locked for the early milestones:

- world space is **Y-up**
- early repo shape is a single Vite app with domain folders under `src/`
- canonical document state lives outside the React tree
- the document is versioned from day one
- early slices began with axis-aligned box brushes and fixed face IDs
- placed imported models are `modelInstances`, not `entities`
- local draft persistence plus explicit JSON import/export is acceptable early
- once scenes depend on external binary assets, portable save/load must move to a project package containing canonical scene JSON plus bundled assets
- broad roadmap items may be split into smaller implementation chats as long as each chat still lands a coherent vertical slice

---

## Milestone structure

### Milestone 0
Foundation and repo discipline

### Milestone 1
First real room: box-based layout + materials + play mode

### Milestone 2
Entities and runtime interaction

### Milestone 3
Imported models, lighting, animation, and audio

### Milestone 4
Whiteboxing and direct geometry editing

### Milestone 5
Project portability, deployment, and quality improvements

### Milestone 6
Power-user systems and ecosystem growth

---

## Milestone 0 — Foundation

### Goals

Create the minimum project skeleton that supports fast vertical slicing without overbuilding.

### Deliverables

- repo structure established under a single app
- TypeScript + Vite + React app booting
- three.js viewport shell visible
- thin external editor store/document skeleton
- versioned scene document and migration entry point
- command history scaffold
- local draft save/load scaffold plus JSON import/export scaffold
- test runner setup
- Playwright basic smoke setup
- docs established and aligned

### Acceptance criteria

- app boots locally
- empty viewport renders
- empty document loads
- command stack can push a dummy command and undo/redo
- one smoke test passes
- docs exist and are consistent

### Out of scope

- real geometry tools
- real runtime
- real asset import
- full inspector

---

## Milestone 1 — First playable slice

### Vision

The user can create a box-based room, assign materials to faces, save the scene, reload it, and run around it.

This is the first proof that the product is real.

### Slice 1.1 — Box brush authoring

#### Deliverables

- axis-aligned box brush schema
- stable box face IDs
- create box brush command
- select box brush
- move/resize box brush
- grid snapping
- viewport render of box brushes
- basic outliner presence
- save/load support

#### Acceptance criteria

- user can create at least one box brush
- box persists through save/load
- selection works reliably
- transform edits are undoable
- no arbitrary brush rotation is introduced yet

---

### Slice 1.2 — Face materials and UV basics

#### Deliverables

- material registry
- small local starter material library
- face selection
- apply material to a single face
- basic UV controls:
  - offset
  - scale
  - rotate 90-degree steps
  - flip U/V
  - fit to face by rewriting explicit UV values
- inspector integration
- material browser panel

#### Acceptance criteria

- user can texture a simple room quickly
- per-face assignments persist through save/load
- UV edits persist
- editor rendering reflects those changes immediately

---

### Slice 1.3 — Runner v1

#### Deliverables

- runtime build from document
- first-person navigation
- third-person mode
- mode switch
- minimal typed `PlayerStart` support
- basic collision against axis-aligned box brush geometry
- play-from-editor action

#### Acceptance criteria

- user can press Run and navigate the scene
- first-person works with keyboard/mouse
- third-person works
- leaving run mode returns to editor state safely
- `PlayerStart` does not need to be rebuilt later as a separate foundation

---

### Slice 1.4 — End-to-end polish for “first room”

#### Deliverables

- status/errors panel
- document validation basics
- improved snapping feedback
- basic toolbar
- coherent save/load UX for the current persistence path
- basic scene settings if needed

#### Acceptance criteria

- creating a tiny room feels coherent
- failures are visible
- no obvious corruption issues
- smoke tests cover the loop

---

### Slice 1.5 — World lighting and environment basics

#### Deliverables

- canonical world settings for environment/background
- authored global ambient light settings
- authored global directional light / sun settings
- optional fog settings if they are easy and coherent
- editor and runner support for those world settings
- inspector or scene-settings UI for editing them

#### Acceptance criteria

- the first-room workflow no longer depends on hidden hardcoded lighting assumptions
- world environment settings persist through save/load
- editor and runner both reflect the authored world settings coherently

---

## Milestone 2 — Entity-driven interactions

### Vision

The user can place non-whitebox entities and author simple interactive scenes without scripting.

### Slice 2.1 — Entity system foundation

#### Deliverables

- typed entity base
- entity registry
- entity icons/helpers in viewport
- entity placement workflow
- entity inspector
- save/load support
- extension of the existing `PlayerStart` path rather than a parallel rewrite

#### Initial entity types

- PlayerStart
- SoundEmitter
- TriggerVolume
- TeleportTarget
- Interactable

#### Acceptance criteria

- entities can be placed, selected, edited, saved, and loaded
- runtime build can consume them
- model instances remain separate from entities

---

### Slice 2.2 — Trigger -> Action -> Target foundation

#### Deliverables

- typed trigger/action/target schema
- action dispatch pipeline
- editor UI for linking
- runtime evaluation

#### Initial trigger sources

- TriggerVolume enter
- TriggerVolume exit

#### Initial actions

- teleport player
- toggle visibility

Sound and animation actions are intentionally deferred until their runtime systems exist.

#### Acceptance criteria

- user can build a basic non-scripted interaction without code
- links survive save/load
- runtime behavior is deterministic

---

### Slice 2.3 — Click interactions and runner prompts

#### Deliverables

- click target support
- prompt UI
- action-on-click behavior
- interaction distance settings
- compatibility with the existing trigger/action/target system

#### Acceptance criteria

- user can click an entity to trigger something
- prompts are understandable
- keyboard/controller fallback rules are documented

---

## Milestone 3 — Assets, lighting, animation, and audio

### Vision

The tool becomes more than brush-only by supporting imported assets, authored lighting, richer scene atmosphere, animation, and native spatial audio.

### Slice 3.1 — GLB/GLTF import

#### Deliverables

- import workflow
- asset registry
- persistent project storage for imported binary data
- place imported model instance
- transform model instance
- save/load support for asset refs and model instances

#### Acceptance criteria

- user can import a GLB/GLTF and place it in scene
- asset metadata is preserved
- imported assets survive reload
- runtime renders imported model instances correctly

---

### Slice 3.1B — Imported model collider authoring

#### Deliverables

- canonical collision settings on model instances
- explicit collision mode selector for:
  - none
  - terrain
  - static
  - dynamic
  - simple
- collision visibility/debug toggle
- generated collider data derived from imported model geometry plus authored settings
- Rapier-backed collision/query integration for imported-model colliders and broad-phase/narrow-phase handling
- runtime build support for imported-model colliders in the runner collision path
- editor/runner debug visualization of generated collision where enabled

#### Acceptance criteria

- author can choose a collision mode per model instance in the inspector
- imported models can participate in runner collision without hand-authored invisible geometry
- collider modes mean exactly:
  - none = no collider
  - terrain = heightfield collider, static only
  - static = triangle mesh collider, fixed only
  - static-simple = voxel-boxified fixed compound collider for static environment use
  - dynamic = convex decomposition into compound collider, dynamic/kinematic capable
  - simple = one cheap primitive or one convex hull
- generated collision survives save/load through canonical settings and deterministic rebuild behavior
- broad-phase and narrow-phase pruning are handled by Rapier instead of app-specific custom collision code
- the slice does not require a full gameplay-physics sandbox
- unsupported or partial modes fail clearly instead of silently pretending to work

---

### Slice 3.2 — Local lights and skyboxes

#### Deliverables

- typed local light entities such as point light and spot light
- viewport helpers and inspector editing for those lights
- runtime support for local authored lights
- skybox / environment-background asset support using persistent project storage
- scene/world settings integration for choosing the active environment background

#### Acceptance criteria

- author can place local lights in a scene and see them in editor and runner
- environment background / skybox survives reload
- lighting/environment settings are explicit, not hidden hardcoded runtime state

---

### Slice 3.3 — Animation playback

#### Deliverables

- detect imported animations
- animation config on model instances or equivalent explicit target config
- play/stop animation actions
- animation trigger binding

#### Acceptance criteria

- imported animated asset can be triggered in runner
- editor exposes basic animation controls/settings

---

### Slice 3.4 — Spatial audio

#### Deliverables

- audio asset import/reference
- positional emitter runtime support
- distance settings
- loop/one-shot settings
- triggerable playback
- browser audio unlock UX
- play/stop sound actions integrated into the trigger system

#### Acceptance criteria

- sound emits from authored world positions
- playback is spatial in runner
- author can test it locally with clear UX

---

## Milestone 4 — Whiteboxing and direct geometry editing

### Vision

The editor stops feeling like a grid-bound brush tool and starts feeling like an intuitive web-native whiteboxing environment for interactive spaces.

These roadmap items may be delivered as multiple smaller implementation chats if that produces cleaner vertical slices.

### Slice 4.1 — Whitebox terminology and float-transform foundation

#### Deliverables

- shift the product/editor language from brush-first toward whitebox-solid authoring
- allow floating point position/rotation/scale for box-based whitebox solids
- free object rotation for whitebox boxes
- grid snapping becomes optional/configurable rather than mandatory
- save/load/runtime support for the new transform freedom

#### Acceptance criteria

- user can place and rotate whitebox boxes freely without being forced back onto a coarse grid
- transformed whitebox solids persist through save/load
- editor and runner both reflect the same transform state

---

### Slice 4.2 — Whitebox component selection modes

#### Deliverables

- explicit object / face / edge / vertex selection modes
- clear viewport feedback for the current component mode
- default object-first selection behavior for transform workflows
- coherent behavior in perspective and orthographic panes

#### Acceptance criteria

- the user can intentionally choose what level of a whitebox solid they are editing
- component selection feels reliable and visible in single-view and quad-view layouts

---

### Slice 4.3 — Unified whitebox transforms

#### Deliverables

- unified transform workflow for object and component editing
- `G / R / S` style modal transforms with `X / Y / Z` constraints
- gizmo + keyboard parity where practical
- move/rotate/scale support for whitebox objects
- move/rotate/scale support for faces and edges where valid
- move support for vertices in the first pass

#### Acceptance criteria

- whitebox editing feels direct rather than inspector-driven
- transform previews, cancel, commit, and undo/redo are coherent
- object and component transforms share one understandable interaction model

---

### Slice 4.4 — Whitebox mesh and collider derivation

#### Deliverables

- derived triangulation for edited whitebox solids
- support for non-planar quad faces via deterministic internal triangulation
- support for non-convex whitebox solids where the editing model permits them
- collision generation through the solid collider path rather than axis-aligned-box assumptions
- validation/diagnostics for invalid or unsupported edited solids

#### Acceptance criteria

- edited whitebox solids render and collide correctly in editor and runner
- non-planar faces do not break the build path
- unsupported geometry states fail clearly instead of silently corrupting the scene

---

### Slice 4.5 — Topology tools after transform foundations

#### Deliverables

- evaluate and add the first topology-changing tool only after the whitebox interaction model is coherent
- possible candidates:
  - clip
  - extrude
  - bevel/chamfer-like edge operations
- preview visualization
- undo/redo support

#### Acceptance criteria

- the chosen topology tool feels consistent with the existing whitebox interaction model
- resulting solids remain save/load safe and runner-safe
- unsupported cases fail clearly instead of silently producing ad hoc geometry

---

## Milestone 5 — Material and ecosystem maturity

### Vision

The editor becomes attractive for repeated use, not just demos.

These items are also likely candidates for sub-slices if needed.

### Slice 5.1 — Better material library

#### Deliverables

- categories
- tags
- search
- favorites
- recent materials
- starter library expansion

#### Acceptance criteria

- material assignment feels fast
- modestly larger libraries remain usable

---

### Slice 5.2 — Prefabs and reusable assets

#### Deliverables

- prefab definition support
- prefab placement
- prefab instance updating rules
- prefab browser

#### Acceptance criteria

- user can create or import reusable building blocks
- instances remain manageable
- update behavior is explicit and documented

---

### Slice 5.3 — Portable project package import/export

#### Deliverables

- explicit project package export path
- explicit project package import path
- canonical `scene.json` plus bundled assets
- support for scenes with or without external assets
- clear diagnostics for missing or incompatible packaged assets

#### Acceptance criteria

- user can move a project between machines and continue editing it
- imported assets survive project package export/import
- project package structure is explicit and understandable

---

### Slice 5.4 — Runner package and embeddable runner

#### Deliverables

- runner package output path
- embeddable or standalone runner route/bundle
- packaged runtime scene plus required assets
- production asset optimization hooks where easy and justified

#### Acceptance criteria

- user can export a playable runner package from authored scenes
- runner package loads reliably in target browsers
- runner package is clearly separate from editable project save/load

---

## Milestone 6 — Power-user growth

### Vision

Expand capability without compromising the core.

### Directional note

The time/living-world direction should stay closer to Majora's Mask than to a fully simulated always-loaded sandbox:

- one global project clock
- deterministic resolution from time + flags + scene context
- unloaded scenes reconstructed on demand instead of fully simulated all the time
- authored day/night came before the current scheduler/notebook foundation; next work should build on that shared time/control path rather than add one-off per-system timers

### Candidate slices

- dialogue robustness, variants, and outcomes on top of the current dialogue foundation
- scheduler conditions / overrides / scene-global rows on top of the current scheduler/notebook
- event/reaction sources that reuse the same control surface as scheduler and interaction links
- deterministic schedules/routines driven by time windows and flags
- cross-scene resolution for actors and dialogue state from time + flags + scene context
- cycle/reset mechanics if loop gameplay becomes a core pillar
- curve/track authoring for interpolatable control-surface channels
- advanced UV tools
- nav waypoints authoring
- later route-finding / A* only after authored-path and scene-connection semantics are clear
- ambient zones and audio buses
- lightweight scripting
- plugin/tool API
- collaborative editing groundwork
- remote asset libraries
- scene templates/starter kits

These are only pursued after the earlier core loop feels solid.

---

## Priority order inside the roadmap

When schedule pressure forces tradeoffs, prefer work that strengthens:

1. edit -> save/load -> run loop
2. whiteboxing ergonomics
3. material/texture speed
4. typed entities and interactions
5. imported asset support
6. runner stability and input/audio quality
7. ecosystem niceties

---

## Definition of “vertical slice complete”

A slice is complete only when:

- the feature is usable by a human end-to-end
- it is represented in the canonical document
- it supports save/load
- it is visible and usable in the UI
- it is test-covered appropriately
- obvious failure modes are handled
- it does not violate architecture boundaries

---

## Known risk areas

### High-risk technical areas

- brush geometry robustness
- per-face UV persistence
- picking accuracy
- collision generation from brush data
- collision generation from imported model data
- imported asset/material compatibility
- browser audio unlock behavior
- input edge cases across browsers
- project package / deployment correctness

### Process risks

- overbuilding infrastructure before a slice needs it
- letting the three.js scene graph become canonical state
- adding too many entity types too early
- under-testing geometry/serialization boundaries
- chasing polish before proving workflow

---

## Quality gates by milestone

### M0 quality gate
- app boots
- empty document loads
- docs stable
- tests run

### M1 quality gate
- user can make a textured room, save/reload it, walk it, and light it coherently

### M2 quality gate
- user can place interactive entities without code

### M3 quality gate
- user can import a GLB and combine it with brush scenes, local lights, environment background, animation, and audio

### M4 quality gate
- editor starts feeling genuinely ergonomic

### M5 quality gate
- project can be moved between machines and exported as a playable runner package

---

## Deferred items list

These are explicitly deferred unless reprioritized:

- full multiplayer collaboration
- advanced physics gameplay
- procedural generation systems
- full scripting VM
- lightmapping pipeline
- GLB/GLTF scene export as a priority feature
- node-based materials
- native desktop packaging as a priority
- R3F integration as a core dependency
- custom rendering backend beyond three.js

---

## Expected first public “wow” moment

The first moment that will make the product feel truly exciting is likely this:

1. create a brutalist room with box brushes
2. texture faces quickly from the library
3. import an animated GLB sculpture or door
4. place a positional sound source
5. hit Run
6. walk through it in-browser with sound and interaction

The roadmap should always move toward making that loop feel better.