# Design Document: Animation Playback ## Overview This slice adds animation playback for imported GLB/GLTF model assets. The feature is deliberately narrow: detect available clips from existing metadata, add optional per-instance configuration fields, wire `playAnimation` and `stopAnimation` into the existing Trigger → Action → Target interaction system, drive playback in the runtime via three.js `AnimationMixer`, and persist everything through a schema version bump with migration. No timeline editor, no blend trees, no cross-fade authoring. The goal is a minimal, explicit, correct first pass that fits cleanly into the existing architecture. ## Architecture The change touches five layers, each with a clear responsibility: ``` Document layer ModelInstance gains animationClipName + animationAutoplay InteractionAction gains PlayAnimationAction + StopAnimationAction Interaction layer interaction-links.ts gains two new factory functions and updated clone/equality Runtime build layer RuntimeModelInstance gains animationClipName + animationAutoplay buildRuntimeSceneFromDocument propagates the new fields Runtime host layer RuntimeHost gains AnimationMixer map, per-frame update, play/stop dispatch Serialization layer migrateSceneDocument bumps version to 12, reads new fields, migrates v11 ``` The editor viewport does **not** play animations — only the runner does. The editor shows the model in its bind pose, consistent with the existing approach of keeping the viewport simple. ## Components and Interfaces ### 1. `ModelInstance` (src/assets/model-instances.ts) Add two optional fields: ```typescript export interface ModelInstance { id: string; kind: "modelInstance"; assetId: string; name?: string; position: Vec3; rotationDegrees: Vec3; scale: Vec3; animationClipName?: string; // NEW: name of the default clip to play animationAutoplay?: boolean; // NEW: whether to start playing on scene load } ``` Update `createModelInstance`, `cloneModelInstance`, and `areModelInstancesEqual` to handle the new fields. The factory accepts them as optional overrides; `cloneModelInstance` copies them; `areModelInstancesEqual` includes them in the comparison. ### 2. `InteractionAction` (src/interactions/interaction-links.ts) Add two new action types to the discriminated union: ```typescript export interface PlayAnimationAction { type: "playAnimation"; targetModelInstanceId: string; clipName: string; } export interface StopAnimationAction { type: "stopAnimation"; targetModelInstanceId: string; } export type InteractionAction = | TeleportPlayerAction | ToggleVisibilityAction | PlayAnimationAction | StopAnimationAction; ``` Add factory functions: ```typescript export function createPlayAnimationInteractionLink(options: { id?: string; sourceEntityId: string; trigger?: InteractionTriggerKind; targetModelInstanceId: string; clipName: string; }): InteractionLink export function createStopAnimationInteractionLink(options: { id?: string; sourceEntityId: string; trigger?: InteractionTriggerKind; targetModelInstanceId: string; }): InteractionLink ``` Update `cloneAction`, `areInteractionLinksEqual`, and `cloneInteractionLink` to handle the new action types. ### 3. `RuntimeModelInstance` (src/runtime-three/runtime-scene-build.ts) Add the animation fields to the runtime data type: ```typescript export interface RuntimeModelInstance { instanceId: string; assetId: string; name?: string; position: Vec3; rotationDegrees: Vec3; scale: Vec3; animationClipName?: string; // NEW animationAutoplay?: boolean; // NEW } ``` Update `buildRuntimeModelInstance` to propagate the new fields from the document model instance. ### 4. `RuntimeInteractionDispatcher` (src/runtime-three/runtime-interaction-system.ts) Extend the dispatcher interface: ```typescript export interface RuntimeInteractionDispatcher { teleportPlayer(target: RuntimeTeleportTarget, link: InteractionLink): void; toggleBrushVisibility(brushId: string, visible: boolean | undefined, link: InteractionLink): void; playAnimation(instanceId: string, clipName: string, link: InteractionLink): void; // NEW stopAnimation(instanceId: string, link: InteractionLink): void; // NEW } ``` Update `RuntimeInteractionSystem.dispatchLinks` to handle the two new action types by calling the dispatcher methods. ### 5. `RuntimeHost` (src/runtime-three/runtime-host.ts) This is where the three.js `AnimationMixer` lifecycle lives. **New private state:** ```typescript private readonly animationMixers = new Map(); // instanceId -> { mixer, clips[] } for looking up clips by name at dispatch time private readonly instanceAnimationClips = new Map(); ``` **`rebuildModelInstances` changes:** After creating the render group for each model instance, check whether the loaded GLTF asset has animations. If it does, create an `AnimationMixer` targeting the render group and store it. If `animationAutoplay` is `true` and `animationClipName` is set, find the clip by name and call `mixer.clipAction(clip).play()`. ```typescript // Pseudocode for the new section inside rebuildModelInstances: const gltf = this.loadedGltfAssets[modelInstance.assetId]; // see note below if (gltf && gltf.animations.length > 0) { const mixer = new AnimationMixer(renderGroup); this.animationMixers.set(modelInstance.instanceId, mixer); this.instanceAnimationClips.set(modelInstance.instanceId, gltf.animations); if (modelInstance.animationAutoplay && modelInstance.animationClipName) { const clip = AnimationClip.findByName(gltf.animations, modelInstance.animationClipName); if (clip) { mixer.clipAction(clip).play(); } } } ``` **Note on animation clip access:** The `LoadedModelAsset` currently stores only the `template: Group` (the cloned scene). The raw `AnimationClip[]` array from the GLTF parse result is not currently retained. We need to store the clips alongside the template. The `LoadedModelAsset` interface gains an `animations: AnimationClip[]` field: ```typescript export interface LoadedModelAsset { assetId: string; storageKey: string; metadata: ModelAssetMetadata; template: Group; animations: AnimationClip[]; // NEW: raw clips from the GLTF parse result } ``` `createLoadedModelAsset` and `extractModelAssetMetadata` callers in `gltf-model-import.ts` are updated to populate this field from `gltf.animations`. **`clearModelInstances` changes:** Stop and dispose all mixers before clearing the render objects: ```typescript for (const mixer of this.animationMixers.values()) { mixer.stopAllAction(); } this.animationMixers.clear(); this.instanceAnimationClips.clear(); ``` **`render` loop changes:** After `this.activeController?.update(dt)`, tick all active mixers: ```typescript for (const mixer of this.animationMixers.values()) { mixer.update(dt); } ``` **New private methods:** ```typescript private applyPlayAnimationAction(instanceId: string, clipName: string): void { const mixer = this.animationMixers.get(instanceId); const clips = this.instanceAnimationClips.get(instanceId); if (!mixer || !clips) { console.warn(`playAnimation: no mixer for instance ${instanceId}`); return; } const clip = AnimationClip.findByName(clips, clipName); if (!clip) { console.warn(`playAnimation: clip "${clipName}" not found on instance ${instanceId}`); return; } mixer.stopAllAction(); mixer.clipAction(clip).play(); } private applyStopAnimationAction(instanceId: string): void { const mixer = this.animationMixers.get(instanceId); if (!mixer) { console.warn(`stopAnimation: no mixer for instance ${instanceId}`); return; } mixer.stopAllAction(); } ``` **`createInteractionDispatcher` changes:** Add the two new methods to the returned dispatcher object. ### 6. `migrateSceneDocument` (src/document/migrate-scene-document.ts) - Bump `SCENE_DOCUMENT_VERSION` to `12` in `scene-document.ts`. - Add a `ANIMATION_PLAYBACK_SCENE_DOCUMENT_VERSION = 12` constant. - In `readModelInstance`, read `animationClipName` and `animationAutoplay` as optional fields: - `animationClipName`: `expectOptionalString` then normalize (trim, empty → undefined) - `animationAutoplay`: optional boolean, default `undefined` - In `readInteractionAction`, add cases for `"playAnimation"` and `"stopAnimation"`. - Add a migration branch for `source.version === LOCAL_LIGHTS_AND_SKYBOX_SCENE_DOCUMENT_VERSION` (v11) that reads the existing fields and produces a v12 document with `animationClipName: undefined` and `animationAutoplay: undefined` on all model instances, and no animation interaction links. ### 7. Inspector UI (src/app/App.tsx) In the model instance inspector section (where position/rotation/scale are shown), add an animation sub-section that is conditionally rendered when `selectedModelAssetRecord?.metadata.animationNames.length > 0`: ```tsx {selectedModelAssetRecord && selectedModelAssetRecord.metadata.animationNames.length > 0 && (
)} ``` ### 8. Interaction Link UI (src/app/App.tsx) In the interaction link authoring section, extend the action type `` populated from the chosen model instance's asset's `animationNames`, or a text input if the asset is not loaded). The existing `getInteractionActionLabel` helper is extended to return human-readable labels for the new action types. ## Data Models ### Document-level changes ```typescript // ModelInstance gains: animationClipName?: string; animationAutoplay?: boolean; // InteractionAction gains two new members: | { type: "playAnimation"; targetModelInstanceId: string; clipName: string } | { type: "stopAnimation"; targetModelInstanceId: string } ``` ### Runtime-level changes ```typescript // RuntimeModelInstance gains: animationClipName?: string; animationAutoplay?: boolean; // LoadedModelAsset gains: animations: AnimationClip[]; ``` ### Schema version `SCENE_DOCUMENT_VERSION` advances from `11` to `12`. ## Correctness Properties *A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* ### Property 1: Animation names are sorted and deduplicated *For any* set of animation clip names extracted from a GLTF file, `collectAnimationNames` should return an array where every name appears exactly once and the array is in lexicographic order. **Validates: Requirements 1.3** ### Property 2: ModelInstance clone round-trip *For any* `ModelInstance` (including those with `animationClipName` and `animationAutoplay` set), `cloneModelInstance` should produce a value that is deeply equal to the original as determined by `areModelInstancesEqual`. **Validates: Requirements 2.8, 2.9** ### Property 3: ModelInstance equality distinguishes animation fields *For any* two `ModelInstance` values that differ only in `animationClipName` or `animationAutoplay`, `areModelInstancesEqual` should return `false`. **Validates: Requirements 2.9** ### Property 4: Autoplay starts the named clip on scene load *For any* runtime scene containing a model instance with `animationAutoplay: true` and a valid `animationClipName`, after `RuntimeHost.loadScene` the `AnimationMixer` for that instance should have an active `AnimationAction` for the named clip. **Validates: Requirements 2.5, 5.4** ### Property 5: playAnimation action starts the named clip *For any* model instance present in the runtime scene with a loaded asset containing at least one clip, dispatching a `playAnimation` action with a valid `clipName` should result in the mixer for that instance having an active action for that clip. **Validates: Requirements 3.2, 5.5** ### Property 6: stopAnimation action halts playback *For any* model instance that has an active animation, dispatching a `stopAnimation` action should result in the mixer having no active actions. **Validates: Requirements 4.2** ### Property 7: Mixer count matches animated instance count *For any* runtime scene with N model instances that have loaded assets with at least one animation clip, after `loadScene` the `RuntimeHost` should have exactly N active `AnimationMixer` instances. **Validates: Requirements 5.1** ### Property 8: Mixer cleanup on scene reload *For any* sequence of `loadScene` calls, after each call the set of active mixers should correspond exactly to the model instances in the most recently loaded scene (no stale mixers from previous scenes). **Validates: Requirements 5.3** ### Property 9: playAnimation factory validates non-empty fields *For any* call to `createPlayAnimationInteractionLink` where `sourceEntityId`, `targetModelInstanceId`, or `clipName` is an empty string, the function should throw an error. **Validates: Requirements 3.5** ### Property 10: stopAnimation factory validates non-empty fields *For any* call to `createStopAnimationInteractionLink` where `sourceEntityId` or `targetModelInstanceId` is an empty string, the function should throw an error. **Validates: Requirements 4.5** ### Property 11: v11 → v12 migration preserves all existing data *For any* valid v11 document, migrating to v12 should produce a document where all brushes, model instances (with `animationClipName: undefined`, `animationAutoplay: undefined`), entities, and interaction links are identical to the source, and the version is 12. **Validates: Requirements 8.2** ### Property 12: Serialization round-trip for v12 documents *For any* valid v12 `SceneDocument` containing animation fields (including `playAnimation` and `stopAnimation` interaction links), `parseSceneDocumentJson(serializeSceneDocument(doc))` should produce a document deeply equal to the original. **Validates: Requirements 8.3, 8.4, 8.6** ## Error Handling - **Missing clip name at dispatch**: `applyPlayAnimationAction` logs a `console.warn` and returns without throwing. The scene continues running. - **Missing model instance at dispatch**: Both `applyPlayAnimationAction` and `applyStopAnimationAction` log a `console.warn` and return. - **Stop with no active animation**: `mixer.stopAllAction()` is idempotent — calling it when nothing is playing is safe. - **Migration of unknown action types**: `readInteractionAction` already throws on unknown `type` values. The new cases are added to the switch; unknown types remain an error. - **Empty `clipName` in persisted document**: `migrateSceneDocument` rejects a v12 document where a `playAnimation` action has an empty `clipName` string (after trimming). - **Asset not loaded at scene build time**: If `loadedModelAssets` does not contain the asset for a model instance, no mixer is created. The model renders as a placeholder (existing behavior). No animation plays. ## Testing Strategy ### Unit tests (Vitest) Unit tests cover specific examples, edge cases, and pure-function correctness: - `collectAnimationNames` with zero clips, one clip, multiple clips with duplicates, clips with blank names - `createModelInstance` with and without animation fields - `cloneModelInstance` preserves animation fields - `areModelInstancesEqual` returns false when animation fields differ - `createPlayAnimationInteractionLink` throws on empty `sourceEntityId`, `targetModelInstanceId`, or `clipName` - `createStopAnimationInteractionLink` throws on empty `sourceEntityId` or `targetModelInstanceId` - `migrateSceneDocument` v11 → v12: model instances get `undefined` animation fields - `migrateSceneDocument` v12 with `playAnimation` link: reads correctly - `migrateSceneDocument` v12 with empty `clipName`: throws - `parseSceneDocumentJson(serializeSceneDocument(doc))` round-trip for a v12 document with animation fields ### Property-based tests (fast-check, Vitest) Property tests use `fast-check` to generate random inputs and verify universal properties. Each test runs a minimum of 100 iterations. - **Property 1**: Generate random string arrays → `collectAnimationNames` output is sorted and deduplicated - `Feature: animation-playback, Property 1: animation names are sorted and deduplicated` - **Property 2**: Generate random `ModelInstance` values with optional animation fields → `cloneModelInstance` produces an equal value - `Feature: animation-playback, Property 2: ModelInstance clone round-trip` - **Property 3**: Generate pairs of `ModelInstance` values differing only in animation fields → `areModelInstancesEqual` returns false - `Feature: animation-playback, Property 3: ModelInstance equality distinguishes animation fields` - **Property 9**: Generate empty-string variants of factory arguments → `createPlayAnimationInteractionLink` throws - `Feature: animation-playback, Property 9: playAnimation factory validates non-empty fields` - **Property 10**: Generate empty-string variants of factory arguments → `createStopAnimationInteractionLink` throws - `Feature: animation-playback, Property 10: stopAnimation factory validates non-empty fields` - **Property 11**: Generate random v11 documents → migration produces valid v12 documents with animation fields defaulted - `Feature: animation-playback, Property 11: v11 to v12 migration preserves all existing data` - **Property 12**: Generate random v12 documents with animation fields → round-trip serialization produces equal documents - `Feature: animation-playback, Property 12: serialization round-trip for v12 documents` Runtime properties (4–8) require a headless `RuntimeHost` instance (`enableRendering: false`). These are integration-level unit tests rather than pure property tests, but they verify universal behaviors: - **Property 4**: For any scene with autoplay model instances, after `loadScene` the mixer has an active action - **Property 5**: For any valid `playAnimation` dispatch, the mixer has an active action for the named clip - **Property 6**: For any active animation, after `stopAnimation` dispatch the mixer has no active actions - **Property 7**: Mixer count equals animated instance count after `loadScene` - **Property 8**: After a second `loadScene`, no stale mixers from the first scene remain ### Manual / e2e verification Use a small animated GLB fixture (e.g., a box with a simple rotation animation) to verify: 1. Import the fixture — `animationNames` appears in the asset panel 2. Place the model instance — animation section appears in the inspector 3. Select a clip and enable autoplay — entering play mode shows the animation running 4. Wire a trigger volume → `playAnimation` → the model instance — entering the volume starts the animation 5. Wire a second trigger volume → `stopAnimation` → the model instance — entering the second volume stops it 6. Save and reload — animation settings survive the round-trip