webeditor3d/.kiro/specs/animation-playback/design.md

Design Document: Animation Playback

Overview

This slice adds animation playback for imported GLB/GLTF model assets. The feature is deliberately narrow: detect available clips from existing metadata, add optional per-instance configuration fields, wire playAnimation and stopAnimation into the existing Trigger → Action → Target interaction system, drive playback in the runtime via three.js AnimationMixer, and persist everything through a schema version bump with migration.

No timeline editor, no blend trees, no cross-fade authoring. The goal is a minimal, explicit, correct first pass that fits cleanly into the existing architecture.

Architecture

The change touches five layers, each with a clear responsibility:

Document layer        ModelInstance gains animationClipName + animationAutoplay
                      InteractionAction gains PlayAnimationAction + StopAnimationAction

Interaction layer     interaction-links.ts gains two new factory functions and updated clone/equality

Runtime build layer   RuntimeModelInstance gains animationClipName + animationAutoplay
                      buildRuntimeSceneFromDocument propagates the new fields

Runtime host layer    RuntimeHost gains AnimationMixer map, per-frame update, play/stop dispatch

Serialization layer   migrateSceneDocument bumps version to 12, reads new fields, migrates v11

The editor viewport does not play animations — only the runner does. The editor shows the model in its bind pose, consistent with the existing approach of keeping the viewport simple.

Components and Interfaces

1. ModelInstance (src/assets/model-instances.ts)

Add two optional fields:

export interface ModelInstance {
  id: string;
  kind: "modelInstance";
  assetId: string;
  name?: string;
  position: Vec3;
  rotationDegrees: Vec3;
  scale: Vec3;
  animationClipName?: string;   // NEW: name of the default clip to play
  animationAutoplay?: boolean;  // NEW: whether to start playing on scene load
}

Update createModelInstance, cloneModelInstance, and areModelInstancesEqual to handle the new fields. The factory accepts them as optional overrides; cloneModelInstance copies them; areModelInstancesEqual includes them in the comparison.

2. InteractionAction (src/interactions/interaction-links.ts)

Add two new action types to the discriminated union:

export interface PlayAnimationAction {
  type: "playAnimation";
  targetModelInstanceId: string;
  clipName: string;
}

export interface StopAnimationAction {
  type: "stopAnimation";
  targetModelInstanceId: string;
}

export type InteractionAction =
  | TeleportPlayerAction
  | ToggleVisibilityAction
  | PlayAnimationAction
  | StopAnimationAction;

Add factory functions:

export function createPlayAnimationInteractionLink(options: {
  id?: string;
  sourceEntityId: string;
  trigger?: InteractionTriggerKind;
  targetModelInstanceId: string;
  clipName: string;
}): InteractionLink

export function createStopAnimationInteractionLink(options: {
  id?: string;
  sourceEntityId: string;
  trigger?: InteractionTriggerKind;
  targetModelInstanceId: string;
}): InteractionLink

Update cloneAction, areInteractionLinksEqual, and cloneInteractionLink to handle the new action types.

3. RuntimeModelInstance (src/runtime-three/runtime-scene-build.ts)

Add the animation fields to the runtime data type:

export interface RuntimeModelInstance {
  instanceId: string;
  assetId: string;
  name?: string;
  position: Vec3;
  rotationDegrees: Vec3;
  scale: Vec3;
  animationClipName?: string;   // NEW
  animationAutoplay?: boolean;  // NEW
}

Update buildRuntimeModelInstance to propagate the new fields from the document model instance.

4. RuntimeInteractionDispatcher (src/runtime-three/runtime-interaction-system.ts)

Extend the dispatcher interface:

export interface RuntimeInteractionDispatcher {
  teleportPlayer(target: RuntimeTeleportTarget, link: InteractionLink): void;
  toggleBrushVisibility(brushId: string, visible: boolean | undefined, link: InteractionLink): void;
  playAnimation(instanceId: string, clipName: string, link: InteractionLink): void;  // NEW
  stopAnimation(instanceId: string, link: InteractionLink): void;                    // NEW
}

Update RuntimeInteractionSystem.dispatchLinks to handle the two new action types by calling the dispatcher methods.

5. RuntimeHost (src/runtime-three/runtime-host.ts)

This is where the three.js AnimationMixer lifecycle lives.

New private state:

private readonly animationMixers = new Map<string, AnimationMixer>();
// instanceId -> { mixer, clips[] } for looking up clips by name at dispatch time
private readonly instanceAnimationClips = new Map<string, AnimationClip[]>();

rebuildModelInstances changes:

After creating the render group for each model instance, check whether the loaded GLTF asset has animations. If it does, create an AnimationMixer targeting the render group and store it. If animationAutoplay is true and animationClipName is set, find the clip by name and call mixer.clipAction(clip).play().

// Pseudocode for the new section inside rebuildModelInstances:
const gltf = this.loadedGltfAssets[modelInstance.assetId]; // see note below
if (gltf && gltf.animations.length > 0) {
  const mixer = new AnimationMixer(renderGroup);
  this.animationMixers.set(modelInstance.instanceId, mixer);
  this.instanceAnimationClips.set(modelInstance.instanceId, gltf.animations);

  if (modelInstance.animationAutoplay && modelInstance.animationClipName) {
    const clip = AnimationClip.findByName(gltf.animations, modelInstance.animationClipName);
    if (clip) {
      mixer.clipAction(clip).play();
    }
  }
}

Note on animation clip access: The LoadedModelAsset currently stores only the template: Group (the cloned scene). The raw AnimationClip[] array from the GLTF parse result is not currently retained. We need to store the clips alongside the template. The LoadedModelAsset interface gains an animations: AnimationClip[] field:

export interface LoadedModelAsset {
  assetId: string;
  storageKey: string;
  metadata: ModelAssetMetadata;
  template: Group;
  animations: AnimationClip[];  // NEW: raw clips from the GLTF parse result
}

createLoadedModelAsset and extractModelAssetMetadata callers in gltf-model-import.ts are updated to populate this field from gltf.animations.

clearModelInstances changes:

Stop and dispose all mixers before clearing the render objects:

for (const mixer of this.animationMixers.values()) {
  mixer.stopAllAction();
}
this.animationMixers.clear();
this.instanceAnimationClips.clear();

render loop changes:

After this.activeController?.update(dt), tick all active mixers:

for (const mixer of this.animationMixers.values()) {
  mixer.update(dt);
}

New private methods:

private applyPlayAnimationAction(instanceId: string, clipName: string): void {
  const mixer = this.animationMixers.get(instanceId);
  const clips = this.instanceAnimationClips.get(instanceId);
  if (!mixer || !clips) {
    console.warn(`playAnimation: no mixer for instance ${instanceId}`);
    return;
  }
  const clip = AnimationClip.findByName(clips, clipName);
  if (!clip) {
    console.warn(`playAnimation: clip "${clipName}" not found on instance ${instanceId}`);
    return;
  }
  mixer.stopAllAction();
  mixer.clipAction(clip).play();
}

private applyStopAnimationAction(instanceId: string): void {
  const mixer = this.animationMixers.get(instanceId);
  if (!mixer) {
    console.warn(`stopAnimation: no mixer for instance ${instanceId}`);
    return;
  }
  mixer.stopAllAction();
}

createInteractionDispatcher changes:

Add the two new methods to the returned dispatcher object.

6. migrateSceneDocument (src/document/migrate-scene-document.ts)

• Bump SCENE_DOCUMENT_VERSION to 12 in scene-document.ts.
• Add a ANIMATION_PLAYBACK_SCENE_DOCUMENT_VERSION = 12 constant.
• In readModelInstance, read animationClipName and animationAutoplay as optional fields:
  • animationClipName: expectOptionalString then normalize (trim, empty → undefined)
  • animationAutoplay: optional boolean, default undefined
• In readInteractionAction, add cases for "playAnimation" and "stopAnimation".
• Add a migration branch for source.version === LOCAL_LIGHTS_AND_SKYBOX_SCENE_DOCUMENT_VERSION (v11) that reads the existing fields and produces a v12 document with animationClipName: undefined and animationAutoplay: undefined on all model instances, and no animation interaction links.

7. Inspector UI (src/app/App.tsx)

In the model instance inspector section (where position/rotation/scale are shown), add an animation sub-section that is conditionally rendered when selectedModelAssetRecord?.metadata.animationNames.length > 0:

{selectedModelAssetRecord && selectedModelAssetRecord.metadata.animationNames.length > 0 && (
  <div className="inspector-section">
    <label>Animation Clip</label>
    <select
      value={selectedModelInstance.animationClipName ?? ""}
      onChange={(e) => {
        const clipName = e.target.value || undefined;
        store.dispatch(createUpsertModelInstanceCommand({
          modelInstance: { ...selectedModelInstance, animationClipName: clipName }
        }));
      }}
    >
      <option value="">— none —</option>
      {selectedModelAssetRecord.metadata.animationNames.map((name) => (
        <option key={name} value={name}>{name}</option>
      ))}
    </select>
    <label>
      <input
        type="checkbox"
        checked={selectedModelInstance.animationAutoplay ?? false}
        onChange={(e) => {
          store.dispatch(createUpsertModelInstanceCommand({
            modelInstance: { ...selectedModelInstance, animationAutoplay: e.target.checked }
          }));
        }}
      />
      Autoplay on scene load
    </label>
  </div>
)}

8. Interaction Link UI (src/app/App.tsx)

In the interaction link authoring section, extend the action type <select> to include "playAnimation" and "stopAnimation". When either is selected, show a model instance picker (using modelInstanceDisplayList). For playAnimation, also show a clip name input (a <select> populated from the chosen model instance's asset's animationNames, or a text input if the asset is not loaded).

The existing getInteractionActionLabel helper is extended to return human-readable labels for the new action types.

Data Models

Document-level changes

// ModelInstance gains:
animationClipName?: string;
animationAutoplay?: boolean;

// InteractionAction gains two new members:
| { type: "playAnimation"; targetModelInstanceId: string; clipName: string }
| { type: "stopAnimation"; targetModelInstanceId: string }

Runtime-level changes

// RuntimeModelInstance gains:
animationClipName?: string;
animationAutoplay?: boolean;

// LoadedModelAsset gains:
animations: AnimationClip[];

Schema version

SCENE_DOCUMENT_VERSION advances from 11 to 12.

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Animation names are sorted and deduplicated

For any set of animation clip names extracted from a GLTF file, collectAnimationNames should return an array where every name appears exactly once and the array is in lexicographic order.

Validates: Requirements 1.3

Property 2: ModelInstance clone round-trip

For any ModelInstance (including those with animationClipName and animationAutoplay set), cloneModelInstance should produce a value that is deeply equal to the original as determined by areModelInstancesEqual.

Validates: Requirements 2.8, 2.9

Property 3: ModelInstance equality distinguishes animation fields

For any two ModelInstance values that differ only in animationClipName or animationAutoplay, areModelInstancesEqual should return false.

Validates: Requirements 2.9

Property 4: Autoplay starts the named clip on scene load

For any runtime scene containing a model instance with animationAutoplay: true and a valid animationClipName, after RuntimeHost.loadScene the AnimationMixer for that instance should have an active AnimationAction for the named clip.

Validates: Requirements 2.5, 5.4

Property 5: playAnimation action starts the named clip

For any model instance present in the runtime scene with a loaded asset containing at least one clip, dispatching a playAnimation action with a valid clipName should result in the mixer for that instance having an active action for that clip.

Validates: Requirements 3.2, 5.5

Property 6: stopAnimation action halts playback

For any model instance that has an active animation, dispatching a stopAnimation action should result in the mixer having no active actions.

Validates: Requirements 4.2

Property 7: Mixer count matches animated instance count

For any runtime scene with N model instances that have loaded assets with at least one animation clip, after loadScene the RuntimeHost should have exactly N active AnimationMixer instances.

Validates: Requirements 5.1

Property 8: Mixer cleanup on scene reload

For any sequence of loadScene calls, after each call the set of active mixers should correspond exactly to the model instances in the most recently loaded scene (no stale mixers from previous scenes).

Validates: Requirements 5.3

Property 9: playAnimation factory validates non-empty fields

For any call to createPlayAnimationInteractionLink where sourceEntityId, targetModelInstanceId, or clipName is an empty string, the function should throw an error.

Validates: Requirements 3.5

Property 10: stopAnimation factory validates non-empty fields

For any call to createStopAnimationInteractionLink where sourceEntityId or targetModelInstanceId is an empty string, the function should throw an error.

Validates: Requirements 4.5

Property 11: v11 → v12 migration preserves all existing data

For any valid v11 document, migrating to v12 should produce a document where all brushes, model instances (with animationClipName: undefined, animationAutoplay: undefined), entities, and interaction links are identical to the source, and the version is 12.

Validates: Requirements 8.2

Property 12: Serialization round-trip for v12 documents

For any valid v12 SceneDocument containing animation fields (including playAnimation and stopAnimation interaction links), parseSceneDocumentJson(serializeSceneDocument(doc)) should produce a document deeply equal to the original.

Validates: Requirements 8.3, 8.4, 8.6

Error Handling

• Missing clip name at dispatch: applyPlayAnimationAction logs a console.warn and returns without throwing. The scene continues running.
• Missing model instance at dispatch: Both applyPlayAnimationAction and applyStopAnimationAction log a console.warn and return.
• Stop with no active animation: mixer.stopAllAction() is idempotent — calling it when nothing is playing is safe.
• Migration of unknown action types: readInteractionAction already throws on unknown type values. The new cases are added to the switch; unknown types remain an error.
• Empty clipName in persisted document: migrateSceneDocument rejects a v12 document where a playAnimation action has an empty clipName string (after trimming).
• Asset not loaded at scene build time: If loadedModelAssets does not contain the asset for a model instance, no mixer is created. The model renders as a placeholder (existing behavior). No animation plays.

Testing Strategy

Unit tests (Vitest)

Unit tests cover specific examples, edge cases, and pure-function correctness:

• collectAnimationNames with zero clips, one clip, multiple clips with duplicates, clips with blank names
• createModelInstance with and without animation fields
• cloneModelInstance preserves animation fields
• areModelInstancesEqual returns false when animation fields differ
• createPlayAnimationInteractionLink throws on empty sourceEntityId, targetModelInstanceId, or clipName
• createStopAnimationInteractionLink throws on empty sourceEntityId or targetModelInstanceId
• migrateSceneDocument v11 → v12: model instances get undefined animation fields
• migrateSceneDocument v12 with playAnimation link: reads correctly
• migrateSceneDocument v12 with empty clipName: throws
• parseSceneDocumentJson(serializeSceneDocument(doc)) round-trip for a v12 document with animation fields

Property-based tests (fast-check, Vitest)

Property tests use fast-check to generate random inputs and verify universal properties. Each test runs a minimum of 100 iterations.

• Property 1: Generate random string arrays → collectAnimationNames output is sorted and deduplicated
  • Feature: animation-playback, Property 1: animation names are sorted and deduplicated
• Property 2: Generate random ModelInstance values with optional animation fields → cloneModelInstance produces an equal value
  • Feature: animation-playback, Property 2: ModelInstance clone round-trip
• Property 3: Generate pairs of ModelInstance values differing only in animation fields → areModelInstancesEqual returns false
  • Feature: animation-playback, Property 3: ModelInstance equality distinguishes animation fields
• Property 9: Generate empty-string variants of factory arguments → createPlayAnimationInteractionLink throws
  • Feature: animation-playback, Property 9: playAnimation factory validates non-empty fields
• Property 10: Generate empty-string variants of factory arguments → createStopAnimationInteractionLink throws
  • Feature: animation-playback, Property 10: stopAnimation factory validates non-empty fields
• Property 11: Generate random v11 documents → migration produces valid v12 documents with animation fields defaulted
  • Feature: animation-playback, Property 11: v11 to v12 migration preserves all existing data
• Property 12: Generate random v12 documents with animation fields → round-trip serialization produces equal documents
  • Feature: animation-playback, Property 12: serialization round-trip for v12 documents

Runtime properties (4–8) require a headless RuntimeHost instance (enableRendering: false). These are integration-level unit tests rather than pure property tests, but they verify universal behaviors:

• Property 4: For any scene with autoplay model instances, after loadScene the mixer has an active action
• Property 5: For any valid playAnimation dispatch, the mixer has an active action for the named clip
• Property 6: For any active animation, after stopAnimation dispatch the mixer has no active actions
• Property 7: Mixer count equals animated instance count after loadScene
• Property 8: After a second loadScene, no stale mixers from the first scene remain

Manual / e2e verification

Use a small animated GLB fixture (e.g., a box with a simple rotation animation) to verify:

1. Import the fixture — animationNames appears in the asset panel
2. Place the model instance — animation section appears in the inspector
3. Select a clip and enable autoplay — entering play mode shows the animation running
4. Wire a trigger volume → playAnimation → the model instance — entering the volume starts the animation
5. Wire a second trigger volume → stopAnimation → the model instance — entering the second volume stops it
6. Save and reload — animation settings survive the round-trip