Work in progress - depth feed from Quest for VACE + reference images from Nano Banana, trying to 'look around' inside the video. Might need to pivot to something the model can handle better!

Goals

Open-world narrative AI video exploration. User with camera walks through some space, 

Stretch goals

• Storyteller LLM uses camera for visual context & Scope to generate ‘reimagined’ video stream.
• 6dof tracking - prompt & reference image change with physical location
• Additional hardware support - VR headset, Rayban Meta (POV glasses), Twitch (audience suggests & votes on story events), Android/iOS
• Basic ‘quest’ (fetch quest to encourage returning to previous locations)

Non-goals

• 3D video projection (reimagined video stream will be ‘flat’)
• Robust game mechanics (primarily exploration, possibly ‘fetch quest’)

Deliverable

Something playable & interesting

Rough work-in-progress project plan attached! Simple proof-of-concept goals & ambitious stretch goals