Overview:

A nice, clear sky refreshes one's mind often (at least for me). Why not design your own right now! This project uses MediaPipe pose detection to create arbitrary "streams", while its UI enables you to control which body parts you use to create the streams. It also gradually changes the time of the day as time passes by, using prompt traveling. 

File Overview:

toe file: The TouchDesigner file (pretty straightforward, isn't it)

(Note: please avoid renaming each component, as it may cause errors!)

prompt_list.txt: txt file containing all of the prompts, used for prompt traveling

How To Use:

0. Make sure that MediaPipe works (as well as your camera). Also, replace the StreamDiffusionTD dummy file if you haven't.

0-1. Get the travel table components from DotSimulate's discord server, and set it up like the following: 

1. Start stream on StreamDiffusionTD

2. In the travel_table, make sure that the clamp schedule is "Loop", and reset is off. Click on AUTO Set Prompt Blocks.

3. Press Right Click on container1 and press on View. Then, enjoy!

Credits:

Mediapipe:  https://github.com/torinmb/mediapipe-touchdesigner

DotSimulate: for not only creating StreamDiffusionTD but also creating prompt traveling

DayDream: server host + program organizer!!!

Aaaand now the journey of how I got here:

How I chose the topic:

I was actually thinking of a totally different project involving pose tracking and avatar pose + background generation. But that unfortunately didn't work out well, so while I was thinking of a different project, I noticed that the MediaPipe output for pose tracking resembled jet streams in the sky. As a person who loves staring at a clear sky for no reason, I thought, "Why not create a jet stream generator?"

The first version only involved a simple path between the body tracking and a single prompt. However, to make things more "realistic", I decided to use prompt traveling to gradually switch the time of the day.

Then, the next version involved creating methods to use "fewer joints" to create fewer streams. Although I love jet streams, I don't want too much sometimes. I was kind of stuck on how to do this, but ultimately, with the help of various online resources (shoutout to Amazing Robot), I was able to create a simple UI, dividing my body into 4 parts: head, arms, hands, and legs. 

So what next?:

It would be really cool if we could involve seasonal features (ex. flowers in Spring, snow in Winter), as well as a clear distinction of the time of day when the sun is up. 

Perhaps involving birds would be another next step.

If you have any other questions/suggestions, feel free to leave that in the comments!