Multiple layers of SD mixed with Blender animation and masking

I’ve been experimenting with other ways to use SD. For my latest music video I made some simple regular animation using Blender then ran it through Stable Diffusion vid2vid with varying strength settings to liven it up. The crazy thing is that made it more human…

In this clip there’s several layers of AI. The city background was one long image I generated with SD. I then used it in Blender as a background behind my walking character. Then I ran that animation through SD, which made it blend better and gave it that signature shakiness… The red dots were added afterwards on top of it (I spilled Tabasco on a transparent sheet on a green screen 😁).

I’ll share other parts of it here, I’m pretty happy with some wild experiments where I generated background behind masked animations than were themselves separately ran through SD vid2vid in several layers 😅

If you like this, know that it’s coming out on May 13, right here: https://youtu.be/EV7pifgfAWc 😊