AI music video
Related item:
Phenomenology in the Design of Melody Generation Algorithms
Phew! Processing her 4 seconds at a time and taking minutes for each 4 second clip, I’m happy to share my first work with the world. Music video produced by Generative-AI! Only the video component was produced with AI technology, the music part was done long ago (March 2021) with old-fashioned algorithmic programming methods and mixed in a DAW.
At the time of this writing, RunwayML can only generate 4-second clips at 2K resolution, but I suspect these limitations will soon be lifted, especially since we just raised a $141 million investment. We have a lot of music tracks that we haven’t produced videos for yet, so we plan to use them as our first set of AI-generated videos. We can only imagine a near future where 4K clips are processed faster. I think $141 million should be able to do it 🙂
As a concept for this video, I started with the general idea of algorithmically composed piano music and images of various automatons (robots, machines, cyborgs, steampunks) associated with it, and then expanded into this piano world. I started exploring the spaces of performances, factories, cities and gardens. Different times of the day, different levels of sociality and isolation, as well as color palettes ranging from solid colors to vibrant and colorful palettes.
There are also some personal anecdotes here. For example, when I was in graduate school, I visited a piano factory to obtain a customized length of piano wire for an architectural model project. I’ve also been stuck with a bunch of old pianos to do soundtrack work, as described in “Tools, Wires, and”. Sound: The story of an old piano soundtrack loaded onto a truck.
This AI video creation method requires patience. I mostly played chess apps on my phone between video renders. Surprisingly, most of the clips were very useful, but I had to decide not to use some because they lacked motion or didn’t quite fit into the overall sequence. I had to.
I chose to take a Godard-like approach and fully embrace the formal peculiarities of technology. This was the same as he often takes his A/B rolls on Steenbeck’s editing table literally, always allowing him only two audio his tracks. Dialogue, or dialogue and effects, or effects and music, as the table could only accommodate two reels of tape. Also, of course, he literally interpreted his splicers on tape for straight (no fade) cuts in the edit. Likewise, I took his 4 second clip as the core building block of the video and refrained from any editing during that period.
I also love the imperfections in video rendering. At $141 million, I know all these flaws will start hitting the connecting blocks of software development, so I feel like making as much early generative AI art as possible in this new day. It is applied to eliminate visual errors which are quite attractive.
Overall, I am very happy with the results! I am also happy to see this new technology entering the digital creative scene at a time when I am relatively young and able to take advantage of it. I hope I have enough years left to evolve my art with this evolving technology (knocking AI generated tree images).
