And the Oscar for Best Robot Picture goes to...

Gen AI videos can be hilarious, horrifying, & hauntingly beautiful. But I don't think this story has a happy ending.

Lights! Camera! AI! Source: Midjourney.

I've written a fair amount about how when it comes to writing and image generation, AI does a damned fine impersonation of a human. (Even if they're not the kind of human you'd want to hang around with at cocktail parties.) But what's even more impressive, in an oh-my-god-we're-all-going-to-be-replaced-by-machines sort of way, is how well it does at creating music and video.

It now requires exactly zero talent to make an AI video. But it requires a substantial amount of talent (and a fair amount of tech expertise) to make a good AI video. [1]

ICYMI: Love, robots, and Electric Dreams

To demonstrate the former statement, I took an image I'd asked Midjourney to generate for an earlier blog post, uploaded it to a tool called Runway ML, and asked it to create a video by having the robots throw punches at each other.

Rock'em (But Don't) Sock'em Robots. I don't think they really wanted to fight.

Again, I really have no idea what I'm doing. In the hands of someone with actual talent and creative inspiration, AI-generated video can become something that makes you laugh, cry, wet yourself, gnaw your fingernails to the nub, or run screaming in terror. You know — just like "real" movies.

The Fantastic Mr. Faux

Here's a simple example. The creative imps at Curious Refuge made a series of classic movie trailers as directed by Wes Anderson. Here's his version of Star Wars (aka "The Galactic Menagerie").

To be honest, when I first saw this I really thought that Wes Anderson had directed a satirical trailer and recruited the usual gang to bring the joke home. But no. This was all done via Midjourney, animation tools, and some clever voice over.

Curious Refuge made these faux trailers to advertise their AI film making courses, which are apparently so popular that there's a waitlist for them. They also did Andersonized versions of Lord of the Rings ("The Whimsical Fellowship"), Avatar ("The Peculiar Pandora Expedition"), Alien ("Extraterrestrial Expedition"), and Jurassic Park ("The Delightful Dinosaur Diaries.") They're all essentially identical, much like Wes Anderson's real movies. [2]

As Darth VaderWilson would say, "Wow."

Here's Curious Refuge's more recent mashup of Barbenheimer. You can see how much better the image quality has gotten in just a few months, as well as how far they still have to go in terms of realistic speech and movement.

With Friends like these...

Did I mention that these AI movies can be both hilarious and terrifying? Some folks called Afraid2Sleep used Midjourney and a text-to-video tool from Pika Labs [3] to recreate the opening sequence to “Friends,” with a healthy amount of Ghostbusters thrown in.

What it lacks in resolution and realism (those characters look nothing like Ross, Rachel, Monica, etc) it makes up for in creepiness. (They also did similarly demented intros to The Office, Boy Meets World, and Saved by The Bell.)

But these mini-films can also be hauntingly beautiful. Like this one, titled Desert Odyssey, created by a visual artist named Jesse Koivukoski using Midjourney and Runway.

He does something similar using imagery inspired by Terminator and Blade Runner.

These videos would not be possible without the visionary work of directors like Ridley Scott, George Lucas, James Cameron, Denis Villeneuve, Guillermo del Toro, George Miller, and many many others. That's where the AI got its chops.

But now it is possible. And there's no turning back.

Directors cut

The first full-length digitally animated movie, Toy Story, debuted in 1995. It was inspired by the much simpler (but still brilliant) 1988 Pixar short called Tin Toy. [4]

We are in the Tin Toy era of AI film making. But it's not going to take another seven years before we enter the Toy Story era of AI, when entire films can be generated using prompts.

So far, at least, AI films still need a human sitting in the director's chair. But they don't need 200 or 300 union employees scouting locations, building sets, creating costumes, doing makeup, laying down cable, putting up lights, running cameras, serving food, shuttling people and stuff from one place to another, editing the final cut, or doing all the post production work.

And, oh yeah: Writers and Actors. You know, the folks on the picket lines trying to avoid being replaced by robots?

ICYMI: AI+Hollywood: We can imagine it for you wholesale

Stemming the tide of AI film making feels a little like trying to stop the actual tide. That these new tech tools will take over many if not all movie-making tasks — similar to how CGI effects are now part of nearly every movie with a real budget — seems inevitable. And that might be the saddest ending of all.

Should the Academy create a new category for Best Performance by a Robot in a Major Motion Picture? Share your thoughts in the comments below.

[1] Music is another area where Gen AI has made some impressive/terrifying gains. Look for more on that in a future post.

[2] Which are now indistinguishable from satires of Wes Anderson movies.

[3] Pika has a beta version on Discord that lets you generate a 3-second clip from a text prompt. I asked it to generate a video of Margot Robbie blowing a kiss toward the camera. The closest Pika could get was Tammy Faye Baker getting ready to give a hummer.

[4] Tin Toy won an Academy Award for Best Animated Short Film, making it the first fully CGI film to win an Oscar.