Artificial Intelligence for Filmmaking and Storyboarding
I was not a fan of podcasts for a long time, but I am a weekly listener for a bunch of them right now. One of them is Inner Cosmos with David Eagleman. He is a famous neuroscientist and author from Standford University. In a recent episode, he discussed Artificial Intelligence and argued that it is a significant turning point in the history of mankind. He compares this turning point to another one, and I really liked this comparison. Around 150 years ago humankind invented camera and started taking pictures of everything. Eagleman says this invention came with chaos for painters because there was no need for them anymore. We could just take pictures instead of paying painters large sum of money. The painters could not reflect reality as well as the camera does. Eagleman says this was not the reality. Unlike the common view, painters found a different purpose to sustain their artistic lives. I think this is a great example because Artificial Intelligence is helpful for many things while we still need human thinking and creativity in every aspect of our lives.
After publishing my first article about AI and how to benefit from it as a filmmaker I participated in a filmmaking workshop in my school. It was not a very advanced workshop but we had the opportunity to work with film cameras and I would do anything to get my hands on those cameras just like any other student with zero budget. During the first class, we were having a discussion and I asked our instructor who is a Canadian commercial director about his opinions on AI-generated storyboards. He mentioned that he used to take pictures and screenshots of the scenes he likes from the movies and gathered them in folders, but now he can also search for a specific shot he has in mind and generate with AI. Currently, he is using both techniques and increasing artistic productivity. However, he said AI shouldn’t replace storyboard artists and I can understand it. AI cannot match the interaction and the flow of creativity that comes with it. But as artists, especially as no-budget student artists we can get help from AI. I think Midjourney and other AI programs can be used in a way to get influence and feedback for our creative ideas.
Following the previous article that I published here, I would like to give some examples of using AI to generate storyboard ideas for no-budget filmmakers. I will use the script of my second short film which is still in pre-production. The story follows a young photographer who needs a job urgently in Toronto. He is having financial problems and his time is running out. While talking on the phone with his mother he mentions that he has money for few more weeks and then he will leave Toronto and go back to his city. A middle-aged man overhears this photographer talking on the phone while sitting across in the subway.
First, we can try to create these two characters’ look for the film.
a) A young, skinny and tall person who is around 25–28 with black hair and black eyes sitting on the subway, and talking on the phone. Looks tired, he has a camera and a briefcase next to him. He is in the middle of a conversation. (prompt that I wrote to Midjourney)
As you can see the details are very different between the first two images even though the concept and the scene stay the same. In my prompt, I did not mention the placement and the shot selection for this particular shot from the scene. However, AI provides a very helpful suggestion for it. We have medium shots in both image, which seems like a good choice because it captures the location that our character, his clothes, belongings and most importantly his feelings. All in the same shot. Additionally, in both images, Midjourney placed our character in front of a window. This is also a significant thing to consider in filmmaking because windows can help with the lighting of the scene. Especially in the second image, we can that the light comes from the window and lighten parts of his face and his belongings. This lighting option plays a key role in the scene.
I highly suggest trying different styles to find the best image that fits your imagination. I tried two different styles by writing things like “neo-noir style storyboard” or “comic style movie storyboard”. Both options would work for me for this particular project. For the following images, I asked Midjourney to generate a “hand-drawn storyboard” and a “70’s noir style storyboard”.
b) A middle aged man (35–40 years old) who starts to look and hear the conversation. Wears casual clothes, a shirt, pants and a hat. There are two seats between them. (Second prompt)
c) Middle-aged man focuses on the conversation he is hearing. He notices something interesting in the conversation and his attention shifts away from his newspaper. He starts to wait for the phone call to finish to say something to the young man. He does not want to interrupt but he looks impatient.
The most exciting thing is the variety in these images. I only mention a few characteristics about the character and leave the creativity to Midjourney. This is a great example of an interaction that artists can benefit from. In every image, the idea of a middle-aged man differs. As filmmakers, we usually create everything in our minds and try to reflect them into reality. But, I think it is always better to evaluate other opinions and options. Artificial Intelligence is able to produce lots of options that reflect our ideas. It is an opportunity we didn’t have before. There was no possible way of seeing every option and detecting mistakes. With Artificial Intelligence, filmmakers will have this trial-and-error method in our tool kit. As I mentioned before, I think Midjourney and other AI programs can be used in a way to get influence and feedback for our creative ideas. I believe AI technology will play a significant role in filmmaking, and it is better if we start using and understanding it as soon as possible.
I will continue writing about Artificial Intelligence, and I will be sharing new articles as well. Until then, I would like to hear from you. Please let me know how you use AI for filmmaking, what you think is missing, and how can we improve ourselves with it. See you in responses.
