Stories in code
From Cinema to Kinema: Film in 2030
From “flatties” to the Holodeck, film is no longer the only choice for moving image stories
In February 2020, the BFI RSU team asked a group of top thinkers and analysts whether they could predict what 2030 might look like for the moving image, audiences and the broader screen sector. Their findings and views were explored in the BFI’s second Insight Exchange forum.
While their discussion was wide ranging and contain a wealth of thoughts and ideas, their key findings could be roughly distilled into four points:
- The choice of films to watch has exploded across all platforms, including gaming and immersive
- The big ideas / drivers that are shaping the film industry are coming from outside the conventional film sector e.g., from computer science, immersive arts, big tech, gaming
- Linear storytelling and workflows are receding as new types of stories and storytelling emerge
- Silos are too limiting. We must become agile and multi-skilled.
These findings are not too surprising. Much of this was alluded to by Janet Murray in 1997 in Hamlet on the Holodeck, which was updated and republished in 2017 to celebrate twenty years (2017). Murray posits that the trajectory for storytelling has already been identified in the creation of the “Holodeck” and “Holonovel” as introduced in the 1990s American TV series Star Trek: The Next Generation and Star Trek: Voyager.
This reflects the continuing cycle of innovation in making, distribution and audience engagement. This doesn’t mean that traditional cinema is dead, but rather that it is no longer the sole occupant of the moving image story space.
Before Covid-19 the figures for box-office in the cinema had remained constant for the last 25 years, revenue only increasing through rises in ticket prices, not more “bums on seats” (The Numbers). During the same period (since 1995), we saw the rise of the World Wide Web, video streaming and social media (to name a few). Mind bending numbers include:
- Around 500 hours of video are uploaded to YouTube every minute (Statista)
- Facebook currently has around 2.8 billion subscribers (Statista)
- YouTube has around 2.3 billion subscribers (Statista)
- WhatsApp has around 2 billion subscribers (Statista)
- Instagram has about 1.4 billion subscribers (Statista)
- Netflix — the leading streamer of linear content has 192.95 million global subscribers (Statista)
These are factors that give us a clue as to what 2030 might look like for film.
The cycle of innovation is often driven by a combination of creative endeavour and pain. The creative endeavour is exploratory and asks what comes next and how can things be improved. The pain is from those who are finding the existing technologies and workflows less fit for purpose as the industry or activity evolves.
On the road to 2030, film is experiencing a significant paradigm shift in the form of virtual production (which has been accelerated through Covid 19). Virtual production is not a “thing” e.g., new piece of software or technology. It is an evolving approach to story creation, production and distribution based on advances in computer graphics and machine learning. It is driven by both creative endeavour and pain. For the remainder of the piece, I will focus on the main considerations driving virtual production.
The first thing to note is the context in which virtual production sits. As outlined earlier, the numbers of people who engage with or access stories is substantially larger on online / social media platforms than through traditional cinema or streaming services. This is proving an existential challenge for the monolithic studio system.
Below are three areas to illustrate the creative endeavour and pain driving virtual production.
Production risk
Most films fail. The analysis and reasons for this could fill a book, but here a two:
Expense and risk:
Getting film into production is incredibly expensive and the investment cycle does not match the spending cycle. Making a film is more expensive than buying a house. This means that raising money to make a film is extremely difficult. Studios fund film through amortising across productions, co-productions and outsourcing. They also provide ancillary services such as providing production services to other studios or even other industries e.g. gaming. Betting on box office is extremely risky.
If you do manage to raise investment, standard industry practice is for it to arrive on the first day of principal photography. This means that development and preproduction is often unpaid or paid for out of the pocket of the producer (bankruptcy is a constant risk for producers). It also means that there is a powerful temptation to cut corners to get to the first day of principle photography as fast as possible.
Consequences of the current model include underdeveloped scripts, poor budgeting, and scheduling, frequent exploitation of cast and crew and poor decision making. All these lead to poorer films with less appeal to audiences. They vote with their feet and watch YouTube instead.
Complexity from too many moving parts:
Have you ever marvelled at how long the credit roles are getting at the end of films? Many of them now take longer than the average pop song often played to accompany the roll. In part this is an attempt to recognise all the people involved, but that number seems to be getting bigger and bigger. Are they really all needed?
One consequence of all of these people and disparate production services is a lack of standardisation and compatibility leading to an increased number of points of failure.
To begin with, many films today — certainly all modern blockbusters — are exceedingly complex productions featuring many moving parts on highly compressed schedules. The process is typically a linear affair, one that resembles an assembly line, encompassing development, pre-production, production, and post. Iteration is challenging and costly while development efforts are frequently siloed off from one another, with incompatible resources. (Virtual Production Field Guide)
This leads to poorer films with less appeal to audiences. They vote with their feet and watch YouTube instead.
Real-time production and games engines
Real-time production is here!
Modern games engines such as the Unreal Engine and Unity 3d provide a means for creating stories in code. A games engine or games framework is an integrated coding and graphics development environment used for creating games. Core functionality typically includes capability to render 2d/3d graphics, a physics engine for handling collisions, a shader engine for handling materials, a particle engine, audio, animation, networking for multi-player games, the ability to create “cinematics” for “cut scenes” and the ability to author to a wide range of platforms.
We are still discovering the types of stories that can be created when we think about power and versatility of games engines. The problem is conventional film production is almost the antithesis of this, as described above. Film is a static medium. Once frames are shot, they cannot be changed. Yes, they can be manipulated with VFX, but camera positions, lenses, shadows, perspective cannot be changed.
Cocteau, a contemporary of the Lumiere brothers and early film maker, while excited by the new medium of film also saw the cinematic image as temporary. This idea permeates the essays contained in “The Art of Cinema” where he declares in an essay from 1934 that the “mind has already started to weary of a succession of flat images continuously unreeling on the old magic lantern screen…” (Cocteau, 1994, p. 27). Storytellers and audiences are always looking for innovative ways to engage. Games engines are currently changing the game!
Data
The most pressing problem is that video content is unstructured data. Unstructured data is information that does not have a predefined data-model or schema accessible outside of itself. It cannot be easily (if at all) accessed and understood by other systems. It is estimated that about 80 percent of today’s data is unstructured in the form of emails, chats, social media, audio, and video (Isson, 2018).
The data in video content consists of “pixels” made up of “bits” containing colour and brightness information. It does not contain any semantic information relating to its content or what the collection of pixels that make up that particular frame actually mean. Every minute more video than any person could hope to watch in a lifetime is being published through YouTube, Tik Toc, created by CCTV cameras, dashcams and the like. Filmmakers, similarly, contribute a large amount of footage to this stockpile of unstructured data.
Being unstructured data means video is not indexable, searchable (Hoy, 2018). In its findings, the BFI referred to this as the tyranny of choice. Too much to see but also too hard to find. Film is also inherently not accessible to people with disabilities such as blindness, people who don’t speak the language the film was created in or people who don’t have access to the device or medium to which it was authored.
It can’t be used for any other purpose except as static video, and it can’t be delivered or viewed on anything except a media player that supports the format or codec that video is created in. In the context of the BFI report, it is losing its fitness for purpose as stories explode across different platforms. It also runs the risk of being lost through obsolescence. If somebody gave you a roll of film or a VHS video, do you have the equipment to play it?
Where does this unstructured data originate and why is it unstructured? The simple answer to this in film is the camera (referred in the field of computational photography as RGB cameras). RGB cameras are essentially the same today as they were at the turn of the 20th century. They have changed from chemical film, to analogue tape to digital sensors and lenses improved, but the fundamentals remain unchanged. They were not developed with structured data, semantics and ontologies in mind. Maybe that needs to change.
Virtual production offers the opportunity to address some of these pain points. Games engines being integrated development and coding environments allow us to annotate and describe our stories in code, making them machine readable, indexable and searchable.
Like climate change, we do need to be addressing these issues well before 2030. What might the film industry look like in 2030? Cinema comes from kinetic, which is relating to or dealing with motion. Kinema is just cinema emphasising its root as a medium of motion. In this sense, the emergence of virtual productions currently means that the motion or direction of travel is up to us.
Stories in code is made for sharing. If you enjoyed this article, please show your support by giving it a clap, following me and sharing with others who you think might be interested.
References
Cocteau, J. (1994). The Art of Cinema (trans., R. Buss). New edition. Bernard, A. and Gauteur, C. (eds.). London: Marion Boyars Publishers Ltd.
Hoy, M.B. (2018). Deep Learning and Online Video: Advances in Transcription, Automated Indexing, and Manipulation. Medical Reference Services Quarterly. Routledge. Vol. 37 №3. pp. 300–305 [online]. https://doi.org/10.1080/02763869.2018.1477718.
Isson, J.P. (2018). Unstructured Data Analytics: How to Improve Customer Acquisition, Customer Retention, and Fraud Detection and Prevention. 1st edition. Wiley.
Murray, J.H. (2017). Hamlet on the Holodeck: The Future of Narrative in Cyberspace. updated edition. Cambridge, Massachusetts: MIT Press.
