From the Archive: In Search of Photorealism

This article was first published at www.stuffhappeningnow.com on 2nd February 2010. It is part of a small series of essays I am re-publishing via The Machine Starts. Each piece explores one example of a close, human relationship with digital technology.

I remember buying the 100th issue special edition of the UK video-games magazine Edge. On the editorial page they had a mock-up of what they thought the cover of the 200th edition might look like. "At last! Photo-realism!" was the headline caption in bold letters.

Ever since the very first instance of computer-generated imagery (CGI) made its way onto a movie screen, ''photo-realistic'' digital animation has been thought of by many as the Holy Grail. Edge magazine published their 200th edition last year and photo-realism is still, arguably, unattained.

In Vancouver last week, some of the top names in visual effects (VFX) gathered at an annual conference to discuss the most significant advancements that have recently been made in their field. The convention, SPARK FX 10, drew artists and engineers whose names crop up in the credits of movies like District 9, Transformers, Harry Potter and the Half Blood Prince and Avatar. I wanted to talk to ask them if photo-realism truly is the ultimate goal – and if such expectations can be justified.

Winston Helgason, President of VFX design house The Embassy and his head computer graphics designer, Simon van de Lagemaat, recently celebrated the success of their work in District 9. The computer-generated models used for the final sequence in the film demonstrate some of the basic challenges that face VFX specialists every day. How do you blend an artist’s concept of a character, object or vehicle into ‘filmed reality’? In the original Star Wars movies, for example, each space ship (a real-life model) was built to look dirty, rough around the edges and overly detailed in an attempt to disguise its simplicity and hint at its intended scale.

"What we want is to get to a level of integration where people don’t think they’re watching effects any more - instead they’re just watching a movie."

Simon van de Lagemaat, who developed the digital models for the gritty environment of Cape Town in District 9, explained that the same technique, though in a different form, still applies today:

"The model shop guys used to have boxes of pipes and other parts from old kits that they would put on models like the Millennium Falcon. CGI guys have the same things lying around in their virtual toolbox now.”

Lagemaat stressed that, perhaps paradoxically, what the industry should be striving for is not overwhelmingly impressive CGI, but CGI which you don’t notice because it slots so seamlessly into the footage. "What we want," he said, "is to get to a level of integration where people don’t think they’re watching effects any more - instead they’re just watching a movie."

It’s obvious that people long for this, though their fascination with the ‘effectness’ of effects is still apparent. Take this clip of a luscious jungle environment in the video game Crysis. The player is obviously searching for an immersive, photo-realistic experience, but strangely excitement is still present in notion of the uncanny – that what you’re looking at isn’t real at all.

The fantasy made possible by digital realities remains an attraction of CGI. Artists and gamers alike don’t really want those images to fade away into seamless representations, instead there exists an obsession with the hyper-real: CGI that’s somehow so good, it’s hyper-real, it’s better than reality.

However, digital effects studios do now spend a great deal of time trying to weave their projects more carefully into movies. Complementing the narrative in an appropriate way is essentially, lest the CGI project fail to connect with the overriding project of the film’s story. Visual FX workshops don’t come up with the script and they certainly don’t get to direct the movie. Instead, movie directors are their clients, and it’s the VFX guys’ job to get to know the script and the director’s vision intimately in order to produce effects which are consistent with their client’s needs.

"And in terms of industry awards like the Academy awards," commented Winston Helgason, "effects are now rigorously judged by their closeness to the scripts. We often see people getting angry at bad effects if they don’t fit the story well. It’s part of the production now.”

Reality Check

The level of involvedness for effects artists working on expansive projects such as feature-length movies has multiplied enormously in the last ten years. Animators like Chris Horvath, who has been working in the industry since the very beginnings of CGI, and who is now Digital Effects Supervisor at Industrial Light and Magic, have begun to take inspiration from outside the VFX industry, bringing surprising sources of stimuli to their desks every day.

In order to approach a truly convincing reality made purely out of pixels, animators have begun to try and understand their own realities in minute detail. Chris Horvath, for one, finds fractal nature a plentiful source of inspirational intricacy:

"I work in adapting physics into artistic spaces. One of the things I’m most fascinated by is what I call growth patterns - the way different plants go from being just a bud to something much larger and complex.

“There’s a vine growing in my backyard and when it buds it has three little shoots which always appear in exactly the same formation. When the vine repeats that over and over you have this incredibly beautiful and complex thing which can nevertheless be broken down to an intensely simple repetition."

For Winston Helgason, it’s classic movies, devoid of VFX, that inspire him to understand the fundamental properties of good story-telling. And Simon van de Lagemaat turns to photography to, in his words, help him trust his "own eye."

He adds, "I don’t think there’s any particular hobby that’ll make you a better artist, but all the best VFX guys I’ve worked with force themselves to be better at observing things and recalling them when they need the inspiration.

“You’ve got to be able to look at things and question them and know that it’ll buzz you when it looks wrong. You’ll have a kind of low-end OCD when things don’t look right. It’s why a lot of our guys like photography or drawing still life."

"You’ll have a kind of low-end OCD when things don’t look right."

At this point we are venturing into the strange world of the VFX artist and realising that in order to produce elements of the hyper-real which satisfy, he is constantly expected to remind himself of and regurgitate aspects of the real – but the real learned in the context of CGI production; the real as building-block or tool-box. Whereas the hyper-real is an unattainable goal which promises to one day sweep itself over the real, cancelling out its questions and mysteries.

In this long, winding quest, many CGI specialists find themselves experimenting with self-initiated side-projects - purely out of a desire to learn more about the physics of CGI and the capabilities of the software they use. Chris Horvath, for instance, plays informatively with a simulator he developed which he calls, quaintly, a "universe builder."

It takes a basic star-cluster and extrapolates it over and over again based on a few predefined rules and the physical laws of matter. The simulation can be set to run in a loop and every time it churns out wildly different results. "What we want," Horvath explained, "is for people to be able to start throwing together believable mock realities in no time at all."

Lost in translation

But there are still some significant obstacles that lie in the way. CGI has reached such levels of unparalleled depth that words themselves have begun to fail the people working at the frontier of digital visualization.

Horvath repeatedly stressed that the acute characteristics of difficult-to-render materials like human skin are impossible to express verbally. Impossible, that is, to express those characteristics objectively and in such a way which succeeds in capturing the visual impression of the object while simultaneously doing justice to the physics and number-crunching that has to be done behind the scenes in order for the CGI to look real:

"It’s not really possible for anyone other than super-geeks to interact with most of the terms we use. The mathematical technobabble is often unrelated to the human understanding of reality.

“For instance, when you walk into a room and the walls are lit a certain way you might describe that room as ‘harshly lit’ and everyone would know what you meant, but how do you break that down into objective, mathematically precise numbers that we can turn into accurate visuals?"

What Horvath yearns for, then, is a linguistic quantum leap - extensions and extrapolations in language so that the processes which comprise CGI may be made more comprehensible. For animators working towards photo-realism, the unrelenting specificity of their work forces them to re-evaluate not just how their creations look or interact, but how they discuss VFX on- and off-set. Understanding the gaps in hyper-reality suddenly illuminates features of reality which had hitherto never been considered but which now appear as inconvenient and inexplicable deficiencies.

After all, Horvath is an effects artist who works with an active knowledge of real physical equations and the forces at work in our universe. It’s this that allows him to create spectacular set-piece sequences in movies such as The Lord of the Rings and last year’s inter-stellar adventure, Star Trek.

Extensive movement of multiple solid objects, rendering of liquids and programming individual particles of dust all present frightful challenges to VFX mechanics. This company specialises in animating such substances alone.

The fruits of this exhaustive labour are already apparent. What Horvath and others are making possible by their pseudo-ascetic dedication is, in Horvath’s own words, a "democratisation of story-telling and film-making." The technologies, software and animation techniques which supergeeks are constantly developing allow amateurs like this to attract the attention of big movie studios in an instant.

And this astonishingly life-like and unusual film has also surfaced online recently. It’s a clear example of an illusive reality that has, until very recently, been an elusive reality for animators.

But for effects specialists like Horvath, Helgason and Lagemaat, the goals they are setting themselves have gone beyond photo-realism. It’s a term which, after all, seems merely to suggest the regurgitation of everyday reality, subject to the same entropy as a photograph. The search for a hyper-realism which incorporates a completely new kind of entropy has taken over and ought to be defined as separate.

"We’re just starting to scratch the surface in constructing these realities," Horvath said. He added that, listening to fellow animator Roger Guyett at the SPARK FX conference, he was inspired so much by one particular remark that he scribbled it down there and then.

Reaching for it as he talks to me on the phone from San Francisco he explains, "Guyett commented that one of the reasons he loves working with filmed elements from movies is, 'No matter how good we get at creating computer graphics we will never be able to fake the joyous imperfections of reality' - there’s something wonderful about that statement."

Reality remains esoteric and the search for photo-realism has become a different project entirely. The horizon seems wider, the obstacles and difficulties endless. But of course, the humans embedded in this quest speak about it with all the fascination of those who stand on the edge of a new frontier. The hyper-real is calling, and it’s changed everything.