An Interactive Experience at Hard Rock Vault
August 3, 2020
Building Connections with Brand Experience Design
September 18, 2020Projection mapping, to me, is a truly fascinating subject.
I became quite entranced with the tech while working on our Heroes and Legends project for Kennedy Space Center Visitor Complex back in 2016.
Our media team was integrating both projection mapping and a hologram/Pepper’s Ghost effect on the actual Gemini 9A space capsule. This allowed the spacecraft to show guests the harrowing tale of its historic mission. In my eyes, this was the most impressive magic trick I had ever seen, and I was one of the lucky few behind the curtain who helped create it.
The concept of projection mapping is simple. A projector shines an image onto a surface, not that different from a traditional movie screen setup. With projection mapping, though, the projection surface is not always a flat rectangle or even a stationary object!
The light hits the object, it scatters, it distorts, and now both the projection and the object have become bound. If the image being projected is adjusted correctly to account for the distortion of the light wrapping around the object, it will appear as if the object is now animated and alive. The object could be something small, such as a sculpture in a room, or as massive as an entire building. The world is a digital artist’s canvas in what is perhaps the most spectacular version of augmented reality for the masses. Beyond traditional hard surfaces, you can also projection map fog, water fountains, or even semi-translucent screens to create hologram-type effects.
Projection mapping is not a new concept; its roots go all the way back to the liquid light shows of the 1960s when projectionists filled trays of inks and paints that created psychedelic visual effects on stages and screens. In the late ’60s, Disney relied heavily on the simple concept to create talking statues, mystical crystal balls, and ghosts for their Haunted Mansion attraction.
At the turn of the century, the evolution of projected technology gave us the ability to create massive dome theater shows, such as Falcon’s own Dragon’s Treasure™ attraction or the revolutionary Turtle Trek® 360° 3D experience at SeaWorld Orlando. These experiences feel very similar to today’s headset-based virtual reality experiences. I refer to this as mass mixed reality because the feeling is the same, but you can experience these theatrical presentations with large groups of friends.
Dragon’s Treasure™ inside of the world-renowned City of Dreams Casino.
In the past couple of years, projection mapping has gained significant popularity, mainly due to social media. Festivals such as the Amsterdam Light Festival, Luma Projection Arts Festival, Burning Man, and the Toronto Light Festival have attracted thousands of people with cell phones who are more than happy to share the awe of the visual spectacle on their social media feeds. Naturally, this sharing brings even more people to these events the next time they are staged. Because of this, the demand for creative teams that can successfully execute such memorable experiences is rising; it’s the best advertising a festival can hope for.
Large-scale venues are not the only ones who have taken notice of the power that social media has to bring more awareness to things like mapping. Le Petit Chef, a restaurant that created an experience by using projection mapping, has become a social media sensation. The Mori Art Museum in Japan, ARTECHOUSE in Miami, and the National Geographic Museum in Washington, D.C. have used projection mapping to entice new audiences into the realms of art and history and reimagine what the museum experience can be. People see these places in posts and want to play and be swept away by the dreamlike quality of these augmented spaces.
Mori Art Museum in Tokyo, Japan. Photo by rabbit_akra. Photo license.
That last point, of course, is something critical for today's times. Due to the pandemic, many people are asking how we can offer deeply immersive experiences for the masses without having them touch an object. I sincerely believe this will be the era that projection mapping puts its true powers on display because now, with modern tools, we can map so much more accurately. We can create projections that use real-time technology to make them both ultra-realistic as well as fully interactive. Recently, we have even seen the ability to have a single projector display completely different experiences to a room full of people, depending on what set of 3D glasses they are handed.
Projection and tracking technology are evolving so fast that projections can now wrap around moving objects and even moving people. We can make entire rooms project images based on the perspective of the audience, so if they move, the entire environment reacts to them.
Beyond this, we are starting to see the ability for people to interface with massive mappings using their cell phones. Guests can use a specific hashtag or direct message on Twitter to influence a projected environment. The inclusion of mobile AR allows for both a mass experience and a personal one, bringing unprecedented depth and illusion to a space. The inverted use of VR controllers to track movement is another unexpected evolution, allowing experiences once constrained to headsets to be brought into the physical world. Then there is the whole aspect of reactive audio that many of these real-time projection mapping programs can be tuned into. Any frequency or sound can be programmed to react to the projection mapping. These tools are now allowing any space to be transformed into an unforgettable experience.
While the tech is impressive, it still needs compelling content to complete the picture. Audiences are demanding more than just eye candy, though; they want to be moved emotionally by a story or a musical score that guides them through the visuals. Artists now have to think about space in an entirely different way, as the concept of traditional media screens changes with mapping. But the technology also offers an abundance of new options because now, characters can interact with physical objects attached to the space.
The movie-making concept of depth and focus is also altered, as now an entire city can be a stage that projects multiple buildings, which serve as locations that characters can leap to. It’s almost as if media is reverting to its theatrical roots, and we now have to think of virtual things in terms of physical presence. Often, creative teams design a project with a 3D model of the environment itself that is part of the new projection. This creates the illusion that the building or object itself has the ability to move and transform with the characters, producing even more realism.
Surround sound can also take on a whole new dimension with projected spaces because we are no longer in a box, per say. Speakers can be placed in or around objects that are mapped to make it feel like the sound is emitting directly from the digital manifestations that are moving around a space, much in the same way that modern video games use emitters for sonic environments in 3D space. Being someone who has heavily focused on interactive audio design, I see massive potential in both storytelling and music interaction as guests interact with augmented objects; sound always greatly improves the believability of an illusion because our ears help us track where objects are in a much more visceral way than sight can.
If you are a creative person and are feeling inspired by the potential you are reading about here, you are not alone. Everyone at Falcon’s Creative Group feels the same way. Many of us are several years down the rabbit hole of discovery with this extraordinary way to bridge the digital world and the physical one.
Photo by Rebecca Hale/National Geographic for “Becoming Jane: The Evolution of Dr. Jane Goodall” exhibition organized by National Geographic and the Jane Goodall Institute.
Our recent collaborative projects with National Geographic Museum have allowed guests to walk through and around the Church of the Holy Sepulcher in “Tomb Of Christ”; explore ancient tombs in “Queens of Egypt”; sit by a campfire while listening to a life-sized hologram of Dr. Jane Goodall and explore Gombe National Park in Tanzania in “Becoming Jane,” all because advanced projection mapping made it possible.
The question is no longer, can it be done but rather, how far can we push the envelope on something that will become the most impressive magic trick your guests have ever seen.
Meet the Author
Jesse James Allen
Editorial Director
Jesse is an experienced technical and creative leader. As Falcon's Editorial Director, he serves as a resource for uncovering emerging technologies and methodologies.
He has been an integral part of our most high-profile projects, always using his advanced technical skillset and courageous attitude to pioneer new ways to tell engaging stories.
