Push the boundaries of digital design using projection-mapping, Arduino, Microsoft Kinect & more

One of this year’s up-and-coming creative trends is ‘experiential’ design. It’s an umbrella term for innovative projects that encompass not just visuals, but also sound, touch and even smell – all driven by real-time feedback generated by the actions of viewers and the world around them. Michael Burns explores how you get start creating your own experiences.

You can think of it as interactive design that has outgrown the computer or cyberspace and entered the real world, where the input driving the experience can be anything from the reading on a temperature gauge to the movement of the ‘participant’ in front of a Microsoft Kinect. Meanwhile the output can range from projecting responsive visuals onto an object to triggering a camera to capture shots at exactly the right moment. Here's some recent examples of the wide variety of projects this field encompasses from Leviathan's groundbreaking tour visuals for Amon Tobin (above) to a hanging garden installation.

Experiential design is an exciting field that’s still in its infancy and one that is, in principle, relatively easy to get into using a mix of inexpensive hardware and free software (alongside standard creative software). “Projects using devices like the Arduino tend to lead to unique, engaging and highly visually creative [work],” says Adrian Campbell of Belfast studio The Design Zoo (thedesignzoo.co.uk). “This is due to the fact that [such work] allows a deeper level of engagement with the viewer and also requires you to keep the engagement levels high through your visuals, animations and interactions. 

“For us, the Arduino opened up a brand new medium for our designs to be shown on, and in a uniquely interactive way,” he adds.

That said, projects aren’t confined to producing rich interactive experiences for attendees at a live event (or users on the web). Experiential work can also be a creative tool, enabling you to produce non-interactive art or designs that would otherwise be impossible. 

What binds all experiential projects together is a keen interest on the part of designers to channel their inner geek to hack and adapt open-source hardware and software, all in the name of furthering the cause of design.

“There’s always been an undercurrent of experimentation and geekiness running through the digital-creative scene,” says Dave McDougall, head of digital at the London offices of cutting-edge agency Studio Output (studio-output.com). “I remember hacking one of the first Lego robotics projects many years ago and working with some quite cool technology then: in-store installations of sound-responsive projections and very early video social networking made by hacking together a heap of technology. 

Perspective Lyrique by 1024 Architecture uses an old theatre in Lyon as a ‘screen’

“But what we’re now starting to see is this technology maturing and forming part of larger, more prominent campaigns. Clients and brands are really embracing it, and that’s where we’re seeing some great things happen.” 

With clients excited by the technology and, more importantly, willing to commission projects that use it, it’s no wonder that experiential design is seeing such a surge in activity. One area where there have been some high-profile projects already is projection mapping – a hybrid of live, VJ-like visuals and 3D modelling. 

The central idea here is to sculpt video content to match the geometry of the surface onto which it’s being projected. VJ software such as the Mac-only VDMX (vidvox.net) and the newer MadMapper (madmapper.com) can be called upon to handle the visuals. Both applications are often used in conjunction with Quartz Composer, Apple’s visual programming environment for processing and rendering graphical data. 

A tablet-based app created to control the visuals for Amon Tobin's ISAM tour.

Chicago-based animation, VFX and motion design house Leviathan (lvthn.com) has been involved in several large-scale projection-mapping projects, ranging from marketing events for Dodge cars to dazzling visuals for musicians like Amon Tobin. Executive creative director Jason White feels there will be even more projects in this genre this year. “It’s fresh to so many because, if executed correctly, it captivates audiences and pushes the boundaries of what video can do.”

Projection mapping is only one of the multiplicity of possible toolsets within experiential design and – as with much in this field – you don’t have to be involved in ‘event’ productions to get in on the act.

“You can create a huge public installation that uses expensive, advanced technology or do amazing, innovative things running a project off a single Arduino kit and a network connection,” says Dave of Studio Output. “As the technology is relatively easy to learn and the entry costs are low, the only limit is your imagination.”

Dave says open-source software and toolkits such as Processing (processing.org) and Open Frameworks (openframeworks.cc) are “reasonably straightforward to pick up”, adding that as with Arduino, “there’s a massive and very open community of programmers, artists and creatively minded people sharing information about these technologies, so there’s always help and inspiration around the corner”.

Note: We may earn a commission when you buy through links on our site, at no extra cost to you. This doesn't affect our editorial independence. Learn more.

Read Next...