There are plenty of opportunities for designers and brands to explore the possibilities of augmented reality (AR) and how it can enhance creative work, provide entertainment or better society.
Tech giants such as Microsoft, Google and Apple – but also everything from children’s books to 3D modelling for gaming – are experimenting with AR. The abundance of free content creation apps are democratising AR, which means anyone (not just developers) can create their own AR experiences.
And with the very near release of the first phone on the market that is AR and VR compatible – the Asus Zenfone AR (seen below) – we take a look at what it does, and what this means for democratising currently expensive and inaccessible technology to most of the public.
For more on the accessibility of AR and VR, see our feature with Blend Media founder Damien Hollier: Is VR peaking too early?
This feature provides a broad overview on what augmented reality is, what it’s used for and how you can use it to create your own content.
What is augmented reality?
With lots of buzzwords being thrown around lately, it’s important to understand exactly what augmented reality provides (yes, it’s moved beyond just QR scanning) for the user compared with mixed reality and virtual reality.
Augmented reality uses existing reality and physical objects to trigger computer-generated enhancements over the top of reality, in real time. Essentially, AR is a technology that lays computer-generated images over a user’s view of the real world. These images typically take shape as 3D models, videos and information.
How this is overlaid depends on the nature of the experience - and the hardware you’re looking at the experience on. The simplest way is using your phone - where what you see through the camera has digital elements added to it. The best known example of this is Pokemon Go - but it has also been used to guide people around places, with virtual signposts added giving directions, or in art galleries to bring up information on artworks as you look at them 'through your phone'.
This can feel clunky, so for a more immersive experience you can slip on a headset such as Microsoft's HoloLens - which frees you up from having to hold up a phone and makes the mix of the real world and CG (computer-generated) elements feel a lot more real.
Augmented reality vs mixed reality and virtual reality
Microsoft brought the term 'mixed reality' (MR) back into the spotlight when it released the HoloLens (see what the HoloLens can do for designers and artists here). Some predict it might have been to distinguish their technology as something new and different. Microsoft says it was used to simplify the VR/AR spectrum. Either way, the new term may have caused more confusion than it’s worth.
Mixed reality lets users interact with virtual holograms and real objects within their physical world. Although this is what augmented reality does – mixed reality is said to anchor virtual objects to a point in real space, making it possible to treat them as ‘real’ from the perspective of the person using MR devices. Essentially, the digital illusion is harder to break.
AR is also not to be confused with virtual reality (VR), which is a fully immersive experience for the user without interaction of the physical world around them. VR is a computer-generated simulation of 3D images (or usually an environment) that can be interacted with by the user in a seemingly real way through the aid of a headset and/or gloves, but it’s essentially quite an isolating experience.
How does AR work?
Augmented reality works in conjunction with headsets and/or digital devices such as tablets, smartphones and even PCs. The devices themselves contain software, sensors and digital projectors that trigger digital displays onto physical objects.
AR can be created and consumed in many different forms. For example, Google Glass displays 2D images onto see-through glasses, whilst Microsoft’s HoloLens embeds 3D images into the world around you. Apps such as Blippar, Zappar and Aurasma allow affordable (if not free) AR content creation for small businesses, education and plenty more industries, and of course AR features within existing smart phone apps and games, such as Snapchat and Pokémon Go.
The AR process uses a camera with "scanning mode". To use some AR creation apps like Blippar, you’ll need to point the scanning camera at different objects in the room to create a database of shapes and corners. Once the camera detects a ‘triggering’ object from it’s database, a digital object onscreen will appear in the position of the target object, as seen in the video below.
For developers this process involves a tonne of underlying algorithms, and sometimes the system has to detect thousands of different target images at the same time, and that can cause teething issues in the software.
Some of you may already use AR in your daily lives without really realising. Pokémon Go (although the game also relies heavily on geolocation), Snapchat’s selfie filters and more recently 3D world lenses, and Facebook’s AR Studio for example, use basic AR to create face masks and animations triggered by motion, gestures, facial expressions and user surroundings.
What is AR being used for right now?
Discovering what AR is and how it works is helpful, but what’s important is how AR can affect and better industries and professions.
Research into what fields AR is currently being used in revealed that the opportunities are endless, but regular integration hasn’t yet moved far from marketing. In recent development, Apple released its own ARKit in iOS 11 (in beta at the moment), which makes it easier for developers to offer AR capabilities in their apps without having to develop the AR system, but it’s only available on expensive Apple products. If you want to know about Apple’s AR developments, read this latest rumours roundup on our sister site Macworld UK.
Another major AR player is Google and its core technology Project Tango. Google's Project Tango (AR) and Daydream (VR) offerings are available with the ASUS ZenFone AR, a smartphone available to pre-order in the UK, with a release date at the end of this month.
This may not seem exciting on face value, but it’s a sign of how AR and VR can be accessed by more of the public, rather than just whoever can afford expensive headsets. Although the smartphone itself isn’t cheap (or a particularly well known brand in the UK) it means that a world of AR and VR entertainment can be made available at your fingertips.
Essentially Project Tango is a set of sensors and computer vision software developed by Google for AR experiences. So much like the functions of other AR apps, the Zenfone AR tracks motion, memorises the physical surroundings and can use that technology to create an interactive world attached to your reality. But the phone is also Google Daydream compatible. This is Google’s latest mobile VR viewer headset – a step up from Google Cardboard. Using the Asus Zenfone AR, you can experience a range of Daydream apps and games such as Netflix VR.
The phone, with its advanced sensor array, was first announced at CES this year, but the UK release was left ambiguous – “this summer” was the most detail given. But now you can pre-order from Asus for £799.99 with an expected release day between August 28 and September 1. One of our colleagues, Lewis Painter, over at our sister website, Tech Advisor, managed to get the Asus Zenfone AR phone in for review. Check out what he thought.
Meanwhile Microsoft quietly revealed a new prototype of AR glasses that look like a normal pair of thick-framed glasses, suggesting it could be providing a less expensive product to the HoloLens released two years ago. More on this is expected to be announced at the Siggraph conference in August.
Industries such as architecture, education, gaming, film, medical, children’s books, military and even the humble grad student (see LCC student Peter Roden’s final project) are experimenting and using AR. Below we outline some of the major industries.
AR is a new strategy adopted by many companies wishing to combine elements of print and online advertising whilst offering an interactive experience. For example, Zappar is an AR company that creates content for brand advertising. Consumers can download the Zappar app for free and point a smartphone at a traditional flyer and then a 3D character or scene will appear, or extra information about the product, or links back to the brand website.
For example, look at what Blippar and Mindshare created for Land Rover.
In turn, businesses can download Zapworks Studio for a small price to create, edit and track the success of their own AR content.
Brands can place trackers in print catalogs, movie posters, packaging or products to trigger AR experiences such as 3D animated characters, extra information, or links directly to online purchasing – even the US Postal Service adopted AR, meaning users could scan collection boxes to watch a holiday-themed experience.
Retail brands are an example of the instant benefits AR provides for a business. IKEA customers can now take pictures of their own homes, then use an AR app to position 3D models of IKEA products wherever they wish within the home, as seen in the video below. IKEA is also expected to adapt Apple’s AR technology, with around 5-600 products available at its launch.
London-based UX designer Radu Fotolescu predicts augmented reality will boost the retail sector, and help simple tasks like finding the right part in a car when it breaks down in the future (as seen below).
Augmented reality is probably best known for its use within games, especially Pokémon Go. AR gaming essentially integrates game visual and audio content with the user’s environment in real-time.
Just type 'Augmented Reality Games' into Google to get an understanding of what’s on offer right now. I tried Zappar’s mini golf game – the experience of hitting a digital mini golf ball with a physical cardboard controller and headset (seen below). We've also featured SwapBots, a children's game which pairs with an iPad, earlier this year.
AR games can be used as another source of branding experience for consumers.
But beyond mere pleasure and entertainment, AR has some serious societal benefits as well.
In a world where simply studying a textbook or YouTube video isn’t enough to stimulate young digital-native minds, AR offers a new experience of transferring information. AR hardware such as the ZapBox kit offer a much more affordable option for schools to purchase for multiple classrooms, in comparison to the HTC Vive, Oculus Rift, or Microsoft’s HoloLens which currently sits at a staggering £2,719 for the development edition.
Educators can create their own learning environment using software like Blippbuilder, monitor student performance and share and collaborate with different teachers, students and content providers.
For example, Volcano Activity is a prototype being tested in classrooms right now, available through Blippar. Students can download the Blippar app onto a smartphone, aim and frame the whole image and watch a 3D model of a volcano erupt before opening detailed information and a quiz.
Museums are also working with AR technology to provide more information about exhibitions. The British Museum has forged a partnership with Samsung and is trialling in-gallery apps.
Architects are experimenting with different augmented reality apps and software – from Microsoft’s HoloLens to free apps – mainly used to build 3D virtual building models which can be experienced from the interior and exterior using an AR headset.
But unlike virtual reality, AR allows architects to project digital information onto physical scale models. From a commercial perspective, architectural firms can sell and present building projects using AR to digitally create a 3D model of floor maps and brochures, for example.
How can I create my own AR content/project?
There are lots of different apps, software and headsets to get your hands on for an augmented reality experience, all of which vary drastically in price and availability. The high-end option is Microsoft’s HoloLens, which claims to be the first self-contained holographic computer. Check out what it can do for designers and artists in our extensive review. And of course, we’ve already mentioned Apple’s ARKit if you own the right devices.
But if you’re a creative professional wanting to get a handle on AR and how to use it, it’s worth experimenting with the following affordable apps and drag-and-drop software.
Speaking to a student at London College of Communication’s (LCC) grad show this year, we discovered Aurasma – a free app he used to create an incredible AR project. Aurasma is available to anyone with an email address. All you need to do is create an account, create content using its drag-and-drop web studio (which also measures commercial results), share it with customers or fans and download the app. You can even create your own whitelabeled instance of the Aurasma app or embed the technology into new or existing apps. Brands like Disney and AMC Theatres already use it.
Aurasma is aimed at marketers, agencies, developers, educators and students. Watch the video below to learn how to create AR in Aurasma Studio.
Another app and AR software provider to "democratise" the technology is Zappar. Not to be confused with Google Cardboard, and drastically cheaper than Microsoft’s HoloLens, Zappar sits at an interesting place in the market.
It began solely to create AR for brands – and continues to grow a strong in-house creative team (including 3D artists) – but branched out to offer software primarily for small businesses to use and create their own AR branding.
Zappar has proved to be popular among the entertainment industry, such as film advertisement and game developers because it welcomes a simple interactive experience. The best AR apps will feature immersive game-play, linking through to other online experiences and the sharing of photos, videos and information.
3D artists, such as game developers and graphic designers, can create 3D models in software platforms they know well (such as Unreal Engine and Maya) then transfer the files to Zapworks Studio to create an augmented reality experience. Learn how to use Zapworks Studio in the video below.
Learn how to use the Zapworks Designer tool in the video below.
The free Zappar app can be used across all smartphones – Android and iOS – making it much more accessible than Microsoft or Apple’s offerings.
It’s currently being experimented with in education, architecture and branding. Anyone can start a free trial and experiment. Along with Zapworks Studio software, the ZapBox kit is available – including a cardboard headset and controllers.
You may have heard or used Blippar already. It's primarily used by brands such as Dr Pepper and Marvel Studios, or as an education tool. By using Blippbuilder or Blippbuilder Script, anyone can create their AR experience.
We haven’t seen a lot of AR-integrated illustration yet, but Scarlet Raven’s The Danger Tree project is an outstanding example of how to use AR as an artist. After downloading the Blippar app, place your phone over her traditional paintings to see new scenes and interpretations of World War I, as seen below.