As Microsoft's AR headset goes on sale in the UK, I spent some time trying out the HoloLens to find out what you could create for it – and what you could create using it.
Using the Microsoft HoloLens is an experience quite unlike any other. You put on the rather snug headset and things appear in front of you that aren't really there. And then you can do stuff to them.
A CG T-Rex appears to sit in the middle of a real living room. I can walk around it, and it feels solid (though it's clearly digital). It roars at me rather intimidatingly, so I move my hand in a gesture to shrink it down to the size of a toy – and it's a lot less threatening.
This is a part of a game called Actiongrams, but it's not a big leap to imagine a similar setup being used professionally by VFX house about 100 metres from where I am for the HoloLens demo (we're at Microsoft's Lift Studios in London, the opposite side of Oxford Street from Soho where the UK arms of Double Negative, Framestore, ILM, MPC and the like are based). Instead of a dinosaur model from the purposes of entertainment, it would be a work-in-progress character, prop or environment from a film, show or ad. The artist – or supervisor, art director or even the director – could walk around the model to see how it would appear on screen from any angle.
Here's Autodesk demoing how it could work with its Maya 3D modelling and animation software.
Helping with the creation of three-dimensional content is one of the two main uses for the HoloLens, says Microsoft - though what it means by content is very broad indeed. 3D models for the entertainment industries is just part of it. Any area where a 3D object is modelled on a computer before production - such as product design, fashion and architecture - could find it incredibly useful. The aforementioned Autodesk is also developing HoloLens support for its Fusion 360 product design software.
As well as assisting with the creation of content, the HoloLens can be used to consume it too. Previous to the dino demo, I'd seen a luxury watch 'experience' created by Microsoft for demo purposes. A CG rendering of a watch floated in front of me, opening up in 3D space to show its components. I could select each area in turn to reveal a label explaining more about each part of the watch.
There wasn't much to it conceptually. In depth of idea and execution it was no better than those early VR demos (or a mid-1990s microsite) - but that's what it is, an early demo. This one existed to merely show that you could do a 'brand experience' for clients. More visually interesting was a starfield where, as you turn round to view the stars, you discover a moving model of the solar system. Again it's simple, but it shows that - as with VR - exploration and the delight of discovery are going to be important to creating really engaging experiences using the HoloLens.
The first demo I experienced had more depth to it. Created with Cleveland Ohio-based Case Western Reserve University's School of Medicine, I saw a full-sized human body that we could add or remove layers of to explore his anatomy. I watched his heart beat from both inside and out, and saw the different ways bones break - with the contrast between a transverse fracture and a spiral one much more obvious when you can walk around a 3D bone than looking at a diagram on a page. You can see Microsoft's promo video for this below.
But these are still early days for the HoloLens.
HoloLens may have been announced back in January 2015 - and it's been shipping in the US since the beginning of 2016 - but it's still early days in both the development of the device and what you can do with it. Microsoft was very keen to stress that what we were using was the 'HoloLens Development Edition'. The company clearly doesn't want to see the HoloLens killed off by being reviewed by the media as a consumer product - as Google Glass arguably was. This makes sense as, in its current form, it's more of an Oculus Rift DK1 (Development Kit 1) than an HTC Vive.
HoloLens field of view
Like the first, developer-only Oculus Rift, it's visually limited. Oculus' original development headset had a low resolution, the HoloLens has a narrow field of view.
Essentially, the way that the HoloLens works is by putting a transparent screen in front of each of your eyes, putting slightly different CG elements on each screen to allow for the interocular distance and make the results look realistically three-dimensional. The HoloLens projects onto an area that's about the middle half of your vision, both vertically and horizontally. Anything outside this is 'off-screen' and not visible.
For those creating apps for the HoloLens targetting consumers with experiences at brand events, theme parks and the like, this can mess with immersion and mean that getting users to turn round is better done with audio cues than visual ones (the HoloLens has built-in speakers, so you get 3D sound from the device as well as 'real' sounds from your environment). Users have to get used to moving their heads rather than their eyes to look at moving objects, so those trying HoloLens for the first time will probably find experiences with slower moving objects more engaging - though after a while you get used to it.
The HoloLens field of view is likely due to the limitations of the device's processing power. Unlike most full-spec VR systems like today's Oculus Rift or HTC Vive, the HoloLens isn't connected to a host PC - which means you're less constrained wandering around than with the Vive (or stuck in a chair as with the Rift). Hopefully as future versions of the HoloLens are released by Microsoft, increased processing power will widen the field of view.
The use of a narrow field of view has likely also allowed Microsoft to keep the resolution of those screens in front of your eyes relatively high. I wasn't marvelling at the level of detail, but text was crisp and it never felt low-res.
Another constraint is that the HoloLens can only be used indoors. Smaller, more evenly lit spaces such as the relatively brightly lit studio (but made up like a couple of living rooms) that I tried the HoloLens are easier for it to process - and it has trouble in the dark as it can't map the space properly. So you can't build an AR version of the Six Flags VR Superman rollercoaster (for now).
How to control the HoloLens
Like Microsoft's Kinect system for the Xbox, you can control HoloLens apps using gestures. However, it's a lot more basic at this time, and there are a limited set of gestures that you can use within an app's interaction model. First off, you look directly at something - which will either glow, or some apps use an FPS game-style crosshair so you know what you're aiming at.
A theatrical pinch gesture in front of you allows you to select. Hold the pinch and you can drag something. Unpinch and you're done. And that's it, basically.
The whole time your hand needs to be in sight of the cameras on the front of the HoloLens, so dance games are out. Though the likes of Razorfish have been experimenting with combining HoloLens with Kinect - so who knows, perhaps soon you'll be able to create your own holographic version of Paula Abdul's Opposites Attract music video.
Once major difference between the HoloLens and the Oculus Rift DK1 is that while the latter felt almost gaffer-taped together, the HoloLens hardware feels like a finished product. It's solid and actually quite stylish in a utilitarian kinda way - at least next to the super-chunky Oculus and Vive headsets. It's also robustly made, says Microsoft, and quite capable of being used by 20 or so people per hour in a traditional brand experience or theme park-style setup.
Battery life is between two and three hours, and it takes about the same to recharge it.
How to create HoloLens apps and experiences
The HoloLens development tools includes the Unity HoloLens Technical Preview - a version of game design software that used for a lot of VR content as well as apps and games. This will not only power a lot of HoloLens apps, but should make creating prototypes a lot faster as a lot of studios will be used to developing projects using Unity. There's also a full SDK - and of course there's a HoloLens emulator so you don't need constant access to a device to develop for it.
HoloLens price and availability
The HoloLens is very expensive. Each HoloLens costs £2,719/US$3,000 right now for the Developer Edition, of which you can order up to five. If you want more, you'll need the Commercial Suite edition (£4,529/$5,000), which also lets you lock down each unit to a single app - which is what you'll need for the kind of theme park or brand experiences mentioned earlier. If you pre-order today, you'll receive unit/s in November.
It's early days for the HoloLens, but if you want to build the next generation of interactive experiences, a step beyond even VR, this is where it's at.