Microsoft Nails ‘Mixed Reality’ With the HoloLens 2
The factory before me was nondescript, a cliche of bygone industry. I stepped closer. Soon, I was towering over it, like the folklorish Paul Bunyan. Reaching for the building, my hands penetrated its slightly translucent facade. I reflexively clenched my fists and started twisting my arms, causing the entire factory to turn on its axis. I lifted it to eye level and pulled it closer to my face, imagining the tiny workers terrified and clambering for the exits.
And then I turned my gaze to the tropical island floating over my shoulder.
None of these things were real, of course: just virtual reality streamed to my eyes through Microsoft’s new HoloLens 2 headset.
Introduced to the world more than three years ago. HoloLens is a self-contained, wearable Windows 10 computer that provides the user with an augmented or “mixed reality” experience. Though HoloLens has so far lived in the world of business, enterprise, and health care (where it’s used, for instance, to help visualize a patient’s MRI results right on top of his body), it has danced around the edges of consumer imagination. My most memorable experiences with the original Development Edition revolved around an interactive AR game where I battled alien machines crawling through the windows and walls of an upscale hotel room.
With HoloLens 2, Microsoft has continued focusing on enterprise, marrying AR with its cloud services through Azure Spatial Anchors and Azure Remote Rendering. HoloLens 2 may look sci-fi, but it’s about getting things done at work. To help that aim, Microsoft addressed a number of pain points from the original headset, radically altering the wear and fit, replacing critical components, vastly expanding the viewport, and fundamentally simplifying user interaction.
These are many of the details Microsoft shared when it unveiled HoloLens 2 at Mobile World Congress in Barcelona this February. But the full extent of what Microsoft’s accomplished wasn’t clear to me until last week, when I took the new headset for a brief yet fruitful test drive.
The original HoloLens put a fair amount of the technology in the front, which, if you didn’t adjust it correctly, would rest on the bridge of your nose. To lighten the load on the user’s face, Microsoft replaced some of the plastic elements on the front of the still mostly black and gray HoloLens 2 with carbon fiber. It’s now 1.24 pounds rather than 1.27 pounds. That difference may not sound like much, but it feels more pronounced because Microsoft radically redesigned the device body. Gone are the two separate “halos”: an inner one that hugged your cranium and the outer one that housed most of the key components. HoloLens 2 offers a single, unbroken headband with a low-profile adjustment knob on the back.
Microsoft also smartly shifted the components and battery for improved weight distribution. The result is a noticeably more comfortable device.
I placed HoloLens 2 over my glasses and reached behind my head to turn the knob and tighten the ring. Inside, fabric-covered foam rested comfortably against the back of my head and my forehead. In front of my eyes was the still-familiar though now more transparent HoloLens visor. It only took a few moments for me to find the sweet spot of pressure, comfort, and viewport visibility — a considerable improvement over the original headset.
While I did not get to unbox the device, I did notice how quickly we progressed through startup and calibration. Microsoft replaced the original HoloLens x86 CPU with a 64-bit ARM-based Qualcomm Snapdragon 850. Company representatives assured me that existing 32-bit apps would still work with HoloLens 2.
Calibration is a critical part of ensuring a good HoloLens 2 experience. Unlike the original headset, which tracked head position to intuit your gaze, HoloLens 2 uses eye-tracking technology. This means you no longer have to go through an inter-pupillary distance measurement routine that involved closing one eye, looking at your finger, and then doing the same with the other eye. This measurement ensured a decent 3D experience but had nothing to do with eye-tracking.
The translucent HoloLens 2 screen resolved into a blue box warning me that calibration was about to begin. I was told to focus, without moving my head, on each of nine red dots as they appeared on the virtual screen floating a few feet in front of my face. This took a few seconds. After the system told me calibration was complete, I noticed that a white outline appeared around each red dot as I looked at it: HoloLens 2 was really tracking my eyeballs.
The other thing I immediately noticed was the size of that virtual screen. Microsoft told me that it’s not only twice as large as that of the original headset but has a different and perhaps more useful 3:2 aspect ratio. (The old one was 16:9). The screen resize offers significantly more vertical view space than before. What hasn’t changed, though, is the resolution: It’s still 47 pixels per degree, but owing to the larger field of view, there are now 5 million points of light.
The larger viewpoint means I could simply shift my gaze to see objects in the mixed reality space — where virtual objects blend with the real world. No need to turn my head around.
Like the first HoloLens, HoloLens 2 is just as aware of your physical environment as you are. It uses a powerful time-of-flight sensor (the same one that’s inside the new Azure Kinect DK) to “see” the floor, walls, and furniture in 3D. As a result, when I first saw my tiny factory, it was resting on a hutch in the real world.
Those sensors, however, add a crucial new level of interactivity, one that makes the HoloLens 2 fundamentally more useful than most AR hardware that’s come before it.
HoloLens 1 could recognize simple gestures, like a finger bend or a pinch. These moves were so specific that I often forgot how to use them and was grateful for the included Bluetooth-based controller. HoloLens 2 needs no such a controller, though it continues to support Bluetooth peripherals. The sensors can recognize both hands and the position of all your fingers in real time.
When I grabbed that building, I did so without thinking about or being told how to interact with it. I just “gripped” the virtual structure with both hands and turned. Microsoft did explain to me that when a three-dimensional bounding box appeared around objects like the factory or a floating island, I could grab a corner with two fingers and pull to enlarge or shrink the AR object. This was also incredibly easy.
HoloLens 2 will also do a better job of capturing your mixed reality. It’s now fitted with an 8-megapixel, 1080p-capable, auto-focus camera. HoloLens 1 got by with a measly 3.2-megapixel fixed-focus camera.
After playing with the factory and moving around Microsoft’s floating island, I was directed to another wall where I found a crystal floating. I said “pop,” and the crystal exploded.
Microsoft explained that HoloLens 2 has a five-microphone array (HoloLens 1 had four), including two in the visor. The idea is that those working in harsh or noise-filled work environments won’t need to shout to be heard. Microsoft claims that I could say “pop” in a 90-decibel workspace (which is in the range of a motorcycle or a landing Boeing 737), and HoloLens 2 would still register my voice.
At another point, five crystals appeared before me. I was coached to look at each one without moving my head. As I did, each crystal spun. I then looked at one and said, “pop.” It popped, as did the others when I looked at them and spoke. Overall, HoloLens 2’s hearing capabilities are vastly improved over the original HoloLens, on which Microsoft’s digital assistant Cortana often struggled to understand what I was saying.
Finally, a larger crystal floated before me and, as I moved my hand toward it, a button appeared. I pressed the virtual button and the crystal shattered, transforming into an iridescent, blue hummingbird. I held my palm up and the bird fluttered over it. As I moved my hand around, the bird followed. It stopped when I closed my palm and moved my hand away.
There was, to the left of the bird, a window of text floating in space. It described the hummingbird. I read the text and, as I reached the bottom, it automatically scrolled up to reveal more. Years ago, Samsung tried to introduce similar gaze-tracking-based reading on its Galaxy S7 devices. It did not work nearly as well.
After wearing the headgear for roughly 15 minutes, Microsoft showed me how I could easily flip up the visor — another new feature — to see the real world. It’s a simple but canny update, one that should make the headgear more attractive in work environments where workers wouldn’t want to take off the headset (or integrated hard hat).
I was a little concerned that when I flipped the visor back down that it did not seem to snap back to the exact same position I had it in before, but Microsoft assured me that the gaze tracking would recognize me and my eyes with Windows Hello biometric authentication and plop me right back where I left off in the augmented world.
Microsoft wouldn’t specify battery life — an important consideration on the job site — but did tell me that it would be on par with the HoloLens 1, which was around two and a half hours.
I can’t recall a more intuitive and immersive augmented reality experience. Yes, I’ve held up my iPhone or iPad and gazed at richer AR imagery, but in those cases, I can’t put the screen down and reach out to engage with that virtual imagery in a natural manner.
HoloLens 2 is technology made obvious, and I can easily envision wearing a hardhat fitted with HoloLens 2 on a job site. Perhaps you engage it when it’s time to work on installing a critical portion of the HVAC system or even to fix a motorcycle you’ve never seen before. HoloLens 2 would have Azure Guides — augmented reality instructions that live in the cloud and download in realtime to guide you as you work on real-world projects — which might download instructions from your soon-to-be-retired HVAC foreman. It would recognize the system and know exactly where you’re pointing or what part of the HVAC system you’re touching so it can offer corrective guidance.
There are some hints that a system like Microsoft’s HoloLens 2 could someday make it to consumers (it contains support for the Epic Games Unreal Engine, for one thing), but at $3,500, HoloLens 2 is clearly positioned for business. All I can say is that I’ve never been so envious of construction work.
Update: An earlier version of this piece incorrectly stated the resolution of the HoloLens 2. It now offers 5 million points of light.
Microsoft Nails ‘Mixed Reality’ With the HoloLens 2
Research & References of Microsoft Nails ‘Mixed Reality’ With the HoloLens 2|A&C Accounting And Tax Services