Seriously, it took me more time to think of a title for this post than it actually took to enrol and manage the Hololens. And this is what I came up with. Anyway, I digress.
A couple of weeks ago I was lucky to get my hands on a Microsoft Hololens Developer Kit device from our campus in Palo Alto. In case you weren’t aware, VMware has an amazing and incredibly talented team in our Research and Development area working on many emerging technologies with Augmented Reality being one of them. Using my contacts within the CTO Ambassador Program I had the opportunity to meet many of them in person and see what they’re working on, and as a result I was able to present our capabilities locally here in Australia at a Technology in Healthcare roadshow.
At this event I presented on how Blockchain, Machine Learning and Artificial Intelligence, Internet of Things, Virtual and Augmented Reality and Digital Twins will shape the future of Healthcare. After the presentation the VMware stand was inundated with clinicians and nurses right through to training coordinators trying out the Hololens and discussing how we can enable these kinds of devices now into organisations.
It doesn’t seem to be that well known or understood how VMware can do this, so I thought I’d do a write up and give a bit of an example of what we’re able to do.
Under the hood (look, I really tried for a pun here), the Microsoft Hololens runs Windows 10 and fits nicely into their Universal Windows Platform. What this means is that we can manage it just like a normal Windows 10 device (mostly). I say mostly, because it doesn’t run x86 applications. A key part of our Windows 10 Modern Management capability is through the Workspace ONE Intelligent Hub agent, which is a x86 application so we need to rely on the built in MDM capabilities of Windows 10. As always, the true list of capabilities for Windows 10 are available on the Microsoft CSP reference. This reference page also now has a dedicated list of CSPs which are supported on the Hololens so you’ll see what the crossover is.
Initially I set out just to have a medical application running on the Hololens for the demonstration, but I always like to tinker. What I found was pretty cool!
There was meant to be a remote control which would have made it a bit easier, but alas it had been lost somewhere along the way. So I had to do all the “typing” with the gestures. I could also use my voice with Cortana to launch the store and select things which worked OK, but trying to get Cortana to type wasn’t great. What I was able to do however is below, and I sincerely apologise for the shaky video:
Management:
This was pretty straight forward. All I needed was to go in to Accounts and then enrol into MDM management. This brought the usual prompts to add a Work Account and I was managed.
Configuration:
One of the coolest things (I think) I was able to do was push down a MobileSSO profile for Workspace ONE. What this allow me to do was deploy a certificate to the Hololens and SSO into my VMware Identity Manager portal and into the Workspace ONE App.
Applications:
Because it runs (a version of) Windows 10, I was able to run the Workspace ONE App as well as all the clinical related apps.
The other pretty cool thing was that I was able to load my Horizon Desktop using the UWP Horizon Client.
Single Sign On:
Because I could deploy a certificate to the Hololens, I could use MobileSSO with Workspace ONE UEM and VMware Identity Manager. I opened up the Microsoft Edge app and launched my vIDM environment. I could then open the Office365 portal and the Google Cloud Developer Console, all without needing to manually enter credentials.
Use Cases:
Once I showed this at the roadshow, the immediate interest was in using it for Clinical Training. The customer wanted to be able to provide access to AR applications to train nurses and clinicians etc. using targeted and specific applications.
My thoughts on some Future Use Cases for AR in Healthcare
Below are a few scenarios where I think Augmented Reality will take off in healthcare:
- Surgeons using Augmented Reality in theatre to get real time information from medical devices (O2 saturation, heart rate, blood pressure etc.) while performing an operation. They could also get real time information about exactly which body part they should be operating on reducing errors and risk (I’ve been told this happens a lot more than we like to think). Also, it would be possible Video Conference with another surgeon or clinician to get a second opinion or assistance.
- Nurses wearing AR devices to identify a patient via facial recognition. My son (2yo) was in hospital a few months back for surgery and ever hour, on the hour, a nurse would come in to take his observations causing him to wake up. This would also mean I would be woken up to answer his name and DOB if there was medication involved. Augmented Reality could remove this requirement leading to better patient experience and outcomes.
- Expanding on the previous scenario, AR can cut down on a lot of the manual processes as well. Currently when a nurse needs to administer medication of a certain schedule (think opioids etc), they need to have another nurse verify its the right dosage and interactions with existing medications etc. In the future, because of all the connected information, a nurse wearing smart glasses could walk up to the medication cabinet, it would highlight the correct medication and display the patient information and dosage. It would also be recorded to stop any potential loss or abuse. This nurse could also now visit the patient and administer this without the “what’s your name, what’s your D.O.B etc”
Where are VMware helping customers NOW?
I’m unable to give specifics the customers, I can talk about a couple of use cases.
- One of the most difficult (and costly if there’s a mistake) tasks in building an aircraft is the wiring harness. VMware working with this aircraft manufacturer have helped deploy a specific application from a 3rd party that allows the worker to easily and efficiently build the wiring loom. The worker sees a short diagram or video of the next wiring task and sees the colour and identifier of which cable is next. The glasses validates if it is the right cable using Artificial Intelligence and if not, prompts the user to re-do the task and may also show them a demonstration video of how it connects.
- In a warehouse, a forklift driver is using a VMware managed pair of smartglasses with picking information. It tells them where to safely drive, which shelve to drive to and will also show them which pallet to pick from. If the worker also needs to get an individual package from this pallet or crate, it will highlight the package and they can then scan the barcode to validate its collection.
There is a lot more to AR than just management, but without the complete end to end management, to application, to infrastructure platform, customers will end up with a bespoke and complex system with little integration. Within R&D at VMware we are working on many of the emerging technologies that underpin this and enable these use cases of the future, but with our strong history of virtualisation, device management and infrastructure abstraction are in a very strong position to deliver the best possible outcomes for any industry.
Leave a Reply