A Wish List of Windows Mixed Reality Features That Could Change Everything for the HoloLens

"Necessity, not novelty," is a phrase I use often when it comes to HoloLens development. It would be fair to call it my mantra, or mission statement, as I prototype and explore software creation on this new frontier of mixed reality.

"What kind of experiences will keep people using these devices until the batteries run dry?" This is a question that drives me constantly. As a software developer, those discoveries are the heart of the journey. And if they are not the end goal for some, they are one of the major milestones on that path.

As a developer that has bought in wholeheartedly to the Windows Mixed Reality platform and the HoloLens, I spend about 14–16 hours a day, 7 days a week, writing about, thinking about, or working with them on a constant stream of freelance and personal projects. Of course, I am constantly thinking about the user experience for my applications and how it works within the Windows Mixed Reality environment.

As a result of my constant hunt for smart holographic solutions, I have had many ideas on how to improve the overall experience — some realistic, some not so realistic (yet) — and I'm not the only one. So I put my head together with many of our readers from around the world in the development community, and together we came up with a wish list of sorts on how Microsoft could help us help them expand this ecosystem and make it even more engaging.

• 3D/live widgets: This one is a bit obvious since they were in the early HoloLens demos, but it's definitely the most requested. I am sure system performance is one of the factors that drove Microsoft to pull that from the Windows Mixed Reality system that we have at the moment. Which brings up the next point below.

A live widget as seen from Microsoft Build 2015.

• 3D notifications: I believe this one falls under the 3D widgets category, but having an area in a spatial map for notifications would make it easier to build up usage habits for the HoloLens enthusiast, developer, or general user.

• Better spatial mapping tools: This is another highly requested feature. Spatial mapping is amazing and adds so much to experiences. That said, it is highly inaccurate. That level of inaccuracy will be fine for the real world, but for the areas that we use our devices regularly in, having sharp corners around walls would make the occlusion illusion more immersive. One potential solution would be the ability to take the spatial map into an external program, edit it so that it has sharp corners and the correct spaces, and then import it back as the data for the HoloLens spatial mapping.

Rounded corners don't occlude well.

• The ability to set static versus dynamic areas: This is a continuation of the previous point. Being able to take the spatial map that is generated and determine which surfaces are walls and floors and what is a moveable object could make a real difference in experiences. Spatial understanding does a good job in helping find those things in a real-time sense. But if this work was done in a static way, could the sensors not be used for other purposes?

• Occlusion and proximity throttling: Obviously, if there are live widget-type experiences, they will take up CPU cycles. While these holograms do need to be updated, often their processes and updates could be turned down when the user is not close. Using a combination of occlusion and proximity (direct distance or a pathfinding system) seems an easy way to determine what programs get CPU priority. Well, easy in a relative sense.

• External persistence for holograms inside unity applications: The blending of Unity applications with the Windows Mixed Reality shell may be a pipe dream, but it is one that makes sense for developers. And maybe it falls on us to come up with some system to make that happen, but it would definitely require some cooperation with Microsoft's HoloLens engineering team to come to fruition.

• Multiple desktops: In a world that your entire home or office is now your desktop, the ability to switch modes seems like a very useful ability. As someone that works from home most of the time, and even more so the jack-of-all-trades type, I would definitely benefit from this idea. I use many different types of software depending on what my current project is, and I would love to change modes based on my needs without having to move all of my holograms around.

• Better interoperability between XAML and Direct 3D: There are many use-cases that come to the mind, of members of the developer community, for taking existing 2D applications and making them work in 3D.

• The locking of holograms to real world objects: Technically speaking, this one is above my head. I can see image recognition systems that exist that could do this while an object was in view. I imagine certain devices could have some sort of close-range tracking device. Regardless, it's an interesting suggestion from one of the community developers that could apply to many imaginable use-cases.

• Above all else, any kind of roadmap: We know Microsoft can't spell everything out in the competitive market we are in. It is like handing your competitors your playbook and then wondering why you can't score a goal. But there are thousands of us out here, wanting to know the hard work we are doing will come to something before long.

Of course, we see the work going into the HoloToolkit and the HoloLens Companion Kit and know that Microsoft is actively doing much to help out the development community. Neeraj Wadhwa, Angela Hillier, and many others on the Microsoft team are constantly communicating and helping the developers simplify, find, and fix solutions.

Special thanks to the HoloDeveloper Slack channel and the HoloLens Developer Facebook group — both highly active and vibrant development communities — for their input and ideas.

• Follow NextReality on Facebook, Twitter, and YouTube
• Follow WonderHowTo on Facebook, Twitter, Pinterest, and Google+

Cover image via Microsoft Build 2015