News: Microsoft Opens Access to Sensor Data on the HoloLens with 'Research Mode' Update

The addition of a new research mode for Microsoft HoloLens will enable researchers and developers to tap into a wider range of data collected by the device's sensors.

Available in the Windows 10 RS4 update for HoloLens, application code in Research Mode can access computer vision algorithms, such as simultaneous localization and mapping (SLAM) for spatial mapping and motion, as well as audio and video streams.

"Microsoft HoloLens is the world's first self-contained holographic computer," wrote Marc Pollefeys, partner director of science at Microsoft, in a blog post. "Remarkably, in Research Mode, available in the newest release of Windows 10 for HoloLens, it's also a potent computer vision research device."

The data is collected by the device's image sensors, as opposed to the standard video camera that applications usually access. It works by using a quartet of light-sensitive greyscale cameras to track visual features in front of the device and in its periphery. These sensors work in conjunction with the depth camera that uses infrared light to measure depth by the time-of-flight method more accurately than standard cameras (which are more sensitive to ambient light).

• Don't Miss: HoloLens Can Now Wirelessly Tap Rendering Power of Any Desktop PC via UWP Apps Built in Unity

"Researchers can still use the results of the built-in computer vision algorithms, but can now also choose to use the raw sensor data for their own algorithms," wrote Pollefeys. "This opens a wide range of new computer vision applications for HoloLens."

In addition to processing the data on the HoloLens, users can transfer the output wirelessly to a PC or cloud environment for more demanding data processing.

"For these applications, HoloLens' abilities to visualize results of the algorithms in the 3D world in front of the user can be a key advantage," wrote Pollefeys. "HoloLens sensing capabilities can also be very valuable for robotics where these can, for example, enable a robot to navigate its environment."

The new research mode, along with the Project Kinect for Azure sensor to be included in the next-generation HoloLens, will be demonstrated at the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) in Salt Lake City on Tuesday.

• Follow Next Reality on Facebook, Twitter, Instagram, YouTube, and Flipboard
• Sign up for our new Next Reality newsletter
• Follow WonderHowTo on Facebook, Twitter, Pinterest, and Flipboard