The team of engineers from the University of Glasgow, UK has developed a new way to create the sensation of physically interacting with holographic projections.
In a new paper titled ‘Pseudo-Hologram with Aerohaptic Feedback for Interactive Volumetric Displays’, published in the journal Advanced Intelligent Systems, the team describes how they have developed a new technique they call ‘aerohaptics’. The system pairs volumetric display technology with precisely controlled jets of air to create the sensation of touch on users’ hands, fingers and wrists.
The technique could form the basis of new ways to interact with virtual objects, advanced forms of teleconferencing, and even empower surgeons to perform procedures remotely.
The system, developed by the university’s Bendable Electronics and Sensing Technologies (BEST) research group, is based around a pseudo-holographic display which uses glass and mirrors to make a two-dimensional image appear to hover in space – a modern variation on a 19th-century illusion technique known as Pepper’s Ghost.
2D views of the final virtual 3D object are projected onto transparent screens placed at a 45° angle from the projection source. This arrangement creates the illusion of an object floating in mid-air.
The screens are arranged in a square-based pyramid structure, allowing the 2D images to be created from orthogonal views of the virtual 3D object. One of the sloping sides of the pyramid structure is modified to create openings at the two bottom corners. As the projected object appears to be floating inside the pyramid structure, the user can reach through the openings and interact with the volumetric image as if it were a real object.
Users can interact with the virtual object by using life-like gestures. The gesture recognition system pairs a commercial (Leap Motion) sensor to track users’ hand movements with a moveable air nozzle to direct airflow to their palms and fingertips, according to the researchers.
In the paper, the team offer an example of how they used the system to create a realistic sensation of bouncing a basketball. With a computer-generated 3D image of a basketball displayed in space, and the Leap Motion sensor tracking the movement and location of the user’s hands, the system varies the direction and force of the airflow to create aerohaptic feedback.
The feedback is also modulated based on the virtual surface of the basketball, allowing users to ‘feel’ the rounded shape of the ball as it rolls from their fingertips when they bounce it and the slap in their palm when it returns. Users can even ‘push’ the virtual ball with varying force and sense the resulting change in how a hard bounce or a soft bounce feels in their palm.
Prof Ravinder Dahiya of the University of Glasgow’s James Watt School of Engineering leads the BEST group, which developed the system.
He said: ‘haptic feedback and volumetric display technology has come a long way in recent years, bringing us closer to being able to convincingly interact with virtual objects. However, current haptic tech often still involves wearable or handheld peripherals, which add cost and complication and could be holding back widespread adoption of the technology’.
‘Aerohaptics creates a convincing sensation of physical interaction on users’ hands at a relatively low cost. We are already looking in to adding additional functionality to the system, such as adding temperature control to their airflow to deepen the sensation of interacting with hot or cool objects’.
‘We believe aerohaptics could form the basis for many new applications in the future, such as creating convincing, interactive 3D renderings of real people for teleconferences. It could help teach surgeons to perform tricky procedures in virtual spaces during their training, or even allow them to command robots to do the surgeries for real. We’re looking forward to exploring the possibilities as we continue to develop the system’.