Game Changer: Device Brings Interactivity to a New Level

Tomás Henriques, associate professor of music and director of the Digital Music Ensemble, has invented a device that could change the way humans interact with machines.

Called the Haptic Controller after the Greek word haptesthai, meaning “touch,” Henriques’ invention is a new type of video game controller that allows the user to manipulate 3-D objects. It looks like a seven-inch Slinky attached to two handles. Each handle houses sensors that independently track motion, orientation, force, and length as the user modifies the shape of the coil by twisting, compressing, expanding, or shaking it.

These actions generate control signals mapped to software algorithms that create and process auditory and visual data, or any other type of data, in real time. The signals are then sent wirelessly to a computer or real object, like a robot or radio-controlled plane. Heart rate and detection of changes in the stress level of an individual using the controller are currently being developed and will be added to the existing sensing mechanisms. Together they are set to play a relevant part in transforming the way we relate to and work with machines.

In terms of video gaming, the Haptic Controller offers a new type of play.

“Imagine you’re a sculptor modeling clay,” Henriques said. “Now imagine you have a 3-D shape on a computer screen. With both hands on the Haptic Controller, you can manipulate and sculpt the shape of that 3-D image on the screen.”

When using the Haptic Controller to move real objects in space in complex ways, the user feels the spring responding, providing force feedback. This provides a much more realistic experience of actually controlling something, currently absent from present controllers.

The Haptic Controller is the first device that brings together sensors providing force feedback and 3-D spatial manipulation, hand-, arm- and wrist-driven motions.

“We’re trying to get the attention of programmers using 3-D Unity—the fastest-growing video game programming platform—so they can create games for the prototype,” Henriques explained.

He also hopes to attract the interest of the major video game console manufacturers like Nintendo, Sony, and Microsoft.

Henriques received an award from the SUNY Technology Accelerator Fund (TAF) that will allow him to produce a set of advanced prototypes. To develop those prototypes, as well as a software development kit for 3-D Unity designers, Henriques needs to partner with highly skilled engineers.

 “I am an experienced programmer, but now it’s time to do the real thing. The TAF funding will allow me to do that,” he said.

While the TAF investment is targeted at the market potential of gaming applications, members of Buffalo’s medical community who were introduced to the technology are eager to get their hands on a prototype for conducting clinical testing in physical and cognitive therapies. Walter Brown, director of physical therapy at Buffalo Spine and Sports Medicine, thinks the Haptic Controller has potential applications for treating patients who have a significant fear of motion due to chronic pain. Pointing out that pain is a process that occurs in the brain, Brown explained that he uses a graded approach that exposes patients to motion implicitly (observing the activity) and explicitly (imagining themselves doing the activity) before having them actually do the activity.

The Haptic Controller could add a valuable virtual step to that progression.

 “It’s fascinating to think about a game controller that straddles the point of imagination of movement to actually doing the movement yourself,” said Brown, who is board-certified in orthopedics. He likes the fact that the device uses two hands, meaning it activates both the left and right sides of the cerebral cortex. 

“Anything fun and engaging can be very powerful with this population,” he said.

Even beyond these applications, the Haptic Controller’s significance may prove to be an existential game changer, as the video game paradigm is present in nearly all fields of human activity—from personal-, commercial-, and law enforcement-drone use and training simulations for pilots, ship’s captains, soldiers, and emergency response personnel to various surgical, social networking, and problem solving applications.