Research uses muscle activity to move virtual objects

Today’s smartphones and computers offer gestural interfaces where information arrives at users’ fingertips with a swipe of a hand. Still, researchers have found that most technology falls short in making people feel as if they’re interacting with virtual objects the same way they would with real objects.

But a team at UW–Madison says it has developed, for the first time, a way to move virtual objects in an immersive virtual reality environment through the use of muscle activity. In addition to making virtual reality more interactive and realistic, the research could have rehabilitation applications for people recovering from injuries or people living with specific disabilities.

“We’re trying to add the dimension of movement and touch to allow people to exert forces against things that are created in front of them with a projector and virtual reality goggles,” says Robert Radwin, a UW–Madison professor of industrial and systems engineering, biomedical engineering and discovery fellow at the Wisconsin Institute for Discovery (WID)’s Living Environments Laboratory (LEL). “What if we could use these virtual exertions as a way of rehabilitating people from an illness or an injury such as a stroke? What if we could alter people’s abilities for different tasks, making them weaker or stronger during certain exercises?”

In previous research, manipulating virtual objects has relied on wands, controllers and other devices external to the body. Though valuable, these devices are not driven by a person’s muscles the same way as when real objects are picked up or moved.

Radwin worked closely with the LEL to develop the software and process for a pilot study in which participants move virtual objects in the CAVE, a fully immersive six-sided room that projects 3-D environments on its walls. Kevin Ponto, an assistant professor of design studies in the School of Human Ecology, Karen Chen, a graduate student in the LEL, and Ross Tredinnick, systems programmer in the LEL, collaborated in the design of the project.

In the study, participants’ arms were hooked up to an electromyography (EMG) device that collects the electrical signals produced by muscles during physical activity. Situated outside of the CAVE, people lifted dumbbells of different weights while the EMG device recorded muscle activity to a nearby computer.

Then, participants did the same exercise inside the CAVE, wearing 3-D goggles, head and hand sensors, and the EMG device. Instead of lifting a real dumbbell, people “grabbed” and lifted a virtual dumbbell instead, stiffening their arms to lift the object.

Karen Chen says the preliminary results show that people can adapt their lifting behavior to a virtual reality environment using the same muscle groups used to lift real objects. The benefit, she adds, is being able to perform lifting in a controlled environment with a reduced risk of strain or dropping real objects.

With more experiments and refinement of the technology, the group may examine downsizing the technology to include a headset rather than an entire virtual reality room.

Ponto says that in addition to rehabilitation applications, the project can open doors to make virtual reality more intuitive, including creating the illusion of lifting objects that don’t actually exist.

Future research could examine how to better tie virtual reality and human muscle groups together so simulations are more personally tailored based on a person’s normal activity while sitting or standing.

—Marianne English