Navigation by Vibration, Rather Than Sight
Electrical and computer engineering professor Wolfgang Fink has spent many years of his career working on computerized vision systems to help autonomous robots see and navigate. For the past few years, he’s worked with a group of students to develop an improved way for visually impaired people to do the same thing.
Fink connected with undergraduate students Kristena Kay, Nick Powers and Bassil Ramadan when he gave a talk to the Department of Neuroscience in 2018. Ramadan, at the time a sophomore in the department, liked Fink’s interdisciplinary approach to his work, and he and his cohort were looking for research experience. The group joined Fink’s Visual and Autonomous Exploration Systems Research Laboratory and started meeting weekly to brainstorm, joined by Andres Nuncio Zuniga, a biomedical engineering doctoral student in the lab.
They learned more about Fink’s past work, such as his role in the development of the only FDA-approved visual prosthesis to date: the ARGUS II by Second Sight Medical Products, Inc. The prospect of working with visually impaired populations sparked the students’ interest, and they came up with an idea for a sensor-equipped belt to help visually impaired people navigate.
One of the first things they did was speak with Sunggye Hong, an associate professor in the Department of Disability and Psychoeducational Studies, who is blind.
“You’re not supposed to design in the absence of the end user, but it happens all too often in engineering,” Fink said. “We wanted to incorporate the end user right from the beginning.”
Many well-intentioned researchers have developed technologies that sound useful in theory, at least to sighted people, but they aren’t practical or desirable for their visually impaired audience. For example, some have proposed attaching devices to traditional white canes that offer haptic feedback, but these devices are sometimes expensive, and obstruct the cane’s basic functions. Others intended to replace the white cane with radar or sonar technology but were criticized for requiring complex training and interfering with natural echo-location cues.
“I found this experience to be very rewarding, because I could provide feedback on a potential product which has great possibilities for making differences in the lives of people with visual impairments,” Hong said. “My comments and suggestions could help make the final product both accessible and usable for end users.”
Good Vibrations: From Initial Idea to Paper Publication
The white cane is popular in part because it helps users detect objects on the ground. It’s also an effective way for others to identify and accordingly accommodate visually impaired people. However, it is doesn’t help users detect overhead obstacles, so Fink and his students decided to create a supplement to do just that.
“Ultimately what we did is develop an assistive device that is by no means replacing the cane, but is widening the experience of the blind person,” Fink said.
The device, which the team named Visual Impairment Subtle Touch Aid, or VISTA, consists of two belts. One is equipped with sensors that detect obstacles around the user, inspired by the radar sensors on the backs of cars. The other belt warns the person when an obstacle is near by vibrating. The closer the person gets to the obstacle, the faster the vibrations become.
Once they had a prototype of the device, the group first tested it themselves, wearing blindfolds. Then they made adjustments. For example, they’d originally designed it so that, the closer the user is to an obstacle, the stronger the vibrations would become. But once they were wearing VISTA themselves, they realized the difference between stronger and weaker vibrations was difficult to feel, so they changed the vibration frequency instead.
Then Hong tested the device and provided more feedback, including making the belts out of rubberized material and making them more adjustable.
“I receive requests like this one once in a while,” Hong said. “However, this was especially meaningful to me, as orientation and mobility—skills that an individual with visual impairments must have to live independently—are so critical for people with visual impairments. In addition, using engineering and technology efforts to provide these solutions seemed way exciting.”
The team went on to publish a paper about VISTA in the Journal of Medical Engineering & Technology.
“We went the whole nine yards, from the initial innocent talk, all the way to undergraduate student research, to a conference presentation and poster and a peer-reviewed journal publication,” Fink said.
Fink recalls a particular night when he and Ramadan spent five hours on a Zoom call, preparing their rebuttals to comments from the reviewers. It was a taste of the nitty gritty work of publishing research results in a journal article that undergraduates don’t often get.
The undergraduates involved in the work have since graduated. Ramadan, the first author of the paper, said the experience was a valuable chance to learn more about engineering and deepen his research experience. Next, he plans to earn his PhD in biomedical engineering, having just submitted applications to Johns Hopkins and the University of Arizona.
“I think one of the best things you can do in STEM is have a multiyear experience, where you go from project conception to publication, and even beyond to branching projects,” Ramadan said. “When it became increasingly likely that a publication would come out of this work, I was thrilled.”