Quickly check your email, look up the weather forecast, or get the latest news in the morning – many people use web services daily to stay informed or communicate with others. The blind or visually impaired, however, face enormous challenges if they want to do the same. And the currently existing assistive devices are often impractical and expensive. Computer scientists are working to develop new instruments to make online services more accessible for the blind.
The male computer voice sounds robotic and monotonous. At a maddening speed, it reads out one word after the other with just a few short breaks in between. A torrent of words descends onto the user. Less experienced listeners will hardly be able to filter out the information they need.
A flood of information without navigation
But the blind and visually impaired depend on such text-to-voice programs for using the internet. A screen reader software reads out everything on the screen – text as well as structural information such as lists, input windows, logos, or links. While sighted people are able to identify all important information and structures at a glance and ignore the rest, the blind have to work their way through everything using their sense of hearing. “It is an almost endless torrent of information, without period or comma,” says professor of computer science Ulrike Lucke. Navigation aids such as spacing or font sizes are nonexistent. “We follow optical patterns. Blind people can’t,” Lucke explains. In addition, screen readers are unable to interpret pictures, diagrams, drawings, or videos. Thus, even just logging into one’s email account may turn into a bumpy ride, as the landing pages of major providers usually contain a lot of advertisements and entertainment.
Apart from the screen reader, there is another way for the blind to navigate the internet: by using their sense of touch. Special devices are connected to the user’s computer or laptop and translate web content into braille. Tiny, electronically controlled pins move up and down on a refreshable braille display. Each braille letter consists of a pattern of dots, and to enable users to read them correctly with their fingertips, they require a certain size. So this technology has its limits, too. “It’s like looking at the internet through a keyhole,” Lucke says. Users of a braille display also have to work their way through a webpage from one section to the next.
Using AI for orientation
Lucke and her team are striving to give blind people better tools for browsing the internet. To that end, the researchers first explored what web services were used most often. They found that music streaming, shopping, online banking, email, and social networks, as well as map services, and dating platforms were most frequently used. In total, the team is examining 40 groups of applications for blind users. First, they analyze the various functions of the sites and the controls that need to be found easily and quickly to make the service usable for the blind. Machine learning is used to identify such relevant fragments on the user interface.
Next, a machine algorithm is repeatedly confronted with common websites and trained a thousand times with the information displayed there. Gradually, the AI develops a reliable program for the quick and systematic detection of certain elements like the login mask. “This is where we are now,” explains Lucke. The enormous variety of information poses immense challenges for the researchers: “The internet is a huge haystack,” says Lucke. And the important interfaces and controls are sometimes well hidden within.
The objective, in the end, is to enable the blind to navigate the internet faster and more systematically with the help of the developed software. At the same time, the computer scientists adapt the program to a hardware that can present much more information than a line of braille letters. The centerpiece of the project is a somewhat clunky device resembling a keyboard that needs to be connected to a computer. Instead of alphabetic keys, the HyperBraille output device made by metec AG has many little pins on its surface.
As the pins move up and down, braille letters can be formed and read. The size of the tactile display allows the blind to read even graphs or diagrams with their fingers.
Eventually, the software developed by the Potsdam computer scientists and connected to the HyperBraille will connect via the web with AI every time a user connects to an online service. The AI will analyze and translate the website and leave out anything unimportant. Then, the actually relevant information will be played back and shown on the HyperBraille display.
More job opportunities for the blind
Devices like the HyperBraille make a real difference in the everyday and working lives of the blind – but unfortunately, they are also very expensive. “The current price is in the 5-digit range,” Lucke explains. At universities and schools for the blind and visually impaired, such devices are already available, but in private homes, they are the exception. Yet the researchers are convinced that the demand will increase in line with expanded options for use and that prices will drop. “When a blind person is able to work with spreadsheets, read diagrams, or format text, this will result in better job opportunities,” Lucke underlines.
The prototype of the new system is supposed to be operational by the end of the year. Then, industrial partner metec AG from Stuttgart will take over and develop it into a marketable product. Lucke hopes that it will be as popular with users as another program developed by her team recently. This allows the blind to explore maps on the internet – also based on a tactile braille display. The desired route is traced by moving pins. “It was very touching to see the first blind person testing the map,” Lucke remembers. “For the first time, she was able to put together a travel itinerary for herself on the internet without any help.”
The Project
In the project “TactileWeb”, researchers are developing software for a tactile braille display to give blind and visually impaired users faster and easier access to web services.
Duration: 2018–2019
Funding: Federal Ministry for Economic Affairs and Energy (BMWi) Participants: University of Potsdam and metec AG Stuttgart
The Researcher
Prof. Dr. Ulrike Lucke studied computer science at the University of Rostock. She has been Professor of Complex Multimedia Application Architectures at the University of Potsdam since 2010.
This text was published in the university magazine Portal Wissen - Two 2019 „Data“.