It may not be apparent to most, but both Apple and Google have included a number of assistive features in their operating systems to help people with vision problems.

Let’s take a look how they work.

Navigating a smartphone

Both companies have developed functions to help visually impaired people navigate their devices. Apple’s is called VoiceOver, while Android has TalkBack -- both read out what appears on the screen.

VoiceOver, which was first introduced to macOS and later made available on iOS, can tell the user information such as battery levels, or which app their finger is touching.

Communicating

The first step in sending a message or an email is to open an app -- something that is second nature to someone with full sight, but takes a little extra time for someone with impaired vision.

Fred Leung, an iPhone user and an advocate for visually impaired people in Hong Kong, explained; “For example if I want to open WhatsApp, I more or less know the app’s position. When I tap that area, the phone will tell me which app I’m touching -- but it won’t open that app unless I double tap.”

There are two ways to write a message.

The first is dictation, which types out what you say when tapping the microphone button on the keyboard. A visually impaired person can locate the button using VoiceOver or TalkBack.

“It’s actually very accurate, even in Chinese, which can be very difficult to differentiate between characters,” says Leung.

Then there’s typing, which also takes advantage of the screen-reading function. Simply place a finger over the keyboard and the phone will tell you which key you’re on, so you can double tap to select the right character.

Reading

Visually impaired people can also have their smartphones read out texts, emails, websites and e-books. Don’t like the voice you’re hearing? You can also adjust the dialect and speaking rate.

Taking photos

“Most people who aren’t VIP -- which is what I call visually impaired people -- don’t realize that we can even take photos,” said Leung.

Turns out, VoiceOver can describe to you what’s appearing in the phone camera.

“Even without a special app, the iPhone can tell me how many people are in the frame or if there is a face on the screen.”

What’s next?

As with other fields like self-driving cars, artificial intelligence is also promising new possibilities for visually impaired people.

Siri is already integrated with VoiceOver, which can describe what appears in a photo, like whether there’s a dog or a tree.

Microsoft recently launched its Seeing AI app -- a smartphone-based narrator that it says can describe what the smartphone is pointing at.

Need to identify a product at the supermarket? It can read the barcode and tell you what it is.

And it claims to go a step further.  

Facial recognition helps family find man missing for more than a year

For more insights into China tech, sign up for our tech newsletters, subscribe to our Inside China Tech podcast, and download the comprehensive 2019 China Internet Report. Also roam China Tech City, an award-winning interactive digital map at our sister site Abacus.