Meet 49-year-old Matt King, a member of Facebook Accessibility Team. King is blind and his dream is to make Facebook fully accessible to people with disabilities. Towards that goal, the social network is developing an artificial intelligence tool that can analyze photos and describe them out loud.
So far, the tool can describe backgrounds and colors, but it’s still working on recognizing the people in the picture – if there are any. But even so, King’s assessment of the tool is impressive: “From my perspective as a blind user, going from essentially zero percent satisfaction from a photo to somewhere in the neighborhood of half… is a huge jump.”
Facebook’s Accessibility Team is trying to adapt the world’s most popular social network so people with disabilities can also use it, including blind and deaf people or people without full use of their hands. The AI tool is currently in the prototype stage, but Facebook plans to eventually share it with the large public.
That would mean assisting almost 50,000 users who are now relying on Apple Voiceover – a popular text-to-speech system – to surf on Facebook, though the overall population of blind users is undoubtedly much larger.
Like social networks in general, Facebook relies heavily on visual cues, but thankfully there are tools like Apple Voiceover that can help visually impaired users to still connect with friends and colleagues over Facebook. Even without the help of Facebook’s experimental AI system, King is able to scroll through his news feed like anyone else.
He just tunes Apple Voiceover to read the posts at a rapid-fire pace, and he can start to understand what’s in a photo from decent captions or meta-data describing. But with the AI system on, backed up by the help of Shaomei Wu, an accessibility researcher, and other Facebook AI engineers, the experience is pushed significantly further.
Facebook’s AI tool can provide context based on nothing else but the photo itself. Jeff Wieland, the founder and director of Facebook’s accessibility team, said they are responsible to making all Facebook’s products usable by people with disabilities. “Long-term, we really want to get to the point where we’re building innovative technologies for people with disabilities,” he added.
Image Source: WERSM