pixel

Independence in an app

The Intersection
AI helps narrate daily life for blind and low-vision people
Audio Described

The power of inclusive design

Saqib Shaikh lost his sight when he was 7 years old. During this time, he was introduced to talking computers at school and would often imagine how great it would be if he had something he could wear that would describe the world around him, like a tiny friend on his shoulder whispering in his ear. As an adult, Saqib held onto his dream and created an app called Seeing AI that does just that. Developed for a 2015 Hackathon, Seeing AI helps blind and low-vision people navigate their daily lives with AI.

Through a smart phone camera, Seeing AI can read text, scan barcodes, describe scenes, identify faces, and more. The app can dig into the details of the world around you and paint a vibrant picture that brings you into the moment, and it continues to evolve through community feedback and advances in AI research.

Saqib and Trevor hear detailed descriptions of what’s surrounding their campus shoot.

More stories

“The image depicts a film or video production set outdoors. In the foreground, a man is holding a camera mounted on a stabilizer.”
A person filming with a camera set up outdoors, viewed through the screen of a mobile phone. A film crew and production equipment are visible in the background.
“There are several trees with green foliage, suggesting a park-like setting. Behind the trees a modern building with reflective glass windows is visible.”
A behind-the-scenes look at a film set, with crew members preparing camera equipment and lighting. A smartphone frame overlays the scene, highlighting specific elements.
”There are some outdoor structures, such as a gazebo or canopy and various pieces of equipment related to the film or video production set.”
A smartphone screen shows a film production scene outside, with a camera, lighting equipment, and crew members. An orange circle highlights a person in the background.
  • “The image depicts a film or video production set outdoors. In the foreground, a man is holding a camera mounted on a stabilizer.”
    A person filming with a camera set up outdoors, viewed through the screen of a mobile phone. A film crew and production equipment are visible in the background.
  • “There are several trees with green foliage, suggesting a park-like setting. Behind the trees a modern building with reflective glass windows is visible.”
    A behind-the-scenes look at a film set, with crew members preparing camera equipment and lighting. A smartphone frame overlays the scene, highlighting specific elements.
  • ”There are some outdoor structures, such as a gazebo or canopy and various pieces of equipment related to the film or video production set.”
    A smartphone screen shows a film production scene outside, with a camera, lighting equipment, and crew members. An orange circle highlights a person in the background.
If you design for people who have the greatest need, that same technology can make its way into the mainstream and help everyone.
Saqib Shaikh
Founder and Lead, Microsoft Seeing AI

Hack the future

In early 2014, Anirudh Koul, a data scientist working with machine learning and natural language processing in Bing, realized the need for an app that could help blind and low-vision users navigate their daily lives. Over a year later, the building blocks came together, allowing Anirudh to recruit people to join his project, Deep Vision, for the 2015 Hackathon. The team competed with 13,000 other participants worldwide and won several awards.

Saqib Shaikh was one of the sponsors of Deep Vision who helped create and release the Seeing AI app for iOS on July 12, 2017. For Saqib, it’s about taking that far-off dream and building it one step at a time.

Watch more about how the 2016 prototype came to life.

Audio Described

Discover more episodes

TrueMedia is using AI to fight the spread of misinformation by helping people identify deepfakes with an app.

IHME helps the world’s forgotten people by mapping where they live and elevating their health resilience with AI.

SEEDS is dedicated to saving communities in India from global hazards by using AI to identify those most at risk.

Radiologists are detecting breast cancer sooner and with more accuracy using AI, giving patients better peace of mind.

QuitBot and AI conversational agents are helping people quit smoking by encouraging behavioral change.

AI is helping doctors detect pancreatic cancer, one of the deadliest known to humanity, earlier than ever before.

Project Guacamaya uses AI in different ways to help preserve the Amazon Rainforest by monitoring its health.

Trevor Noah

Trevor Noah is one of South Africa’s top stand-up comedians. He has toured in the US and internationally and made appearances on The Tonight Show with Jay Leno, and The Late Show with David Letterman. When Jon Stewart announced his departure from The Daily Show With Jon Stewart in 2015, Trevor was named as his replacement and hosted the show until December 2022. Trevor is Microsoft’s Chief Questions Officer, bringing his curiosity and humor to the role as he explores the future of nonprofits making an impact with AI.
A person with a striped sweater stands in a modern building with staircases and glass-walled rooms in the background.

Related stories