Computer “Vision” for the Visually Impaired: I have a dream

Throughout my life I have never dealt with a visually impaired person, never had a friend who can’t see and never knew about their problems and how their life looks like. As a Computer Scientist i thought they definitely have enough support from technology since we are in 2015 and i used to see many graduation projects focused on helping blind people 10 years ago. Sadly, that was found to be untrue.

The Shock

This year I had my first experience with a Visually Impaired friend. She is a highly educated intellect. She is independent and can do everything by herself even cooking. To cut it short, she is more productive and valuable than many lazy guys out there with 2 useless working eyes.

As a curious entrepreneur and a post-grad student who was studying Computer Vision at that time; i was observing how she manages to do everything, what are the difficulties and what Apps and Tools she uses. I was always asking her and myself “How can technology help in your daily life”. One day we conducted an experiment to check whether Mobile Apps can help her navigate and cross the streets or not.

The overall conclusion of my experience was

  • Unfortunately, technology is not effective in helping them doing their daily basic tasks

Their Needs

As mentioned in the previous section i was always asking my friend about her expectations from technology and then try to map that to an actual technical solution. Below are some requirements i got from her in addition to other ideas and personal observations.

They need

  • To Recognize objects around them
    Case: If something fell on the ground, they can’t find it easily. Also they may forget the location of their belongings (the keys or example).

My Dream

My dream is an “effective personal assistant device” for blind people. A supplemental digital eye + a smart personal assistant which provide them all kinds of support they need in their daily life …

Let’s call it “Project S”

S is an Augmented Reality wearable device, specifically “Google glass” or a Google-glass-like device which doesn't make the blind person look weird. It is equipped with camera, headphones (plus bone conduction) and internet connection (WiFi and 4G).

Image for post
Image for post

S is also connected to a super fast small processing unit which can placed in a bag or attached on a belt. Smart Computer Vision Learning system is installed on that processing unit to handle, analyze and respond to sensory inputs(camera and mic). It is also needed for the device to work in offline-mode.

S can do the following functionalities

  • Recognize Faces and Objects using a “deep-learning system” — Convolutional Neural Networks (CNN) like GoogLeNet — The system can recognize faces and will ask for a name for each face to associate with so next time it can tell you who is in the room. The system is also capable of recognizing objects indoor and outdoor and is able to change modes when triggered by some objects.

Now it is time for some Non-functional requirements

  • This device shouldn't be commercial, it should be given for free or at least at production price. I believe organization interested in the welfare of blind people and philanthropic foundations such as Bill & Melinda Gates Foundation and Chan Zuckerberg Initiative can help in that

Conclusion

That was my dream. I am looking forward to that day when i can see it happening, a day where i can obtain an early version of that device and give it as a personal gift to my dear friend and see the effect of technology on her life.

So far i can think of 2 companies capable of creating such device: Google and Facebook.

Life is full of false hopes, let us give them something true, a thing we know for sure that it will work …

Bellow are some relevant projects, none of which managed to cover all the functionalities listed earlier. The nearest project in terms of vision and functionality is Horus (well done guys).

Projects

http://horus.tech/en
http://www.openshades.com
http://www.orcam.com
http://www.project-ray.com
https://www.seeingwithsound.com (UPDATE)
http://pages.iu.edu/~hsseth/ (UPDATE)
https://sites.google.com/site/younghoonleehome/projects/computer-vision-for-the-visually-impaired (UPDATE)
http://www.va-st.com (UPDATE)
http://www.wired.com/2016/01/2015-was-the-year-ai-finally-entered-the-everyday-world/ (UPDATE) — Dulight

https://www.youtube.com/watch?v=R2mC-NUAmMk (UPDATE)

Videos

https://www.youtube.com/watch?v=SrpDlGxPJJc https://www.youtube.com/watch?v=4v8Mm5JotrY https://www.youtube.com/watch?v=4UGpDAmiVPM
https://www.youtube.com/watch?v=nrWfLAh2T3k
https://www.youtube.com/watch?v=Q07oHm3zh04
https://www.youtube.com/watch?v=I0lmSYP7OcM (UPDATE)
https://www.youtube.com/watch?v=monzuLsqcRc (UPDATE) https://www.youtube.com/watch?v=Xe5RcJ1JY3c (UPDATE)

Apps

http://www.knfbreader.com

Other links

http://blind.tech

Written by

Freelance Consultant — Data & Product. Writing about Data, Entrepreneurship and Life. https://karim.ouda.net

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store