Running Blind: How AI helped a blind runner finish his first marathon solo

Neither furry nor human to guide him

Fatmir Seremeti, a visually impaired athlete (aka blind) recently made headlines by achieving an incredible feat at the Stockholm Marathon. Despite the bustling crowd and the complex course, this athlete from Sweden (Fatmir) ran a half marathon without bumping into anyone nor did he stray off-course. His secret? Not a guide dog, or a human guide—he relied entirely on AI technology on his phone.

In an inspiring display of innovation and determination, Fatmir became the first blind person to complete a half marathon completely solo, with only his phone's AI guiding him along the way. The AI helped him stay on course, navigate obstacles, and achieve what seemed impossible

Only last week, my children and I had been debating if we should use AI tools like ChatGPT. I had argued that tools like ChatGPT numbed our brains and had the potential to make us lazier. Just a day after this dinner-table discussion, I came across this story of an AI tool guiding a blind person through a marathon!

Here was an AI tool that had helped a person get up from his couch and run alongside others on the roads of Stockholm. It had helped him feel more alive and independent! I had to revisit views on AI!

This week’s post on the Lighter Side highlights how technology can empower people and push boundaries of what’s achievable. Fatmir’s race is both a personal victory and a testament to the growing capabilities of AI. Hope you enjoy reading this story:)

Photo by Sage Friedman on Unsplash

Google’s Project Guideline

A few years ago, a visually impaired athlete person Thomas Panek felt frustrated. Every time he wanted to go outside—whether it was to the grocery store or just for a walk—he always needed help from someone, either a person or a guide dog. He didn’t like depending on others and started wondering if his phone could help him become more independent.

Panek decided to reach out to Google, a company known for its groundbreaking technology. He hoped they could create something that would help him and the 253 million people around the world who live with blindness or low vision.

Google listened to Thomas Panek and thought it was an interesting challenge to solve using its new AI tools. That’s how Project Guideline was born. The idea was to use artificial intelligence (AI) to guide a blind person through different environments. But here’s the catch: there had to be a painted line on the ground in front of them.

The AI was loaded onto their Android phone, and the phone’s camera would constantly watch for the painted line. Through a headset, it would tell the person if the line was to their left, right, or straight ahead, helping them stay on track.

Panek was the first to try this out, running a 5k all by himself in Central Park, New York, using only AI to guide him!

Panek finished running a few hundred metres in the park, stopped and began to weep. He had never experienced such independence for over 25 years. This felt like rebirth to him. Panek wiped his tears, picked himself up and finished his 5k successfully (without bumping into a tree or another human along the way).

“I cannot imagine this day has come. I am here all by myself, without a human volunteer or a furry friend to guide me.

This feels like real freedom.”

How the Artificial Intelligence works

Google’s Guideline system uses three devices / inputs

• a mobile device worn around the user’s waist (often it is a regular Android phone)

• a line on the running path / road marked with paint or tape, and

• bone conduction headphones.

The device worn by the runner takes pictures every few milliseconds using its camera and quickly classifies each image as either "guideline" or "not guideline." If the camera detects a “not guideline” image, the app immediately sends an audio alert to the blind runner, helping them adjust and stay on the line. If the runner strays too far from the line, the app might even tell them to stop and guide them back carefully. All of this happens through sound alerts sent to the runner’s headphones.

Panek’s 5k run in Central Park was the first step in helping blind people run independently in a controlled environment, like a park with a marked road. But Google wanted to push the challenge further—onto a crowded street full of other runners!

Google reached out to a competitive athlete in the Paralympics

When Google began searching for a visually impaired athlete to test their technology, Fatmir Seremeti’s name stood out. The tech team reached out to him, and Fatmir was thrilled at the chance to run in a competitive marathon—without relying on a guide dog or being tethered to a human. For the first time, he could smell the excitement of running free.

Who was Fatmir Seremeti?

Fatmir wasn’t born blind. As a teenager, he lost most of his vision because of a serious eye disease. Imagine being a teen, playing football with your friends, and then suddenly, you can barely see and can’t even leave your house on your own. Going to school wasn’t the same anymore either.

But Fatmir still loved sports, so he didn’t give up. Instead of football, he found a new sport called goalball, which is specially made for athletes who are visually impaired. Goalball became his way to keep doing what he loved—playing sports, even without his sight.

A lot like football, goalball is a game where the player throws the ball into a net which is guarded by your opponents.

Goalball played at the Tokyo Paralympics

Even though Fatmir was still playing goalball, he was afraid to walk in the nearby park or on the road, worried he might bump into someone. But things began to change when he got his first guide dog, an amazing pet named Comet. With Comet by his side, Fatmir slowly gained the confidence to move around safely and explore the world again.

“I vividly remember the feeling of receiving a guide dog for the first time. It felt almost like regaining my vision. Having Comet or a trusted person to hold on to as I live my life has become second nature to me at this point.”

Photo by Matt Seymour on Unsplash

Google and Fatmir joined hands

Fatmir teamed up with Google to prepare for the Stockholm Half Marathon, and their plan was both simple and groundbreaking. Fatmir wore a phone strapped to his chest with a harness. Unlike earlier tests that used a painted guideline on the road, this time the phone’s camera would track a guide runner, a few meters ahead of him.

As the camera took pictures, it sent audio signals to Fatmir, alerting him if he strayed too far left or right. The sound in his headphones would get louder if he moved off track and quieter as he corrected his direction, guiding him back towards the lead runner. This allowed Fatmir to run freely through the streets of Stockholm without needing to be physically connected to anyone—just guided by the AI technology.

Months before the marathon, Fatmir and Google’s tech team started testing the system through morning walks and jogs in Stockholm. They fine-tuned the technology after encountering a few stumbling blocks. Soon enough, Fatmir was running several kilometers every morning with the help of the AI tool. Like Thomas Panek before him, Fatmir experienced a sense of freedom he hadn’t felt in years—finally able to run without being tethered to his dog or a human guide.

The Final Run

The day of the Stockholm Half Marathon had arrived, filled with excited onlookers and loud cheers, just like any big race. This was the first time Google’s tech team was testing their AI tool in such a bustling environment. To ensure Fatmir's safety, they decided to have two backup ‘safety runners’ with him, ready to give him verbal directions only if absolutely necessary.

As Fatmir lined up at the start, he felt a mix of thrill and nervousness. His hands weren't tied to anyone—not even to his guide dog. When the starting horn went off, he joined hundreds of other runners, rushing through the lively streets of Stockholm. The audience cheered them on, and for the first time, Fatmir was running completely by himself in a large crowd. Tears of joy streamed down his face as he soaked in the experience of being part of the race on his own.

He ran on for a couple of hours, skillfully weaving through the crowd without bumping into anyone. Thanks to the AI guidance, he maintained an excellent pace, feeling confident and free without needing to slow down or worry about going off course. This was a moment he had dreamed of, and it was finally happening! When he crossed the finish line, he said

I was overwhelmed with pride and joy. I had not only completed the race, but I had done it feeling truly free.

Partly, it's a cool thing to be able to be the first blind person to finish a half-marathon without physical contact with a guide. But also to be part of research that hopefully can help other visually impaired people.

Fatmir Seremeti, once he crossed the finish line at the end of the marathon. Source: Google’s blog on Project Guideline

Recommended stories for you to read

1. The people who fought to make equal pay in tennis a reality. These women were a great combination of brilliance, fun and grit.

2. Chandrayaan-3: First anniversary. The story of how it all started.

3. Kings used to hunt wildlife. But there were some who protected lions too. Stories of maharajas who saved the Indian lions.

Would you like to join our team of Young Writers?

As we near the end of 2024, we are coming out with a printed magazine of ‘Lighter Side’ highlighting some of the most interesting news stories of 2024. If you would like to contribute to these stories, you can send in your written stories via email to me at hello@wsnt.in.

• Choose any news story you liked this year. Check out our podcasts on What’s new today, for interesting ideas. You can listen to them and rewrite the story in your own words. You are welcome to research and add more to them as well.

• Let your final work be no more than 300 words.

If you have any queries, write in to me at hello@wsnt.in Always happy to hear and respond:)