What's Under the 'Hood' of Self-Driving Cars? Headed by tech giants like Google and Yandex, the rise of automated vehicles seems inevitable, but what are the intellectual mechanics at work?

By Olga Megorskaya

Opinions expressed by Entrepreneur contributors are their own.

Waymo, a unit of Google's Alphabet Inc, and Yandex Self-Driving Group, a division of the Russian-based Yandex corporation, are among more than a dozen leading names in automated vehicle software and hardware. The former recently launched a pilot self-driving taxi program in San Francisco, the latter has been testing its automated cars worldwide for the past several years and has self-driving rovers on several college campuses in the U.S. Not to be outdone, the Chinese tech company Baidu Inc., along with the Toyota-backed Chinese self-driving startup, Pony.ai, is set to debut a 100-car fleet of paid driverless taxis in Beijing in 2022, and has preliminary plans to launch a similar program in California in the same year (see link below).

The main challenge to such vehicles' broader applicability, however, is training the AI that powers self-driving to the point of flawlessness.

It's all about data

Software and data engineers train the algorithms that lie at the core of artificial intelligence, and for self-driving cars, data relating to the roads and driving conditions on and around them is key for navigation. To do so, AI must first learn what this data means, and this is where "data labeling" plays a critical role. Labeling data (essentially attaching a meaning to it) is the first step in creating this kind of full-fledged AI. Such labels, or meanings, must be informative, discriminating and independent. They also need to be precise and correspond to reality (the ground truth), which is why data labeling is a human-based process — often a tedious and arduous one, requiring thousands of people to examine and annotate (to label a vehicle as either a truck or a car, for example, or to distinguish among traffic light colors).

Related: Chinese startup Pony.ai to offer driverless robotaxi service by 2022 in California

Imagine a self-driving car spotting a vehicle with a bicycle on top of it. It has detected an object, yes, but is it a bike or a car? Is it both? And most importantly, how should the system behave in response to it? This is where human help becomes indispensable, since only people can train computer vision models (the self-driving car's "brain") to properly detect complicated objects. They are also the ones fine-tuning AIs so that the latter understand different landscapes and avoid issues. Yandex self-driving cars, for instance, needed to label additional images from Las Vegas roads, such as traffic lights, which might be blinking yellow, unlike traffic lights in other countries. This way, Yandex helps its cars "wrap their head around" the town, one with little resemblance to Moscow, where they'd been originally "trained".

Data labeling types for self-driving cars include segmentation, 2D bounding boxes, lane marking, video tracking annotation, point annotation and 3D object recognition — each requiring careful treatment, as together they teach AI to understand what's happening on the road. The more often data is labeled accurately, the faster the AI will pick it up, produce patterns, and avoid generating a distorted picture, which impedes them from differentiating safe from unsafe situations.

Related: Elon Musk's Tesla Delays the Long-Awaited Cybertruck Until 2022

The labeling rush

Sudden high demand for data labeling gave rise to a host of new labeling services. Estimated to reach $8.22 billion by 2028, what's referred to as the "data annotation" market offers a number of services to scale, with the aim of making the process increasingly quick and affordable. They include Toloka, Scale, Mighty AI, Appen, Cloud Factory, Amazon's Mechanical Turk and many others. Yandex Self-Driving Group, for example, used Toloka to collect data in an expedited manner, saving multiple millions of dollars on its annotation. By now Yandex's driverless cars have traveled more than 10.5 million miles. Another such company is Scale, a San Francisco-based startup that has created significant buzz in Silicon Valley. In 2018, the company secured $18 million to label raw data from clients such as Lyft, General Motors, Zoox, Voyage, nuTonomy and Embark. Scale's mission is to improve built-in AIs by reviewing images, radar, and lidar data from cars alongside other sensor data to better identify objects on the road, including pedestrians and cyclists.

Related: Ways In Which Artificial Intelligence Will Empower Business

How mainstream will self-driving become?

Although self-driving technology is making enormous leaps forward, admittedly, it needs a lot more data to go mainstream. Despite launching driverless taxis in San Francisco and floating the idea of expanding to trucking, logistics and personal vehicles, Waymo is still putting a driver behind the steering wheel —a legal requirement as well as an acknowledgment that this technology is in the early stages of development. Yandex Self-Driving Group, meanwhile, is gearing up to launch an unmanned taxi program in Moscow this winter, making it possible to ride a robotic vehicle within certain areas.

Ultimately, much depends on building stable pipeline systems with automated quality control. If they become ubiquitous, this burgeoning industry could become commonplace in as few as ten years.

Wavy Line
Olga Megorskaya is the founder and CEO at Toloka AI, a global data labeling platform. Toloka was featured in Gartner’s Hype Cycle Report for Data Science and ML as one of the top data labeling solutions on the market.

Editor's Pick

A Leader's Most Powerful Tool Is Executive Capital. Here's What It Is — and How to Earn It.
One Man's Casual Side Hustle Became an International Phenomenon — And It's on Track to See $15 Million in Revenue This Year
3 Reasons to Keep Posting on LinkedIn, Even If Nobody Is Engaging With You
Why a Strong Chief Financial Officer Is Crucial for Your Franchise — and What to Look for When Hiring One

Related Topics

Growing a Business

My Startup Scored a Multimillion-Dollar Contract With a Fortune 100 Client in Just 3 Years. Here's What We Learned.

There's no perfect litmus test to gauge if you're ready to go after big business or not — but if you don't take the risk, you'll never realize the reward.


5 Questions to Ask a PR Pro Before Hiring Them

You probably haven't considered asking these questions, but they're a great way to find the right PR firm for your business.

Business News

The Virgin Islands Want to Serve Elon Musk a Subpoena, But They Can't Find Him

Government officials would like to talk to Tesla's owner as part of an investigation into the Jeffrey Epstein case.


This Location-Based Marketing Technique Is the Key to Boosting Retail Sales

Let's take an in-depth look at geofencing marketing and how it's helping retail locations drive foot traffic and boost sales.

Growing a Business

The Inevitable Challenges You'll Face as Your Business Grows — and How to Handle Them

There's going to be some discomfort as your business expands, but it doesn't have to stop you from achieving massive success.

Business News

'Just Say You Are Going Broke': Starbucks Slammed For Price Increase On Popular Item

The chain will start charging $1 extra for customization on its popular Refresher beverages.