How AI is powering the next generation of robotaxis

Dmitri Dolgov has worked on self-driving cars for 20 years. But even he was surprised when a Waymo vehicle in San Francisco was able to spot a pedestrian hidden behind a bus and swerve to avoid them.

“I was like, ‘What’s going on here?’,” said Dolgov, Waymo’s co-CEO, on a recent podcast. “I know we have pretty darn good sensors, and the software is very capable, but we don’t see through stuff.”

The answer lies in a key change in how the world’s most advanced creator of robotaxis, which is owned by Google’s parent Alphabet, develops its autonomous driving systems. Robotaxis have in recent years benefited from broader advances in artificial intelligence, allowing autonomous vehicles to “generalise” their experiences to new cities and situations — and even to predict a pedestrian’s next steps.

This is how self-driving cars have finally been able to make the leap from small-scale tests to today’s rapid international expansion — and how new operators such as Canada’s Waabi and the UK’s Wayve are setting out to challenge Waymo’s lead.

After decades in development, robotaxis are now facing their biggest test yet as they start to appear on city streets all over the world.

How has the technology evolved?

Waymo began as a Google project in 2009. For the first few years its robot driver had to be taught the rules of the road one at a time. Engineers at Waymo and other early pioneers, such as China’s Baidu, largely had to “hard code” key safety and navigation directives.

These early robot vehicles travelled millions of miles on city streets, first to scan them to build high-definition 3D maps of their environment, then to learn how to detect and respond to changes in that environment. They gathered data using an expensive array of sensors: radar and lidar that bounce radio and light waves off their surroundings and multiple cameras pointing in every direction.

Graphic showing the three stages of evolution of driverless cars

The first step to broadening the robots’ horizons was simulation. All that sensor data was used to build virtual cities, inside which robot cars could be subjected to many more surprises than they encountered in the real world, helping solve many “edge cases” — unlikely but still safety-critical events — without the possibility of hurting anyone.

It took more than a decade and four generations of Waymo’s technology before its vehicles were ready to be deployed as a fully autonomous commercial service, beginning in suburban Chandler, Arizona in 2020.

How AI is transforming the latest models of self-driving cars

Silicon Valley is a test bed for a technology that is quickly becoming available across the world. At the push of a button, I'm going to hail a robotaxi. Within minutes, a vehicle will be dispatched from a local depot or rerouted from an earlier trip. What will arrive is a car that doesn't have a person behind the steering wheel. So this SUV has been transformed into something quite special. Waymo, which is owned by Google, has kitted it out with about 40 different sensors, cameras, lidar, radar. All of that's going to help it navigate public roads. Like the drivers it shares roads with, the AI model that is being used to operate the vehicle needs to make assumptions about the cars and pedestrians around it. When should it proceed at a junction, or switch lanes on the freeway. Besides obeying traffic rules, it has to deal with the unexpected, hitting the brakes when, for example, a child jumps out onto the road. These are all decisions you and I would make when we're driving around, but it's quite a different beast when a computer is making those judgments. The goal is to build the model driver, a system better than people. Robotaxis are reshaping the traditional ride-hailing market, where Uber and Lyft completely revolutionised hailing a cab, where robotaxi is changing how people live and work in a city. And now Waymo and other operators are trying to expand their services. Waymo operates in more than 10 cities across the US and has about 3,000 vehicles on the road. It has quickly emerged as the leading robotaxi player and set its sights on several key ride-hailing markets. London is on Waymo's agenda this year, with the UK government permitting trials in one of Europe's largest cities. The company is also testing in Japan's capital, Tokyo. Uber chief executive Dara Khosrowshahi has hailed the $1 trillion opportunity tied to robotaxis. He's investing billions of dollars buying up fleets of robotaxis to operate on the network. He's also investing in companies like Nuro, Waabi, and Wayve, all of whom want to enter the robotaxi market. But one operator is building something slightly more unique. Here in California, Zoox is developing what it calls a horseless carriage. Founded in 2014, the Amazon-owned robotaxi developer has developed a bespoke vehicle to run on its own ride-hailing network. The company started to roll out services in Las Vegas last year and has been testing services in San Francisco with a small selection of customers. Zoox is hoping that this toaster-shaped vehicle is the best representation of the end state for robotaxis. Jesse Levinson is one. Zoox's co-founder and its chief technology officer, he's been tinkering with autonomous vehicles for the better part of two decades. So Zoox is a custom vehicle that was designed from the ground up just to be a robotaxi. So we had an unusual idea way back in 2014. That if you had autonomous technology, you could build a better type of a vehicle that was designed to move people around cities without all the traditional manual controls and architecture of a car that was designed to be driven by humans. When you start from scratch and design a vehicle that's a different shape from any other vehicle that's ever been made, you have some opportunities and you have some challenges, right? We do have some additional expenses that a retrofitted car doesn't. The retrofitted car also has a lot of expenses that we don't have. For example, batteries, a retrofitted car is not going to be able to drive all day and all night on a single charge as a robotaxi. What that means is if you have a retrofitted car, you're gonna have to be taking it to your depot, and you're gonna have to be charging it in the middle of the day. And that's time that you could have been carrying, paying customers if you had a big enough battery, but instead, you're wasting time driving to the depot and you're using charging infrastructure and charging during, you know, peak energy hours. So, there's a lot of these sort of second-order effects that might not be as obvious that actually are meaningful advantages for Zoox. Zoox, like Waymo, has its own bespoke app, but the vehicle that customers hail is quite a bit different. The Zoox vehicle that arrives is about the size of a compact car, unlike a more conventional road vehicle, it can move in either direction, that allows it to navigate tricky urban environments. Inside the cabin, riders experience something that's quite spacious. They sit side by side with their friends and also across from them, allowing them to easily communicate and engage socially. To the side of each rider is a control panel. So here you can control things like the temperature of the Zoox, or set your own music. Because the Zoox vehicle is designed very differently to a conventional road passenger vehicle, the airbag system within it is also quite different. So rather than striking passengers from the front, the airbags deploy from around the passengers, engulfing them instead. Robotaxis like this one will still face some hurdles before they're widely adopted, with questions from regulators about their safety. Bryant Walker Smith is a leading expert on the law and policy of emerging transportation technologies, particularly autonomous vehicles. I mean, the first question really is sort of how are regulators responding to the technology as it's being deployed, particularly here in the US at the moment. Regulators have taken a variety of approaches. Some of it is preparing the groundwork, trying to think through the frameworks. Are our existing frameworks OK? Because in some ways they might be. Do we need new frameworks? This is what the UK has emphasised. And then a lot of it, particularly in places where automated vehicles are deployed, has been A much more responsive, interactive approach. Now, in some ways, we can criticise that as regulatory whack a mole, right? Problem comes up, you try to hit it. Problem comes up, you try to hit it. Uh, in other ways, that's kind of the, the flexible approach that allows regulators to learn in real-time and act in at least near real-time. Operators have self-reported hundreds of incidents with NHTSA, the US Safety regulator. This includes incidents where the vehicle has struck another passenger car and even cyclists. A number of these incidents would have taken place with a driver behind the wheel, but robotaxi operators are promising better than human capabilities. This does appear to be the case so far. NHTSA estimates that there's roughly 1 fatality for every 100 million miles traveled with a person behind the wheel. In a safety report published in late 2025, Waymo said that it had carried passengers for more than 127 million miles with no fatal crashes. The group's own analysis of driving data to date estimates a 90% reduction in incidents leading to serious injury. Despite a robust safety record, each accident risks undermining an operator. Waymo is a company that is getting to the point of having statistically significant mileage, where they can start to make pretty grounded comparisons, at least between injuries. As between their fleet and some hypothetical comparable human-driven fleet. There we can start to say that, specific to Waymo, not automated driving everywhere, that in the environments in which Waymo is operating, there seems to be encouraged. emerging evidence that the kinds of human crashes that we see are less prevalent or less severe with Weymouth. There are a lot of caveats there. We don't have statistically significant information, for example, about fatalities. And if a robotaxi were tomorrow to go through a school bus stop sign and hit a dozen kids, all of the data collected over the last few years would be, would be irrelevant. Takira Muakana, Waymo's co-chief executive, has said that she expects society will accept a fatal crash caused by a robotaxi. Speaking at an event in late 2025, she said that the challenge was ensuring that the bar for safety was high enough for the sector to avoid incidents that erode trust in the entire market. As we go into new markets, we really take our, our time and care to introduce our vehicle and our service to the local community. We think that's really important. Uh, I think it's important for robotaxis in general, maybe especially important for Zooks cause our vehicles do look different and they stand out. We'll have pop-ups with local businesses, let people sit in it, and we have a lot of material online to familiarize people with the Zooks concept, including a lot of information about our safety case, how we do testing and validation. It's definitely a journey, right? Even if the regulatory framework today does not allow this technology in every single major city, we believe over the next several years, as the technology demonstrates not only its safety, but it's many, many benefits to communities, uh, that the states and cities where it's not yet allowed, uh, we are optimistic they will change their mind. When these services do eventually arrive in cities, consumers are going to have to decide between driverless cars. And hailing more conventional vehicles. By some estimates, riders are paying roughly 33% more to ride in a robotaxi in San Fran Francisco than an Uber with a driver behind the wheel. Ultimately, what will shape the adoption of these vehicles is whether customers can access them and afford to use them. In Silicon Valley, a place where trillions of dollars are spent on moonshot bets, well, they've fast become an easy way to get home.

How AI is transforming the latest models of self-driving cars © FT

The challenge of proving the technology has also meant stop-start progress from other operators. GM’s Cruise closed its project following a mishandled pedestrian dragging incident in 2023.

Waymo has only started to significantly scale services in the past year. Rivals such as Amazon’s Zoox have tentatively launched services, while smaller developers like Wayve and Waabi are still testing on public roads.

While Baidu and Waymo’s robotaxis still look like conventional cars — albeit with a bigger roof rack loaded with sensors — Zoox, founded in 2014, has gone a step further. Rather than retrofitting a traditional car, it has built its toaster shaped robotaxi, which has no steering wheel, from the ground up. The company is awaiting regulatory approval of its unconventional design before it can start charging for rides.

© Graphic: Ian Bott Photo: Dreamstime Sources: Waymo; FT research

How robotaxis find their way around

A typical driverless taxi, as in this Waymo example, uses three main types of sensor to build a 3D picture of the environment around it

Lidar

Transmits millions of laser pulses per second, the direction and distance of the reflected pulses builds a detailed virtual 360 degree 3D model

Radar

Similar to Lidar but using electromagnetic waves to detect presence and movement. Particularly useful when visibility is low, eg in rain, fog or snow

Optical cameras

Useful for detecting and differentiating the colour of things such as traffic lights, construction zones and emergency vehicles

An inertial measurement unit fixes location by recording direction and distance from a specific reference point. Other sensors detect elements such as emergency vehicle sirens

Software processes sensor data into a real-time view and uses a database of existing information supplemented by AI to predict behaviour of surrounding vehicles and people to plan its own movement

How does AI bring everything forward?

The “big jump” for Waymo came with its fifth-generation Driver, introduced in 2020, said Dolgov. This is the latest version of the integrated hardware and software system that underpins the company’s autonomous vehicles.

The fifth version runs primarily on the Jaguar I-Pace vehicles that are already ubiquitous in San Francisco and are now starting to appear in London, where Waymo began autonomous testing this month.

Generation five “was when we made this big bet on AI”, Dolgov told Stripe’s Cheeky Pint podcast. Whereas the fourth generation system used some AI models and machine models, “we made a much bigger bet and jump to AI as the backbone for the fifth generation”.

The car that stunned Dolgov by seeming to see right through the bus had been able to detect and predict the pedestrian’s next steps even though they were largely hidden by the vehicle.

By inferring just enough movement from the person’s feet underneath the bus it predicted that they were about to step out around it. “It just blew my mind,” Dolgov said.

Similar to the large AI models that sit behind OpenAI’s ChatGPT or Anthropic’s Claude, Waymo has its own foundation model that underpins three separate systems to build a safe autonomous car: the “driver” itself; a “simulator” for virtual testing; and a “critic” that rates the driver’s performance.

Combined with a constant stream of new data from Waymo’s vehicles on the road, this creates a feedback loop to enable constant improvement.

A visual language model, in Waymo’s case trained using Google’s Gemini AI system, sees and interprets the road ahead. A decoder system takes these inputs to predict how other road users might respond and plot the best way forward.

Graphic showing how the systems that operate driverless cars could move to a more sophisticated system

How are advances in AI shaking up the competition?

The combination of Waymo’s early start and Google backing has propelled it to the forefront of the robotaxi industry, driving more than 200mn miles fully autonomously with passengers.

However, it is now facing competition from new entrants in not only Silicon Valley but the UK, China and beyond. Many are betting that starting with a clean slate in the new era of AI will allow them to scale more quickly and cheaply than Waymo.

Tesla and Wayve believe this AI-first approach can be taken much further, and operators differ on what technology they believe is required for a safe and viable vehicle. While Waymo uses an expensive array of custom hardware, its rivals’ systems use fewer cameras and radar sensors — and with some even potentially doing away with lidar altogether, claiming it is not necessary — to make a much cheaper self-driving car.

Alex Kendall, chief executive of Wayve, which was founded in 2017, claims his company was the first to use “end to end deep learning” to teach its cars to drive themselves simply by watching how human drivers behave behind the wheel.

Kendall describes the Wayve system as a “general purpose foundational model for driving”, meaning its cars can be deployed in a wide variety of locations and environments that they haven’t seen before.

“We can generalise and scale anywhere,” he says, meaning its vehicles are not restricted to a set “geo fenced” area like Waymos are today. However, despite high-profile backing from chipmaker Nvidia and the ride-hailing app Uber, its cars have not yet been tested at scale with the general public so it remains to be seen how successful its approach will be.

What’s next for the rollout of robotaxis?

Waymo already operates in ten US cities and plans to deliver 1mn paid weekly rides across 17 cities including London by the end of this year. It is forecast to account for more than 7 per cent of the overall US rideshare market by 2030, according to JPMorgan.

Uber is targeting 15 cities globally with a host of robotaxi operators and plans to compete directly with Waymo in San Francisco with a fleet of vehicles from carmaker Lucid and AV developer Nuro. Zoox and Tesla have also laid out an ambitious roadmap for expansion.

Chinese operators are rolling out fleets of robotaxis in Asia, having deployed limited domestic services across cities such as Beijing and Shanghai. Guangzhou-based WeRide in March expanded services into Singapore. Waymo meanwhile is testing vehicles in Tokyo.

Jesse Levinson, co-founder and chief technology officer of Zoox, says that while a bespoke vehicle has higher upfront costs than a retrofitted production car, it enables the company to incorporate larger batteries that ensure the vehicle could run “most of the day and most of the night”. This enables the vehicle to also operate for more hours each day than a traditional cab, helping offset costs.

“These things actually make more of a difference than the actual bill of materials,” he says. “It just makes sense to have a design that was specifically created for the purpose it was [intended].”

Technology companies and investors are spending tens of billions of dollars on robotaxis in a race to deploy them on public roads and gobble up market share in the ride-hailing market. Uber alone has committed the group to more than $10bn in AV investments and vehicle commitments.

“The same thing that’s happening in Generative AI is happening in autonomous vehicles as well,” Uber chief Dara Khosrowshahi told investors last year. “We estimate that the US market alone is $1tn opportunity.”