If you’re wondering when we will see autonomous vehicles traveling down our nation’s highways, don’t hold your breath. At least that’s the word from Dr. Phil Koopman, who has been tracking the development of autonomous vehicle technologies for more than 25 years.
An associate professor at Carnegie Mellon University’s (CMU) electrical and computer engineering department, Koopman is a leading expert on driverless vehicles and the safety systems they require. He has extensive experience in software safety and quality, and as a CMU faculty member, he teaches young engineers the software skills needed for mission-critical systems. He recently spoke to DC Velocity Group Editorial Director David Maloney about the prospects for autonomous vehicle technologies and the challenges they present.
Q: You’ve been working on autonomous vehicle technologies for more than 25 years. Why has it taken so long for these technologies to reach their potential?
A: I started getting involved back in the mid-1990s. I worked with the Carnegie Mellon University Nav Lab team, which hired me as the safety guy. Just before they took me on, they had gone coast to coast [with an autonomous vehicle], from Washington, D.C., to San Diego. It was 98% hands-off-the-wheel. Think about it: 98% hands-off-the-wheel, coast to coast in 1995! I got hired because after that trip, they were like, “You know, maybe we need a safety guy.”
And then, I think it was 1998, there was an automated highway system demo, where they closed off part of the freeway near San Diego and drove a bunch of platoon cars and a city bus down the road—no hands on the wheel on a closed stretch of interstate highway. Again, almost 25 years ago, right? So, the way I like to look at it is that we were 98% hands-off-the-wheel across the country in 1995, and ever since, we’ve been working on that last 2%.
Q: Why has it taken so long?
A: Well, the catch is the last 2% is really tough, and this is fundamental to the issues of this industry. You can have a vehicle that is good at the easy stuff—and that is certainly an impressive achievement—but we have been there since the 1990s in some sense. It is really that last 2% that’s tough because it is always something new. It is something you haven’t seen before. There is an infinite variety of weird stuff in the world, and handling it all turns out to be a lot harder than people want it to be.
Q: There are a lot of driver-assist technologies available on our cars today—lane departure, automatic braking. Even my Toyota can pretty much park itself. Are these merely steps toward autonomy?
A: They are a contribution. In reality, what was going on in the ’90s was more like that than full autonomy. It was automatic lane keeping and things that today we call driver-assistance. Those are important to have, but making those better and better doesn’t actually solve the autonomy problem. The reason you need a human operator in the driver-assistance vehicles is that the machine learning part is good at knowing what it knows, but it is really, really “brittle” at stuff that it hasn’t seen before. That’s the purpose of having a human driver—to deal with the stuff it hasn’t seen before.
Q: So, when will we realistically see autonomous trucks on our highways?
A: It is more a question of how than when. If you want to completely replace a truck driver, that is a long way in the distant future—and by the way, truck drivers do more than drive. I don’t have to tell your audience that. But even for just the driving part, it is a long way off if you don’t want to put any limitations on what is going on.
If you’re willing to do something like take one stretch of interstate highway, and every day somebody goes through and makes sure all the lane markers are there and there haven’t been any paint or oil spills to obscure the lane markers and there’s no big pile of sand and there hasn’t been a landslide and everything is perfect—if you’re willing to do that and maybe there is a guide vehicle that the automated trucks all follow in a conga line and the guide vehicle is responsible for making sure that if there’s an animal on the road, it gets scared off—if you’re willing to make those kinds of concessions, it could happen in the next few years. But I don’t see next year somebody just saying, “OK, here are a thousand trucks. Let ’er rip!” I don’t see that coming as soon as a lot of people are saying.
Q: Do you see that as the next step—where you’d have a lead vehicle with a driver that’s followed by a platoon of autonomous trucks?
A: I think that a guide vehicle makes a lot more sense than just having every truck do everything in the next year. But I don’t see anyone trying to commercialize that.
Q: Do you think there are going to be dedicated lanes for autonomous trucks—or even dedicated highways?
A: It is really hard to know how that is going to go. There is a tradeoff between how much infrastructure you want and how hard it is to get the vehicle to do everything a human driver would do. I would think it is completely reasonable to do things like have dedicated on/off ramps at logistics centers. Maybe there is an HOV lane. It is going to depend on the road. It is going to depend on conditions.
Another way to go is to have designated times of day—periods when traffic is light—for autonomous trucks to use the highway. The more you have the road to yourself, the easier it is to ensure safety.
Q: You and I live both live in Pittsburgh, where they’ve been testing autonomous vehicles on city streets for a number of years. But wouldn’t it be easier to test the technology on interstates and limited-access highways than in urban environments?
A: Well, we are going to see everything. Right now in Arizona, Waymo is in fact running robo-taxis without drivers in a very, very benign environment.
The thing about urban roads versus highways is not that one is easier; it is that the challenges are different. In urban environments, you have a lot of crazy stuff happening all the time. It can be a very chaotic environment, depending on where you’re driving, but the good news is that if you’re driving slowly enough, a lot of times you can just jam on the brakes and you’re fine. You just have to know when to jam them on.
If you are on a truck going 60, 70, or 80 miles an hour, however, jamming on the brakes isn’t a very attractive option because you’ve got a lot of weight and mass—and a lot of momentum. So, it is less about weird chaotic stuff, like pedestrians jumping off curbs in front of you, and much more about planning ahead. I would say intuitively it feels like the highways are easier, but it isn’t that easy; it is just that the problems are different.
Q: We have seen a lot of advances in machine learning and artificial intelligence over the last few years. How much will these technologies play into the development of autonomous vehicles?
A: Artificial intelligence means the stuff that is really hard to do. And every 10 years, its meaning changes because some things become easy, while the next thing is hard. People tend to use “AI” as a catchall term for all the new technology, but it doesn’t really mean anything.
Machine learning, on the other hand, is a very specific technique. In machine learning, you show the computer system a bunch of examples and it performs a statistical analysis. And then if it sees a new sample, it will compare that new sample with the original sample. So, if it sees a person, it doesn’t actually know it’s a person. It says, “You know, that thing looks a lot like all the other people I have seen before, so it must be a person.” And that is great.
If you train it on things that it has seen, it works great. But that is like 98% or 99%. If there is something it has never seen before, it not only struggles, but it doesn’t even know that it doesn’t know what’s going on.
For example, there was a case where the system was having trouble seeing people wearing yellow. It turned out that this system hadn’t been trained to recognize anyone in yellow, and so, if you were wearing yellow, you were basically camouflaged from the machine learning system, which is not so great if you’re directing traffic at a construction site or you’re a bicyclist in a yellow raincoat.
So, the instances where it makes weird, crazy, or stupid mistakes sometimes come as a real surprise to people, and that is why I was talking about the “long tail,” the rare things that you haven’t seen before. That is why everything is taking longer than everyone wants it to.
Q: In the logistics market, there are obvious advantages to using autonomous vehicles, such as helping to alleviate the truck driver shortage. There are other benefits as well—trucks can be spaced closer together, which could help with congestion on our highways, and driverless trucks might be able to operate for longer stretches of time if they’re exempt from the driver hours-of-service regulations. What are the main benefits that will help push this technology along in the next few years?
A: Well, let me go back to the jobs thing because that is so central. If someone is a truck driver today, I don’t think they should worry about losing their job before they’re ready to retire. This technology is going to take a long time to take hold.
And even if there are a thousand [autonomous] trucks on the roads in the next four or five years, that is just a drop in the bucket. It’s going to take a long time to scale this technology up to be able to go on roads that aren’t the easiest, most benign roads. So, the scare headlines about truckers being out of a job next year—that is just not going to happen. On the other hand, I think the prospect of finding some relief from the driver shortage is fantastic.
In terms of other things, all of the things you mentioned hinge on safety. Until we can get safety right, none of that good stuff is going to happen. And the industry is at a point where it is just now starting to really think hard about safety.
There is a saying we have in the computer world that the first 90% of the project takes the first 90% of the time. And the last 10% of the project takes the other 90% of the time. Ultimately, it boils down to, Can you really ensure these things are going to be at least as safe as a human driver? We are not at the point yet where we have an answer to that, so there is still some more work to be done.
