Road warriors: robots get smarter, but who will buy them?

The concept of a car driving itself through city sweets is no longer relegated to the realm of science fiction.

Automobiles competing in the Defense Advanced Research Projects Agency's Urban Challenge in November proved themselves capable of navigating through a closed course in California without any humans sitting behind the wheel or remotely controlling the vehicles. Loaded with sensors and computing technologies, the cars, sports utility vehicles and trucks dodged obstacles, pulled into parking spots and merged into moving traffic with calculated precision.

While the technologies to enable fully autonomous vehicles have advanced, robotics experts say there is still more to be done to make them viable in military and commercial applications in the next decade.

"It was a great demonstration of what's possible. But we still have a lot of hard work in terms of engineering and product development in front of us to make it broadly available and broadly feasible," says Bill Thomasmeyer, president of the Pittsburgh-based National Center for Defense Robotics, a non-profit organization.

To accelerate the development of robotics technology, DARPA in 2004 sponsored a contest for unmanned vehicles in the California desert. Fifteen teams attempted the 142-mile course, but none of the competitors came close to finishing the race. In 2005, four vehicles completed the second challenge, running through the 132-mile Nevada desert course in less than 10 hours.

It was about that time that roadside bombs in Iraq began taking a heavy toll on the U.S. military. Suddenly, the need for unmanned ground systems in an urban environment became more urgent.

Few robotic systems were available because of the perceived danger of operating unmanned vehicles in populous areas. Also, technologies were considered difficult to develop and to test.

DARPA soon after announced the Urban Challenge--a robotics race similar to the grand challenges with a decidedly different twist: this time, vehicles would have to navigate 60 miles through a city-like course and contend with moving traffic composed of stunt drivers in other vehicles.

"We started two years ago with the idea that the use of robots in an urban area was so far out that we really needed to create believers among the community," says Norman Whitaker, program manager of the DARPA Urban Challenge.

In the previous contests, teams had demonstrated that the sensor technologies were readily available to help vehicles "see" their desert surroundings to navigate autonomously through the course. But to accomplish the same feat on paved roadways with curbs, lanes, stop signs and oncoming traffic would require more sensors and sophisticated computer networks to process and interpret the data.

"We wanted this to be a software race," says Whitaker.

The participants relied on commercially available sensors, including cameras, lasers and light detection and ranging systems, to help their vehicles discern the environment.

"To the casual observer, it seems easier to drive in a city versus in a desert environment. But if you think about it, it's probably an order of magnitude more difficult than trail driving," says Chris Urmson, director of technology for Carnegie Mellon University's team entry, "Boss," which won the Urban Challenge.

"The challenge wasn't really a navigational challenge--it was more of a sensing and classification challenge and being able to navigate through an urban terrain," says software engineer Chris Terwelp, cofounder of Blacksburg, Va.-based TORC Technologies. The company partnered with the Virginia Tech team, which placed third in the challenge with its Ford Escape hybrid, "Odin."

The teams that were most successful used multimodal sensors--combinations of cameras, lasers and range detection systems--to create a comprehensive understanding of the environment, says Thomasmeyer.

Some of the vehicles were equipped with a roof-mounted spinning light and range detection system--made by Velodyne--to attain a 360-degree, three-dimensional view. The system fires 64 lasers simultaneously and spins at 10 hertz to generate a million measurements per second. "It's the kind of thing we may need in the future to have self-driving cars," says Whitaker.

During the course of the competition, there were thousands of vehicle interactions, where one vehicle faced another vehicle and managed to pass by safely for the most part without getting into accidents.

"I was surprised by how well they did. We expected more vehicles to pull into the wrong lane or turn at the wrong time," he says.

Of an initial pool of 89 competitors, 35 teams competed in the semi-qualification rounds in Victorville, Calif. Eleven made it to the final event on the grounds of former George Air Force Base, where the military trains some units for urban operations.

DARPA awarded the top prize of $2 million to Team Tartan Racing from Carnegie Mellon University. Stanford University placed second and Virginia Tech came in third.

Carnegie Mellon's winning entry, a Chevy Tahoe, employed long-range radars that were mounted on the front of the vehicle to spot objects and keep tabs on the environment, says Urmson. When Boss came to an intersection, it could point those sensors in different directions to look for traffic before turning or merging. "Boss was able to do that in traffic moving up to 30 miles per hour," he says.

[ILLUSTRATION OMITTED]

Stanford's team also mounted lasers on its Volkswagen Passat to search for curbs and lane markings. It put radars on the front bumper for longer range obstacle sensing and vehicle tracking and employed a global positioning satellite and inertial sensing system, says David Orenstein, a spokesman for the university's school of engineering.

For the team that won DARPA's second Grand Challenge three years ago, the problem wasn't so much navigating, but rather processing the data from the additional sensors, he says. Inside the vehicle's trunk, two Intel Quad-Core processors computed the sensory data.

The networking was one of the most challenging and crucial components of the robots, because the data processing had to occur quickly and in a synchronized fashion to allow the vehicles to keep moving in the race.

"If you're trying to drive a car, and all of a sudden your network becomes slow, your vehicle is going to crash," says Paul DeBitetto, leader of the cognitive robotics group at Boston-based Draper Laboratory, which assisted the team from the Massachusetts Institute of Technology. The team finished fourth in the race.

Because of safety concerns, DARPA officials required vehicles to have the basic skills that drivers must acquire to obtain a California driver's license. In some cases, deficiencies, such as a lack of defensive driving skills, came to light.

"If the teams had totally mastered the skills, the two little fender benders that took place on the course wouldn't have happened," says Whitaker.

One of those incidents occurred at an intersection when a vehicle from Cornell University slowly approached the MIT vehicle, which failed to detect the imminent collision and move out of the way.

"If they had backed up, if they had hit that technical requirement, then they would've been safe," says Whitaker.

Team Oshkosh tweaked its software to help its TerraMax truck better navigate curvy roads lined with cars, says John Beck, chief engineer at Oshkosh Truck Corp. But when the vehicle navigated a parking lot, a software bug showed up. TerraMax found its parking spot, pulled in and backed out perfectly, but when it attempted to find its way out of the lot, the vehicle's lower level controls stopped responding.

"It started rolling forward at one mile an hour and needed to be paused," he says. Race officials subsequently disqualified the team from the race.

Most participants in the DARPA race believe that their technologies have a bright future ahead.

"If the Defense Department can come up with some specific requirements for these vehicles, they're going to have a lot of commercial off-the-shelf options available to them," says Terwelp.

The Army is developing a next-generation family of manned and unmanned vehicles known as Future Combat Systems. But it has not yet formulated formal requirements for the unmanned systems.

Thomasmeyer says those requirements would help autonomous technologies to take off in the commercial sector.

"That will guide industry in engineering the technology into real solutions," he says.

Robotics technology is very expensive, the race finalists say. Some teams spent upwards of $250,000 in sensors and computers. Stanford's vehicle cost half a million dollars.

DeBitetto says that work has to be done to make the sensors much cheaper and more reliable. "Driving down the size, the power and the cost of all the systems to do the Urban Challenge - that's a huge undertaking right there," he says.

"There's a lot more thought and work and software to be written to make sure that it would really be fool-proof," he adds.

For instance, the cars didn't have to contend with traffic lights or pedestrians, Orenstein points out.

There are also subtle cues, such as turn signals, that the robots would have to be able to detect to determine other vehicles' intentions. Figuring out pedestrians' intentions would be even more complicated.

Despite those hurdles, the teams and other robotics experts say that the technologies are mature enough for the military to begin harvesting them for its needs.

The technologies could be adapted for convoy operations or mine clearance.

"You can have one of these vehicles run back and forth on this route over and over and over again," says Beck, of Oshkosh. "The technology today would allow a vehicle to do that type of mission."

Another possible application is in logistics, says Thomasmeyer. "If you're talking about creating a convoy of 100 trucks that are all operating without anybody in them, in my mind that's a long way off," he says. But robotics technology could help automate some of the driving functions and free up convoy drivers to do other things.

The technologies also could help mitigate non-combat vehicle accidents, points out DeBitetto. Troops are driving fast in combat zones and having accidents. If the vehicles could automatically perform anti-rollover maneuvers or detect imminent collisions and initiate evasive maneuvers, then lives potentially could be saved.

The technologies developed for the Urban Challenge also could get a boost from the commercial sector. A number of automotive companies sponsored many of the teams and their interest is indicative of the implications of autonomous systems for passenger vehicles.

Shortly after the Urban Challenge, an official from General Motors announced that many of the teams' technologies would appear in passenger vehicles within the next decade.

Many of the sensors used in the DARPA race were intended for highway driving.

"You can get a sensor that has a 12-degree horizontal field of view and a three-degree vertical field of view and that will work perfectly fine for freeway driving. But it's insufficient for urban driving," says Urmson. "Now that the automotive industry and the military are starting to push into this domain, then the sensor manufacturers will start to make sensors that mate up with this problem and that will reduce the complexity of the sensor suite that you need and will also make them lighter and more effective."

Companies are improving laser-radar technologies through solid-state laser work. Such sensors would send out a strobe of light to capture a 3-D view of the environment instead of relying upon rotating lasers.

At Draper Laboratory, researchers are focusing on artificial intelligence so robots can learn from their mistakes, just as humans do. For example, if an autonomous vehicle is repeatedly being shelled by mortars on its route, it can learn that that road is dangerous and will find a safer route the next time. Or it can recognize that it's running low on fuel or is damaged.

Scientists say it will be years before autonomous vehicles are commonplace.

"I suspect that the military will arrive first at fully autonomous convoys before we'll see fully autonomous vehicles driving us to work every day," says Thomasmeyer.

EMAIL COMMENTS TO GJEAN@NDIA.ORG