Sony trains AI to leave world’s best Gran Turismo drivers in the dust

Having thrashed mortal champions at poker, chess, Go, and Starcraft, an artificial intelligence program has delivered another humiliation, leaving the world's best video game racing drivers in the dust.

Researchers at Sony trained an AI called GT Sophy to play the PlayStation game Gran Turismo and found that it could outrace 95% of human players after two days and continued to shave tenths of a second off its lap times over the following week.

Beyond learning the fastest line to take around different race tracks, GT Sophy honed its tactics around acceleration and braking, when to tuck in behind a fast car in front, and when to break out of the slipstream and attempt to overtake. Faced with cars that blocked its path, GT Sophy switched to another racing line to bypass those ahead.

To add to the challenge, GT Sophy had to take account of race penalties that were dished out for infringements based on ambiguous rules - echoing the fine human judgements that can make the difference between winning and losing in Formula One.

"The success of GT Sophy in this environment shows for the first time that it is possible to train AI agents that are better than the top human racers across a range of car and track types," the authors write in the journal Nature.

The team led by Peter Wurman, the director of Sony AI America, based GT Sophy on a common AI technique called reinforcement learning. The AI was fed information from multiple Gran Turismo games and by maximising rewards for fast lap times and minimising penalties for collisions and such like, the program steadily mastered the game.

As with other AIs that have trounced human champions, there is more to GT Sophy than playing games. Excelling at Gran Turismo demonstrates how an AI can learn strategies for working in a demanding situation where the vagaries of human judgment are crucial. For example, it may seem sensible to drive as close as possible to the car in front, but if the lead driver touches the brakes too early, the inevitable rear-ending suggests the strategy needs more work.

More importantly, the breakthrough will feed into a debate about the best computational approaches to use in driverless cars. Neural networks of the kind that underpin GT Sophy and other AIs, such as those from DeepMind, have proven themselves at identifying objects in the world, but other driving tasks such as planning the best line to take are often based on pure physics. GT Sophy suggests neural networks might do a good job of this too.

Despite the AI's impressive debut, the Sony team concede that GT Sophy still has much to learn. On a long straight, the AI overtakes at the first opportunity, sometimes leaving enough road for the opponent to use GT Sophy's slipstream and pass back. Without collision penalties, the authors noted GT Sophy became "notably more aggressive".

In an accompanying article in Nature, Prof Christian Gerdes, co-director of the Center for Automotive Research at Stanford University, who was not involved in the research, writes: "What really stands out is GT Sophy's performance against human drivers in a head-to-head competition. Far from using a lap-time advantage to outlast opponents, GT Sophy simply outraces them."