Sony trains AI to leave world’s best Gran Turismo drivers in the dust

Having thrashed mortal champions at poker, chess, Go, and Starcraft, an artificial intelligence program has delivered another humiliation, leaving the world's best video game racing drivers in the dust.

Researchers at Sony trained an AI called GT Sophy to play the PlayStation game Gran Turismo and found that it could outrace 95% of human players after two days and continued to shave tenths of a second off its lap times over the following week.

Beyond learning the fastest line to take around different race tracks, GT Sophy honed its tactics around acceleration and braking, when to tuck in behind a fast car in front, and when to break out of the slipstream and attempt to overtake. Faced with cars that blocked its path, GT Sophy switched to another racing line to bypass those ahead.

To add to the challenge, GT Sophy had to take account of race penalties that were dished out for infringements based on ambiguous rules - echoing the fine human judgements that can make the difference between winning and losing in Formula One.

"The success of GT Sophy in this environment shows for the first time that it is possible to train AI agents that are better than the top human racers across a range of car and track types," the authors write in the journal Nature.

The team led by Peter Wurman, the director of Sony AI America, based GT Sophy on a common AI technique called reinforcement learning. The AI was fed information from multiple Gran Turismo games and by maximising rewards for fast lap times and minimising penalties for collisions and such like, the program steadily mastered the game.

As with other AIs that have trounced human champions, there is more to GT Sophy than playing games. Excelling at Gran Turismo demonstrates how an AI can learn strategies for working in a demanding situation where the vagaries of human judgment are crucial. For example, it may seem sensible to drive as close as possible to the car in front, but if the lead driver touches the brakes too early, the inevitable rear-ending suggests the strategy needs more work.