Computer scientists have designed an artificial intelligence package that can beat almost every professional e-sports player at one of the world’s most challenging real-time strategy games, StarCraft II.
In recent years an AI agent beat human players at the challenging board game Go, only to be beaten itself soon after by its own upgrade.
Until now, however, real-time online multiplayer strategy games – the bedrock of the rapidly growing multimillion-dollar e-sports industry – have proved challenging for AI designers.
Past attempts to use machine-learning to play games such as StarCraft, Dota, or League of Legends have required rather unsporting modifications, such as simplified rules.
And that is perhaps understandable, because beneath their fantasy or science fiction trappings, these types of e-sport games are computationally complex.
“Each game consists of tens of thousands of time-steps and thousands of actions, selected in real-time throughout approximately ten minutes of gameplay,” writes a team of scientists headed by Oriol Vinyals of DeepMind Technologies in the UK, in the journal Nature.
Vinyals and colleagues tackled the challenge of StarCraft by creating an AI agent called AlphaStar. The system combines new and existing techniques for constructing artificial neural networks, including imitation learning, reinforcement learning and multi-agent learning.
The scientists trained AlphaStar using games recorded and stored on a publicly available archive. The training had to teach the system to recognise not only visible threats and allies, but also other bad guys who lurked outside its avatar’s field of view.
A key challenge was training the system to respond to a mixture of opponents. In the game of Go, the opponent is always a Go-player. In StarCraft there are three types of characters – Terrans, Zergs and Protoss – all of which have different game-playing mechanics, abilities and strategies.
Different iterations of AlphaStar, effectively at beginner, intermediate and experienced levels, were then signed up to Battle.net, a website that pitches anonymous users against each other. After 90 games, the most experienced iteration, dubbed AlphaStar Final, had blitzed the competition, achieving a ranking above 99.8% of human players and earning itself the coveted title of Grand Master.
Vinyals and colleagues suggest that the triumph may have applications outside the world of e-sports, particularly in “real-world domains such as personal assistants, self-driving cars, or robotics”.
It might also prove, perhaps, a handy way for cash-strapped researchers to make their own lives a bit easier. In 2018, a total of just over $4 million dollars was won in StarCraft II tournaments. On the evidence, AlphaStar could clean up.
