Artificial intelligence (AI) and its subset machine learning (ML) have long been the preserve of experimental labs. However, over the past few years, both technologies have been moving from labs into tools that are available to everyone. One sector where this has been particularly noticeable is the development of video game machine learning bots.
Deep learning – the technology that’s behind such devices as autonomous vehicles, natural language processing (NLP) and optical character recognition (OCR) – is beginning to make a strong impact, with labs such as DeepMind and its competitor OpenAI developing video game bots that are capable of beating professional players.
We’re moving closer and closer to the point at which immersive and intelligent AI-enabled games will become a reality. With access to the right tools, game developers could build sophisticated experiences able to respond according to player behaviour, with characters that develop and evolve over time.
Let’s take a look at the current state of artificial intelligence in video games.
How Is Machine Learning Used In Video Games?
Before we get started, let’s take a quick look at video game machine learning and what the term actually implies. Machine learning, as we mentioned above, is a subset of artificial intelligence that uses algorithms and statistical models to enable machines to make informed decisions. Artificial intelligence, on the other hand, is the whole science of teaching machines to think like humans.
When it comes to video games, AI and ML techniques are already in use in the form of non-player character (NPC) control and procedural content generation (PCG). At this point in time, it’s pretty difficult for the general public to get detailed information about video game machine learning, as game companies tend to withhold information in order to protect their intellectual property.
Therefore, most of the information available stems from research projects and is mainly related to the use of deep learning in complex strategy games such as Minecraft and Starcraft. But at the same time, machine learning has been applied to traditional games such as chess and Go.
Top 5 Examples Of Video Game Machine Learning Bots
Video game machine learning actually goes back much further than one would think. Back in the 1950s, Alan Turing, the “father of computer science” created a computer chess programme that foreshadowed the rise of video games AI. But things have moved on a lot since then. Here are some prominent examples of video game machine learning.
1. AlphaGo
While Go is originally a board game, AlphaGo was a major milestone in deep learning game AI. In October 2015, they made waves when AlphaGo beat a professional human Go player with no handicaps. A few months later in March 2016, AlphaGo went on to beat a new record by defeating Lee Sedol in a five-game match. DeepMind went on to spawn three AlphaGo successors (AlphaGo Master, AlphaGo Zero and AlphaZero).
AlphaGo’s 2016 achievement was considered a breakthrough in artificial intelligence research. Murray Campbell of Deep Blue defined the victory as “the end of an era… board games are more or less done and it’s time to move on.” Go had been considered until that point as a major challenge in artificial intelligence compared to other games such as chess, as its larger branching factor renders traditional AI techniques, such as alpha-beta pruning, heuristic search and tree traversal, far more difficult to employ.
2. OpenAI Five
OpenAI Five is a video games machine learning project that pits a team of bots against flesh-and-blood players in the game Dota 2.
It was demonstrated for the first time in 2017, where it defeated Dendi, a professional Dota 2 player, in a live one-on-one game. The programme was capable not only of performing as a complete five-player squad. It also could beat teams of professionals.
The video game bot has gained acknowledgement from both the video game and the artificial intelligence community. Bill Gates himself defined it as a “big deal”, as the achievement “required teamwork and collaboration.”
3. AlphaStar – Video Game Machine Learning Bot
Hot on the heels of OpenAI Five, AlphaStar was the first artificial intelligence programme to beat a top professional StarCraft player. StarCraft is perceived to be one of the most challenging Real-Time Strategy games and a hard problem for artificial intelligence research. Historically, it’s also one of the longest-played eSports.
A few months ago, Grzegorz “MaNa” Komincz, one of the world’s most reputed professional StarCraft players, was defeated by AlphaStar in a series of test matches that took place under professional match conditions. AlphaStar uses supervised learning and reinforcement learning as part of a deep neural network trained on raw game data.
4. Capture the Flag AI
Yet another project by Google’s DeepMind research company. Capture the Flag AI uses reinforcement learning-based AI agents to play Capture the Flag, a variant of Quake III Arena. In this game, two teams compete across a randomly-generated map to locate and capture each other’s flags. After 450,000 games, the bots were capable of defeating professional flesh-and-blood players. It is considered another milestone in AI-enabled teamwork.
5. Insert Your Bot Here
Feeling inspired? You don’t have to be part of a state of the art research lab to build your own video game machine learning bot. OpenAI Gym, a platform for reinforcement learning algorithms, offers users a streamlined interface for managing dynamic environments. There are also thousands of options available. These include classic games such as Minecraft and Grand Theft Auto. Why not build your own bot and try it out? We look forward to seeing your work on this list one day!
