Card Against Humanity: The Evolution of Poker AI

Since the mid-1980s, scientists and poker players have tried to create a poker artificial intelligence (AI) that can compete with the best poker players in the world. Recently, they succeeded. But that success didn’t come without investing a tremendous amount of time and work. Poker and AI have been intriguing topics ever since. In this article, we’re going to walk you through the evolution of poker AI.

The Evolution of Poker AI Infographic

↓ Click here to scroll down to the article ↓

A Short History of Poker AI

We will guide you through every milestone that the poker AI has passed and tell you what the situation is today. 

• 1984

The first person to create serious software that can compete against poker professionals was Mike Caro. He is an American professional poker player, pioneer poker theorist, and writer of many books related to poker. 

All this happened in 1984 when the Orac software (it’s Caro backwards) competed with professionals at the World Series of Poker. Looking from today’s perspective and standards, even though it was simple software, it managed to win against Doyle Brunson, a two-time winner of the World Series of Poker.

• 1991-1999

Later on, in 1991, a computer research group from the University of Alberta (UoA) started working on poker bots. The best-known face in this group was Darse Billings, also a part-time poker and game scientist. 

In 1997, the UoA group released Loki. It’s the first in a long line of AIs that would impact the poker world. Moreover, it is said to have a skill level slightly below an average human poker player. Loki was developed to play a full table poker (nine players). 

In 1999, Loki was turned into Poki, which also focused on Texas Hold’em instead of the No-Limit version. 

Generally, the first poker bots were based on the concept of a Nash equilibrium. Simply put, it’s about making the best possible decision while considering the opponent’s decision.

• 2004

Many people contributed their time and investments for us to have what today represents poker and AI. 

Professor Tuomas Sandholm was the driving force behind some poker AIs. He started working on a poker AI in 2004 at Carnegie Mellon University. 

As the year passed, Sam Ganzfried, Noam Brown, and Andrew Gilpin made significant contributions to Sandholm’s research. 

• 2015

After years of development by many masterminds, finally, in 2015, the UoA released a bot that would solve Heads-Up Limit Hold`em. The name of this poker bot was Cepheus, and it mastered Limit Hold`em by playing itself for two months. Cepheus was more brilliant than the rest of the bots at that time, and it was referred to as “unbeatable.”

• 2017 – till now

UoA reveals DeepStack AI

As time passed, poker AI was changing and upgrading with lightning speed. So, in 2017, UoA released a bot called DeepStack AI, which won a test match against professional poker players. What is special about this bot is that it can imitate human intuition and learn as the game progresses.

Famous Libratus

Despite being Claudico’s successor, this poker AI bot was made from scratch. Libratus delivered probably the most devastating blow in history during the human vs. poker AI battle in January of 2017. Libratus, a brand-new AI from Carnegie Mellon, didn’t just outperform humans… It crushed them! Also, it had gathered a phenomenal $1,766,250 (appr. £1,276,152) by the time the last hand of the 20-day, 120,000-hand challenge was dealt.

Welcome, Pluribus

As the AI is constantly updating and reaching perfection, a new poker bot has arrived, Pluribus. With a pedigree from the previous bots, this beast managed to beat five players in six-player Texas Hold’em. 

Artificial intelligence makes it easy to beat the unbeatable game of odds. Some of the most interesting historical contests where poker and AI were exploited are: 

• ICCM PokerBot 2004
• The World Series of Poker Robots 2005
• Brains vs. AI Competition 2015
• Brains vs. AI Competition 2017
• Poker AI Lengpudashi vs. Poker Pros 2017

Poker AI Basics

Scientists Behind Poker AI

As we mentioned, many scientists and poker players invested their time to constantly upgrade the bots, taking the AI world so far. 

The most famous of them are Michael Bowling, Tuomas Sandholm, and Noam Brown. These three men, accompanied by many co-workers, poker players, and so forth, did incredible work in the development of the artificial intelligence poker bots. 

Michael Bowling

Michael Bowling, the mastermind behind DeepStack poker AI, works as a full-time professor at the University of Alberta. He focuses on researching the problem of how computers can learn to play games through experience. Moreover, Bowling is the leader of the Computer Poker Research Group, which has built some of the best poker-playing programs on the planet, some of which won international AI competitions.

Tuomas Sandholm

Tuomas Sandholm works at Carnegie Mellon University in the Computer Science Department. He is mainly interested in artificial intelligence; hence, he was the mastermind and the leading developer behind Libratus, the best poker AI today.

Noam Brown

Noam Brown is a young research scientist at Facebook AI Research, with a Ph.D. from Carnegie Меllon University. Together with Tuomas Sandholm, he created Libratus and got nominated for an “Innovator under 35 years.”

Best Known Poker AIs

Many universities, scientists, and individuals have been highly interested in artificial intelligence and poker. Since the best-known poker AIs were time-consuming projects,  it often took a whole group of poker players, scientists, enthusiasts, AI experts, etc., to come up with an innovation.

The University of Alberta remains the most productive in this field since it has seven well-known poker bots, and they are: 

• Loki
• Poki
• PsOpti/Sparbot
• Vexbot
• Hyperborean
• Polaris
• Cepheus

Afterwards, Carnegie Mellon University comes with:

• Tartanian
• Claudico
• Libratus
• Pluribus

The products of the University of Auckland include:

• Casper
• Sartre 

As we mentioned before, many individuals are contributing to AI on their own, and it is worth noting the independent work of Fredrik Dahl, who invented Texas Hold’em and Heads-up poker.

The future lies in AI, and we should expect even more productive, intelligent bots which will change the gambling industry for good. 

EHS Poker AI Algorithm

Effective Hand Strength (EHS) is an algorithm used for opponent modelling in poker AI. It was conceived by Denis Papp, Jonathan Schaeffer, Duane Szafron, and Darse Billings. 

EHS was first published in a research paper in 1998. Since then, it has been considered a reference to poker artificial intelligence, becoming the basis of further research.

So, how does it work? 

EHS is a complex algorithm that cannot be computed manually and has to be used in an artificial intelligence context, given its complex nature. The underlying assumption is that it consists of the current Hand Strength and its potential to improve or deteriorate. This algorithm is applicable to many poker games, including Limit Texas Hold’em, No-Limit Texas Hold’em, etc.

Claudico Poker AI

Claudico is a poker bot developed by Tuomas Sandholm and Carnegie Mellon University’s science team. The bot’s name is derived from Latin, meaning “I limp,” which refers to limping into a hand without raising — a strategy often employed by the bot. However, its original name was Tartanian7.

This poker AI plays No-Limit Texas Hold’em heads-up. Interestingly, it was the first poker bot to play against the top poker players in the world. With its 2 TB of data, Claudico can come up with enough strategy to puzzle even its creators.

So, how does this poker AI work? 

The developers did not program in human poker expertise. Instead, they used algorithms with just poker rules run on a supercomputer called Blacklight. Also, Claudico doesn`t use one strategy per one hand. Instead, it mixes a couple of them. This makes Claudico’s algorithms implementable in many other fields such as business transactions, cybersecurity, and healthcare.

Libratus Poker AI

Libratus is a poker AI created by Tuomas Sandholm and Noam Brown of Carnegie Mellon. The bot’s name is derived from the Latin word meaning “balance.” 

It consists of an algorithm that computes the strategy, defining how the software will play in certain situations. In most cases, Libratus mixes two or more strategies to find the best solution. This poker AI bot is one of the awarded bots, which beat some of poker`s top heads-up players in the “Brains vs. AI” competition. In addition, Libratus won the “Best Use of AI” award in 2017.

This poker AI bot has three main modules, with new algorithms in each of the three:

• Precomputes a solution before the event.
• Offers solutions to the subgame during the play.
• Improves the strategy by augmenting the pre-computed blueprint over time.

To do this, Libratus scans the opponents’ moves and locates the gaps.

Bridges Supercomputer

Many poker AI bots use supercomputers to develop algorithms and strategies to win against the opponent. Brown and Sandholm used Bridges supercomputer to create different strategies for Libratus (the awarded bot mentioned above).

Bridges is a supercomputer that is roughly 30,000 times faster than an average desktop PC. It costs around $10 million (appr. £7.2 million) and has 274 TB of RAM. 

The most interesting thing about this supercomputer is that in order to create different strategies, Bridges plays against itself for many days. Some internet facts claim that playing can last up to two months. 

Poker AI vs. Poker Professionals

Artificial intelligence has overtaken the greatest humans at several of our favourite games in recent decades: 

• Checkers, with its long-term planning
• Chess, with its distinctive strategy
• Go, with its intricacy
• Backgammon, with its element of chance
• Poker, with its imperfect information 

Now we will present to you the poker AIs that won against the human brain.

Claudico vs. Pros

At the event held at Rivers Casino in Pittsburgh, three of four professional players won against Claudico. Over $170 million (appr. £123 million) of virtual money was involved in this game, with 20,000 hands per player. 

The final chip count was $732,713 (appr. £529,968). Let’s look at the players’ results: 

DeepStack vs. Pros

DeepStack is a poker AI created at the University of Alberta. It has an intuition that needs to be trained like human intuition. 

In 2016, DeepStack beat 11 professional poker players during the research that ran 44,000 hands, with only one player falling outside the statistical significance margin.

DeepStack won 49 big blinds/100 across all games played (always folding would only lose 75 bb/100), more than four standard deviations from zero. Thus, it became the first poker AI to beat professional poker professionals in Heads-Up No-Limit Texas Hold’em poker.

Libratus vs. Pros

Libratus competed in a challenge that lasted 20 days and featured 120,000 hands (30,000 hands per player). This competition was between AI and the human brain. The players and AI had 20k chips for every hand, with the blinds at 50/100.

Four well-versed and distinguished poker players represented the humans in this challenge: Dong Kim, Jason Les, Jimmy Chou, and Daniel McAulay.

• Kim is a very successful online high-stakes player.
• Les was twice in striking range of a WSOP bracelet in 2015, taking second and third places.
• Chou won the Asia Championship of Poker.
• MacAulay won several hundred thousand dollars online. He is an expert in online tournaments. 

These four opponents excel at Heads-Up No-Limit Texas Hold’em, the game played during this challenge. 

However, Libratus won against each player at a rate of $14.72 (appr. £10.63) per hand. It was $1,766,250 (appr. £1,276,152) ahead at the end of this experiment. 

These are the results of the players (in chips):

• Dong Kim: −$85,649 (appr. £61,877)
• Daniel McAulay: −$277,657 (appr. £200,593)
• Jimmy Chou: −$522,857 (appr. £377,738)
• Jason Les:  −$880,087 (appr. £635,818)

Pluribus vs. Pros 

Another poker AI which was exposed to an experiment with human professional poker players was Pluribus. Before facing the human players, the AI played trillions of hands against itself. 

In the competition, one of the opponents was Chris Ferguson, a six-time World Series of Poker champion. However, he was defeated. Ferguson said, “Pluribus is a very hard opponent to play against,” and some other players claimed that Pluribus is a more efficient bluffer than a human. 

When professional brains announce this, then you realize that the everyday poker player would lose for sure. 

Pluribus played 10 hands against 12 professionals, which lasted 12 days. It managed to win $5 (appr. £3.61) per hand, which roughly amounted to $1000 (appr. £722) per hour.

Conclusion

With the finest poker players being challenged by computers, one question looms: is poker destined to fail? Since there is a distinction to be made between live and online poker, the answer is twofold.

It should also be emphasized that the difficulties that the poker business is experiencing are not new. Bots have already proven their capacity to defeat competent human players.

After being present in online poker for at least eight years, bots have become prohibited on all reputable sites. Any players discovered using them will have their earnings seized, and those harmed will be compensated.

Undoubtedly, the use of poker AI bots highlights the tremendous progress that poker AI has achieved. But does the future hold the odds of AI winning against humanity? We will need to wait and see or make a bot who will predict that for us.