Using Artificial Intelligence Methods to Win In Poker

See also: Poker AI: Can the Ultimate Computer Beat the Best Human?

Posted by Ori Cohen

imperfect informationThe concept of winning in Poker is easy. In theory all you have to do is obtain the winning hand–a combination of five cards that are of the highest value. However, due to its many forms and various dynamics, in practice, Poker is a complex game that relies in part on chance and requires a deep understanding of strategies; some of which are mathematical; others are based on personal experience.

Using artificial intelligence (AI) to solve games is a common practice. Games that do not allow randomness, follow predefined rules, and measure performance and winning using a relatively simple factor, (i.e., in Chess you win by taking the king, in Checkers you win by locking your opponent in place or capturing all his pieces), are classic test-beds for AI. These games can be perfectly defined on a computer [1], and therefore be solved completely by using mathematical formulas, statistical methods, machine learning methods (ML), or strategies and heuristics. In today’s CPU processing terms it is not expensive and relatively easy to accomplish. Games such as Chess [2] and Checkers [3] have already been completely solved by matured artificial intelligence software that is able to play as a human grandmaster and beyond. By utilizing AI programming to build a deep tree of all actions, they are able to traverse the tree in great speed and choose the optimal actions for winning. In contrast, games such as Poker introduce an element of luck, or in other words when you or your opponent draw a card from the deck, it is unknown whether this card will improve or hinder your chances of winning. Therefore, Poker is a harder game to solve than those that do not involve a random element, and cannot be solved by using exactly the same techniques that apply to non-random games.

Several influential studies have focused on different approaches to winning a Poker game. In this review we will introduce these aspects. Typically, winning in Poker can be done by modeling your opponents and exploiting their weaknesses; by using an algorithm that learns expert strategies and applies them in real time; or by “reading” your opponent’s thoughts in order to determine if your opponent is bluffing or not. All of these methods were studied solely for the purpose of winning in Poker.

Poker is an imperfect information game, which has competing opponents, risk management, probability of success and deception. Unlike Chess where ignoring opponent modeling is insignificant, in Poker it is highly valuable to acknowledge them, therefore many efforts had been invested in modeling opponents [4,5,6,7,8]. There are several methods of predicting [4] a probable action that a hostile opponent may choose:

      1. Expert systems - to hard wire our own or a fixed strategy, this method is generally good as a baseline measurement.

      2. Heuristics - a problem solving approach that results in an approximation for a decision, rather than an optimal one.

      3. Statistics - predicting an opponent to behave according to his track record. However this method is susceptible to an opponent that constantly changes his betting habits.

      4. Neural Networks (NN) - a generic system that is able to predict the opponent's next action, based on ML methods and inspired by our brain’s biology. NN are relatively easy to create, are accurate, and significantly better than the previous three, but we cannot obtain the learned information from them (Figure 1a). In general terms, NN receives a large input (possibilities for our next move), and processes it until an output (the next move) is chosen.

      5. Decision Trees (DT) - are another method in classic ML that can classify an opponent’s future action, by asking a question in each tree node (Figure 1b) and climbing down the tree to the end leaf which holds the final decisions. DT are not as robust as NN, but are human readable, and achieve similar results.

 

neural networks

Loki [4] and Poki [5] are two Poker programs that were designed to observe an opponent, model their behavior and dynamically adapt to their game-play in order to exploit their weak game-play patterns. This “skill” provides enormous benefit in the real world, as it can enhance a player’s natural ability to spot bluffing or understand the opponent’s intentions from his game play. Opponent modeling programs were designed to detect players that fold often and those who do not fold often; how passive or aggressive their game-play is; their betting or raising behavior based on their hand; and how well they adapt to dynamic strategies.

Loki and Poki were certainly an advancement toward defeating strong Poker players, but they certainly had their limitations, as human players are also strong at opponent modeling and can change their strategy in real-time. However, the Alberta group kept researching Poker and recently announced that for the first time, they have created a computer program called Cepheus [9,10] that solves “Heads-up Limit Texas Hold’em” (HULHE). This highly complex game has over 10^14 decision points and has challenged AI research for over a decade. To realize Cepheus, it took 68 days, using a high performance computer cluster with 4800 CPUs, coupled with data compression and a state-of-the-art computational algorithm. HULHE is a game that requires human players to use deception and bluffing, which are obviously not machine-like characteristics; and Cepheus is able to competitively play human players without deceiving them and without them knowing that they are playing against a machine.

EEG recordingAnother approach using AI for winning in poker is by detecting bluffing with an Electroencephalography (EEG). EEG is a method for reading electric signals from the brain, usually by using anelectrode-based cap that is fitted on a person’s head as seen in Figure 2. Measuring covert human states [11], can be performed with EEG, which provides robust signals that are registered as physiological traits, and can be classified by ML methods. Bluffing, which is an intentional act of misleading your other opponents in the game, is a complex task that entails premeditated risk assessment, an act of false intentions, and consistent follow-through, especially in maintaining your composure. AI methods were used to classify whether the player was bluffing or not. This method can aid a player to win in a poker match by knowing the mental state of his opponent and counteracting his hidden intentions. Empirical results indicate that it is possible to detect bluffing on an average of 81.4%. In other words, it is possible to detect bluffing in 8 out of 10 people, consistently, within a time frame of 200ms. The time frame is short enough in order to win the majority of games that use bluffing or other covert states as a strategical tool. EEG can also record additional improvements made if eye movement or muscle tension will be taken into consideration.

There are already consumer solutionsfor reading EEG brain signals. Although EEG has been used for over 20 years, the consumer products are still in their infancy. I hypothesize that EEG gadgets will be further miniaturized, made accessible for non-techy people, and fine-tuned to read specific brain patterns across a room with a high degree of accuracy. Automatic bluffing analysis methods will also improve, thus enabling covert usage of this technology in tournaments and private high stakes games. However, even with these advancements there are always ways to detect or block new technology from working wirelessly, and preventing Poker players from having an unfair advantage.

While Poker is still a challenging game to solve in many terms, past experience shows us that with advancement in research and hardware, artificial intelligence’s future can lead to solving other Poker variants within our lifetime.

References:

[1] S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 1995.

[2] M. Campbell, A. J. Hoane, and F h. Hsu. “Deep Blue. Artificial Intelligence”, 134(1-2):57{83, 2002}.

[3] J. Schaeffer, J. Culberson, N Treloar, B. Knight, P. Lu, and D. Szafron. “A world championship caliber checkers program”. Artificial Intelligence, 53(2-3):273{290, 1992}.

[4] A. Davidson. “Opponent Modeling in Poker: Learning and Acting in a Hostile and Uncertain Environment” (2002)

[5] D. Billings, A.Davidson, J. Schaeffer, D. Szafron. “The Challenge of Poker”. Artificial Intelligence, Issue 134, 201-240, (2002).

[6] A. Davidson, D. Billings, J. Schaeffer, and D. Szafron. “Improved opponent modeling in Poker. In International Conference on Artificial Intelligence” (ICAI'2000), pages 1467{1473, 2000}.

[7] Billings, Darse, et al. "Opponent modeling in Poker" AAAI/IAAI. 1998.

[8] Southey, F., Bowling, M. P., Larson, B., Piccione, C., Burch, N., Billings, D., & Rayner, C. (2012). “Bayes' bluff: Opponent modeling in Poker”. arXiv preprint arXiv:1207.1411.

[9] Tammelin, Oskari, et al. "Solving Heads-up Limit Texas Hold’em."

[10]Michael Bowling, Neil Burch, Michael Johanson, and Oskari Tammelin. “Heads-up limit hold’em Poker is solved”. Science, 347(6218):145–149, 2015.

[11] Jessika Reissland, Christian Kothe, Sabine Jatzev, Matti Gärtner, Sebastian Welke and Thorsten O. Zander. “BCI detection of deliberately hidden user states – or: Can we detect bluffing in a game?”

 

Photo credits:

Neural Network and decision tree model charts provided by the author.
Image of an EEG recording setup, Thuglas at English Wikipedia, in the public domain.

 

IF YOUR INTELLIGENCE IS REAL, DON'T MISS THIS:
SPECIAL OFFER: Get a £5 bonus to play at Titanbet Poker: Please list getbonus in the Promo Code field in the Cashier when you make your first deposit and you will immediately receive a £5 bonus. (T&C apply*)

Begin the poker download right now to get the bonus!

 


Ori CohenOri Cohen is Ph.D. candidate studying Computer Science with a focus on artificial intelligence. His research is in the fields of real-time fMRI BCI, Machine Learning, Data Mining, and AI. He has developed a real-time fMRI Brain Computer Interface system, where subjects are able to control the movement of a 3D avatar or tele-operated humanoid robot by motor imagery commands. He is also an avid landscape photographer and a passionate computer graphic artist.


 

Poker AI: Can the Ultimate Computer Beat the Best Human?

Posted by Steve Ruddock

The University of Alberta created quite a stir when it announced that their poker-playing computer "Cepheus" had solved Heads Up Limit Texas Hold’em. Cepheus relies on Artificial Intelligence (the ability to learn) allowing the thinking computer to adapt to its opponents.

The announcement had some wondering if this was the end of poker – if the game was now "solved" and humans would be at a disadvantage when playing unerring poker computers.

I'm here to tell you this fear is unwarranted.

poker ai

 

The fear of Artificial Intelligence

Few terms instill as much unfounded anxiety as Artificial Intelligence, or as it has become commonly known, AI, but most of this fear is based on myth and not reality.

What started out as more of a philosophical exercise with HAL 9000 in Stanley Kubrick's “2001 a Space Odyssey” and Skynet from the “Terminator” movie series, has now progressed into the realm of reality, with AI-programmed robots, robotics, and toys capable of handling a variety of tasks.

The most widespread and longstanding use of AI could very well be found in the discipline of game theory. First there was Chinook, playing checkers champions to a draw in 1990 and besting the top Checkers players in the world in 1994. Then there was Deep Blue besting Gary Kasparov in chess in 1997. And we now have a poker-playing computer that has been branded “unbeatable,” playing a specific poker variant.

The good news is it appears the fears of AI bringing about the downfall of civilization seem to be misplaced. AI computer programmers spend decades creating specialized Game Theory Optimal programs that can beat chess Grandmasters and the likes of Daniel Negreanu and durrrr in a poker game, so I'd imagine creating HAL 9000 is still a ways off.

There is a level of concern over the rapid growth of computer processing power, but all the way back in 1965, Hans-Joachim Bremermann noted that processing and computers have a ceiling (h/t FiveThirtyEight.com):

 “[the] speed, memory, and processing capacity of any possible future computer equipment are limited by certain physical barriers: the light barrier, the quantum barrier, and the thermodynamical barrier. These limitations imply, for example, that no computer, however constructed, will ever be able to examine the entire tree of possible move sequences of the game of chess.”

Now before you put on your tinfoil hat and start laying out the conspiracist's case on how it’s just a quick jump from GTO poker programs utilizing poker AI to world domination, let me explain why an unbeatable heads-up Limit Holdem computer isn’t even a big deal in the poker world – unless of course you’re one of the 14 people in the world who make their living exclusively playing in Heads-Up Limit Holdem games.

 

The sliding scale of poker complexity

In the world of mind games (such as checkers, chess, backgammon, poker, Go), Heads-Up Limit Hold’em is somewhere between a 1 and a 2 in terms of complexity on a scale of 1-10. It's a complex game, but still pales in comparison to other mind games.

According to a FiveThirtyEight.com column, Heads-Up Limit Hold’em is wedged in between Connect Four and checkers in terms of complexity – and it's much closer to Connect Four than checkers.

Heads-Up Limit Texas Hold’em has “state space” (the number of possible positions) of 10^14. The “state space” of Heads-Up No Limit Texas Hold’em is 10^140.

For a little perspective, chess, which has been branded an unsolvable game, is 10^50, and the number of atoms in the universe is 10^80.

And bear in mind we are talking about HEADS UP poker, imagine how complex a six-handed game, or a full table of nine players would be?

What Cepheus has solved is one of the simplest forms of poker. Cepheus has mastered the Lego Duplo of the poker world. It hasn’t even come close to mastery of building with regular Lego bricks, let alone Lego Technic blocks.

Yes, Cepheus could beat Daniel Negreanu in a game of Heads-Up Limit Hold’em. Switch the game to three-handed and Cepheus loses its edge. Switch the game to Pot Limit or No Limit Hold’em and Cepheus is once again in over its head.

But this isn't the only reason Cepheus falls short.

 

The Difference between GTO and maximizing value

Another reason to not be too concerned with what the University of Alberta has managed to accomplish was revealed by David Sklansky in an interview with Bloomberg: “If the computer is playing a bad player, it will win, but it won’t win as quickly as a human being playing a bad player.” Sklansky, an author of poker books who has appeared on television shows including the “World Series of Poker” also said: “I will destroy that beginner to a greater degree than this computer program will.”

The way Cepheus is programmed is to win, to defeat its current opponent, does not maximize value during that win.

if a computer is playing a bad player

 

If you are intelligent, artificially or naturally, here are the reasons why you should play at Titanbet Poker.

 

History of AI in poker

AI and poker date back to the 1980's, and like the fears of AI destroying civilization, the poker world has dubious concerns of unbeatable poker bots. For the most part these fears were, and are, completely unfounded.

Yes, in recent years poker bots have grown in skill and are now capable of beating some low limit games, which is certainly a frustrating development, but they are still light years behind competent, winning, players in terms of poker acumen.

In fact, their best usage is not as a money-making poker player, but as a training tool, which is how scrupulous developers have used them.

poker computer

 

ORAC

At the 1984 World Series of Poker Mike Caro unveiled ORAC, the first computer program for poker that used AI. In 1990 Caro would create the first poker odds calculator, Poker Probe.  

 

Wilson's Turbo Texas Holdem

In 1989 Bob Wilson unleashed Turbo Texas Holdem , a $195 computer program that taught people how to play Texas Hold’em Poker - James McManus wrote about his use of Wilson's Turbo Texas Holdem software in his preparation for the 2001 World Series of Poker in his New York Times bestseller, Positively Fifth Street.

 

Cepheus Project

Cepheus is the culmination of over 20 years of work by the University of Alberta's Computer Poker Research Group.

Poker artificial intelligence developed by machine learning, poker computers, is still a ways off from dominating the game, but that doesn’t mean we’ll stop arguing the poker vs. AI debate any time soon.

 

If you are intelligent, artificially or naturally, here are the reasons why you should play at Titanbet Poker.

Photo credits:
Poker AI , Morn, CC-BY-SA-3.0
Replica of HAL 9000, Photojunkie, public domain

 

IF YOUR INTELLIGENCE IS REAL, DON'T MISS THIS:
SPECIAL OFFER: Get a £5 bonus to play at Titanbet Poker: Please list getbonus in the Promo Code field in the Cashier when you make your first deposit and you will immediately receive a £5 bonus. (T&C apply*)

Begin the poker download right now to get the bonus!

 


Steve RuddockSteve Ruddock is a veteran writer, analyst, & consultant in the poker and iGaming industry who covers nearly every angle of online poker in his job as a full-time freelance poker writer. Follow Steve on Twitter at @SteveRuddock.

 


 

Further Reading:  

Titan Poker Benefits
Get Amazing Benefits by Signing Up

Moses
Poker and the Bible

texas hold'em


Texas Hold'em Poker Guide

poker better than sex


Why Poker Is Better than Sex

 

Dear player,

This site is regulated for players located in the United Kingdom. Visit Titan Poker and enjoy similar games, features, promotions, and benefits!

Contact our 24/7 support team for assistance.

Visit Titanpoker.com