Poker Solvers. The Tool That Turned the Game Upside Down

Solvers are a fairly young phenomenon in poker. They only became available to a wide audience 11 years ago, but they were in enormous demand right away. The arrival of these programs marked the shift towards GTO-oriented poker. Even renowned professionals were forced to follow this trend, because otherwise they would stop understanding their opponents' actions and, accordingly, end up losing. However, solvers, like any other novelty, still have their sceptics and opponents to this day. 

What is a solver

A solver is a program that, for a scenario set by the user, calculates an "unexploitable" strategy as close as possible to the Nash equilibrium. In game theory, that is the name for a set of strategies in which no participant can increase their winnings by changing their approach to the game, as long as their opponents do not change their own strategies. In the context of poker, such an equilibrium means that a player will not be able to increase their winnings simply by changing their own strategy.

Modern solvers are based on the counterfactual regret minimisation (CFR) algorithm. It was presented by researchers from the University of Alberta in Canada in 2007, who were seeking to teach a program to win at poker. The name of the algorithm sounds complicated, but in practice it is very similar to the experience-based approach used by people who want to learn poker. In essence, it is a method of trial and error, thanks to which a person remembers, over the course of their games, which decisions brought them the greatest success. 

CFR works by accumulating "regret" about the missed gains that could have been obtained with different decisions, and uses this data to change its strategy in future iterations. Ultimately, the program is meant to reduce the average "regret" to zero and get close to the Nash equilibrium. So it turns out that the program simply records the best moves and builds a strategy that is maximally optimised from the standpoint of game theory — that very same GTO.

The era before solvers

The first and very important shift towards a mathematical approach and GTO happened long before solvers appeared, back in the early noughties. That was when the first mass-market poker tool emerged — the free equity calculator PokerStove, developed by Andrew Prock.

PokerStove was revolutionary, because it made it possible to calculate the equity not of individual hands, but of an entire range. The calculator made it possible to compute EV both preflop, using Monte Carlo methods, and postflop thanks to exhaustive enumeration. It was this program that pushed players to move away from analysing individual hands and switch to studying ranges.

Poker trackers were developing in parallel, for example PokerTracker and Hold'em Manager. The second version of PokerTracker, released in 2003, featured the first commercially successful HUD (Heads-Up Display). It gave an enormous informational advantage to players who knew about this tool and used it actively.

The HUD allowed poker players to track their opponents' statistics at the table in real time. Players began to see how often an opponent voluntarily puts money into the pot (VPIP), how actively they raise preflop (PFR) or fold after a 3-bet. Thanks to these and other metrics, it became possible to understand more quickly what kind of player you were up against: tight or aggressive. And intuitive conclusions about opponents gained statistical confirmation, making it possible to build a more optimal strategy. In addition, HUDs help to review sessions and find your own weak spots.

Development of PokerStove ceased in 2008, and the site closed down in 2013. Andrew Prock subsequently published the program's source code on GitHub. PokerTracker and Hold'em Manager, meanwhile, merged into a single company in 2014, although both programs continued to develop separately from one another. Trackers still remain an important tool for analysing sessions, however some large online rooms either ban the use of third-party HUDs or offer their own alternatives with a limited set of statistical data.

The first program to beat a poker player

Scientists had been trying to "solve" poker ever since the 1990s in the context of studying game theory and machine learning. This question was taken up by the University of Alberta's Computer Poker Research Group (CPRG). For many years it developed programs that were meant to find the perfect game strategy. 

In the late 1990s, the Loki and Poki models were released, which were examples of expert systems. In them, human ideas about correct strategy played a major role, and with Loki, researchers had to manually search for mistakes and add new rules to make the bot stronger. 

However, no-limit hold'em is too complex to be fully solved. That's why scientists focused on heads-up limit hold'em and started training programmes on a simplified model of the game. Bots received abstractions of cards and actions, which grouped similar hands and board textures into special "buckets", and also limited the number of considered lines and sizings.  

The next step was PsOpti — one of the first strong bots, which was built around game theory and solving a simplified version of heads-up limit hold'em. The fourth version of the programme achieved an important breakthrough in research, since to achieve strong play it used not a set of human rules, but calculations of an approximate equilibrium. This approach was also applied in the next lineup of bots called Hyperborean.

In 2007, researchers at the University of Alberta published a paper on the CFR algorithm, which we mentioned at the beginning of the article. This algorithm made it possible to significantly increase the volume of calculated abstractions and create a balanced foundation of solutions that could then be adapted. It was thanks to CFR that scientists managed to create a bot that would finally be able to beat humans in limit hold'em.

That bot was Polaris, released in 2007. This programme was already a system of several strategies, which during play could choose between a cautious and an aggressive strategy. The bot played against professionals Phil Laak and Ali Eslami. The format was unusual — duplicate matches of 500 hands each. To minimise variance to a minimum, pairs of players in each match received the same cards, but with mirrored positions. That is, if the bot was dealt bad cards in a game against Laak, then in the same hand against Eslami it received good cards from Phil's hand.

The first session ended in a draw, Polaris won the second, and humans won the third. The outcome of the match was decided in the fourth session. Laak and Eslami managed to beat the programme, finishing the final session up $570. However, after the match, Phil Laak admitted: "The nuance is that we won, but by a small margin. Bots are catching up with humans. That's the real takeaway from this event." 

In 2008, an updated and improved Polaris got a rematch against six professional poker players. Among the bot's opponents was Matt Gavrilenko, who in 2009 would become a World Series of Poker bracelet winner in the $5000 No-Limit Hold'em Short-Handed event. 

The new version of Polaris learned to track how humans adapted to its tactics and change strategy in response. Laak and Eslami were able to win the first match precisely because they found weaknesses in the programme and started exploiting them. In the rematch, Polaris won three sessions with two losses, while one session again ended in a draw. The result was entered into the Guinness Book of Records as the first case when a bot beat human professionals in a poker tournament.

In 2015, researchers from the CPRG announced that they had, in effect, "solved" limit hold'em in heads-up format, presenting the Cepheus model. It used a new version of CFR — CFR+ — and became effectively unbeatable for a human. According to calculations, the optimal counter-strategy against the bot would bring a poker player an average win of 0.000986 big blinds per game. This means that even if a human played against Cepheus their whole life, they would not be able to beat it.
"We're not saying it's going to win money every single hand. But over the long run, the computer cannot lose — it will be a tie or a win for the AI," said Michael Bowling, one of the developers of Cepheus, in an interview with The Verge. 

Solving no-limit hold'em

Researchers Noam Brown and Tuomas Sandholm from Carnegie Mellon University went even further. They developed a programme that challenged humans in no-limit hold'em. In that same year, 2015, the scientists created the bot Claudico, which played heads-up matches against Doug Polk, Jason Les, Dong Kim, and Bjorn Li. Three of the four professionals won matches of 20,000 hands each.  

In 2017, Brown and Sandholm presented a new programme, Libratus. This model compiled a general base strategy in advance, solved non-standard problems on later streets in such a way as not to disrupt the balance of the previously calculated approach, and after each day analysed the exploits found by humans and adapted its defence against them.

Against the bot played Jason Les, Dong Kim, Jimmy Chou and Daniel McAulay. The matches kept duplicate hands, just as with Polaris, and were held at the Rivers Casino in Pittsburgh over the course of 20 days. After 120,000 hands, Libratus beat all the professionals with a total profit of 1,766,250 notional dollars. Les lost more than $880 thousand, Chou lost $520 thousand, McAulay lost $277 thousand, and Dong Kim lost $85 thousand. The bot's winrate came to 14.72 bb/100.

The appearance of Libratus was an important milestone in the development of solvers. This programme showed that in no-limit hold'em heads-up play, AI is capable of handling situations that differ from the simplified poker model very well. But the researchers at Carnegie Mellon University didn't stop there.

The next step was moving to multiway pot play. To do this, Brown and Sandholm, together with Facebook AI, developed Pluribus. This model also built its base strategy by playing against itself. This tactic was mainly needed for preflop play, while on postflop streets the programme ran a depth-limited search, calculating actions without playing the hand out to the end. At the same time, Pluribus turned out to be far less demanding in terms of computing power than Libratus.

Professionals who had earned at least $1,000,000 over their careers played against the bot. Among them were WPT titles record holder Darren Elias, six-time WSOP bracelet winner Chris "Jesus" Ferguson, Nick Petrangelo, Linus Loeliger and Michael Gagliano. There were two experiments with humans. In the first, five people played against a single copy of Pluribus, and in the second, one poker player faced off against five separate copies of the programme.

In the first case, 10,000 hands were played over 12 days. The bot's opponents changed every day. From a pool of 13 professionals, the organisers selected five available players each day. Pluribus won, showing a winrate of around 5 bb/100. It was noted that the programme frequently changed tactics and resorted to donk bets, that is, it played passively and then bet into the opponent who had been the aggressor on the previous street.

In the second experiment with five copies of the AI, Elias and Ferguson each faced the bot, each playing 5,000 hands. Pluribus won here too. However, Ferguson's loss percentage turned out to be lower than Elias's. The researchers suggested this could be a result of variance, skill, or a more conservative strategy involving folds in unfamiliar situations.

"The bot wasn't playing against mediocre players. It was playing against some of the best players in the world. Its main strength lies in its ability to use mixed strategies. Humans try to do this too, but most of them fail to do it consistently," said Elias after the experiment.

"It was incredibly fascinating to play against a poker bot and watch some of the strategies it chose. There were a few moves that humans simply don't make. This is especially true when it comes to bet sizing," Gagliano shared his impressions. 

Commercial success

The PokerSnowie programme, released in 2013 by the company of Olivier Egger and Johannes Levermann, came very close to the functionality of modern solvers. This neural network learned no-limit poker on millions of hands played against itself and used the resulting data set to evaluate user-submitted situations.

However, PokerSnowie had one important drawback. True solvers calculate the best solutions for specific spots defined by the user. PokerSnowie couldn't do that. The programme instead predicted the strongest move by analogy with situations it already knew, but in rare spots its effectiveness dropped noticeably. Nevertheless, it was this tool that introduced the wider poker community to the principles of GTO.

The breakthrough came in 2015, the same year Claudico and Cepheus were released. Polish developers Piotr Łopuszewicz and Kuba Straszewski released PioSOLVER. It was inspired by chess engines and became the first accessible desktop solver that didn't require enormous computing power. This programme could solve a typical flop situation on an ordinary home PC within minutes.

SimplePostflop was also released around the same time, and it stands out for supporting not only heads-up play but also multiway pots. This solver focuses on visual clarity and ready-made game tree templates. Pio, on the other hand, wins out in terms of customisation options and offers more tools for studying non-standard situations. In addition, it has a Node Locking feature, which allows you to fix your opponent's strategy in certain pots and search for exploits against them more effectively. 

In 2017, the desktop solver lineup grew with GTO+ from the developers of Flopzilla, and MonkerSolver. The first tool managed to achieve a true Nash equilibrium without residual error. In addition, GTO+ stands out from competitors thanks to its lower price (a one-time purchase for $75) and an advanced data storage system that cut memory "consumption" from hundreds of megabytes down to hundreds of kilobytes. On the downside — no ICM calculations and no full-fledged preflop solutions.

MonkerSolver, meanwhile, became the first to support Omaha simulations. It also allows calculating solutions for multiway pots. However, this program has a very complex interface, a steep price of $500, and very high hardware requirements. Calculating preflop spots requires at least 64 GB of RAM, and for 4-max — up to 256 GB.

Then in 2021, GTO Wizard entered the market, which is now the most popular solver. It sparked yet another revolution in the field by moving calculations from the user's computer to the cloud. It can be used right in the browser, without installing any additional programs on a PC. 

GTO Wizard provides millions of pre-calculated solutions with very high accuracy. In addition, the user can get a calculation for a custom spot and use hand history analysis. And over the past year, the solver has gained support for PLO and preflop multiway, including for 9-max situations.

What poker professionals think about solvers

Most successful professionals admit that in modern poker it's extremely hard to stay competitive without using solvers in training. Six-time WSOP bracelet winner Jeremy Ausmus said that at first he didn't attach much importance to these tools, but quickly realised he needed to study them. "Solvers destroyed ideas about poker that had built up over decades. I didn't rush to study GTO, relying on my raw talent instead. But closer to 2017, I started seeing the most respected players, like Stephen Chidwick, showing lines I simply didn't understand. That's when I realised I couldn't afford to wait any longer, or I'd just be left behind. I started working in a solver and reinvented my game."

Nick Petrangelo, who played against Pluribus, is convinced that a solver can help players significantly increase profitability at mid stakes. That said, the American regular stresses — to get the most benefit, you need to understand the logic behind the solutions, not just copy them.

"Suppose some recreational player plays Sunday tournaments with an ROI of minus 40-60%. All they need is to open a solver once a week, work a bit on preflop — and their EV will grow significantly. Solvers have been used in high roller tournaments since 2015, and it's hard to say that high-stakes poker has died out because of it. 

But for me there are two clear signs of a bad "good player" — when a poker player attaches too much importance to their opponent's small mistakes in GTO play, and when they blindly copy solvers without understanding at all why they're doing it." 

Although there are skeptics of solvers and GTO too. WSOP bracelet record holder Phil Hellmuth still relies more on his "white magic". After beating Daniel Negreanu in High Stakes Duel, he said outright that he had no intention of adapting his style to new trends in poker. 

Back in 2011, Tom Dwan didn't believe in the success of solvers at all: "Maybe the Nash equilibrium is wrong — I don't understand it at all — or the reasons might lie in something else, but I know for certain that there's no optimal, unexploitable strategy in no-limit hold'em heads-up. I'm sure I could easily prove this to a smart person with rational thinking in a face-to-face meeting."

Dwan was so confident he was right back then that he nearly signed up for an insane bet. The American professional stated that within 10-15 years he'd be willing to play a heads-up match against a GTO bot over 500,000 no-limit hold'em hands. Alex Millar offered a $100,000 bet on Dwan losing, but Tom never accepted the wager.

The impact of solvers on poker

The widespread adoption of solvers has undoubtedly raised the overall level of players. The subtleties of bluffing and sizing selection, previously known only to experienced and seasoned pros, have become accessible to anyone willing to devote time to studying the game. Studying strategy itself has become an integral part of a regular's work. Now it's necessary to spend hours away from the tables in order to remain competitive. 

Of course, it's impossible to learn poker with solvers alone. Before moving on to these programs, you need a solid foundation. You can get one on the free FF Start course, where you'll get acquainted with the basic principles of GTO strategy. And at the next stages, solvers will become a reliable tool for your progress.