Poker-AI.org

Hi!

So, it seems apparent that at the very least you need to use Monte Carlo tree search in combination with the counterfactual regret minimization algorithm to create a good bot but what isn't clear to me is what kind of hardware specs would be required to pull off something like this. How fancy does your hardware need to be exactly? Can someone who has successfully implemented a bot using these two techniques let me know about minimum and recommended hardware specs to get the job done?

Depth limited solving is important to creating a strong and robust agent, and there are currently two approaches. The goal is to solve until some depth limit (usually the end of the round), and get values for each hand given both players range at that point instead of having to solve until the end of the game. The first approach trained counterfactual value nets. Agents like Deepstack, Supremus, and Rebel use this type of approach. The second approach is to use multiivalued states. You first create a set of 'blueprint' strategies to choose from at the depth limit. The choice of this strategy is essentially just another node in the game tree, and is treated identically to a spot where you are making a decision between folding/call/raise. The choice is made on each iteration (so each strategy has a regret), and the choice is made separately for each hand. This is used by Modicum and Pluribus.

Multivalued states far outperform counterfactual value nets when you have limited resources. You can train a superhuman headsup agent on only a few gigabytes of ram. 6max is much more expensive, but can still be done on consumer hardware.

Monte carlo tree search is not currently used in any top agent. They usually just solve until the end of the round (pluribus may only solve until something like the second raise in the round, the specifics are in the paper).

Oh I see, thanks for your input! I'm still confused about the process and looking into it. I guess the tree is too big to search through so you need to limit the depth.

Very promising to hear that this can be done on consumer hardware!

Building a bot using Monte Carlo tree search and counterfactual regret minimization is definitely an exciting challenge.