Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 4:49 pm

All times are UTC




Post new topic Reply to topic  [ 4 posts ] 
Author Message
PostPosted: Sat Mar 13, 2021 4:30 pm 
Offline
Junior Member

Joined: Sat Mar 13, 2021 3:43 pm
Posts: 23
Hi!

So, it seems apparent that at the very least you need to use Monte Carlo tree search in combination with the counterfactual regret minimization algorithm to create a good bot but what isn't clear to me is what kind of hardware specs would be required to pull off something like this. How fancy does your hardware need to be exactly? Can someone who has successfully implemented a bot using these two techniques let me know about minimum and recommended hardware specs to get the job done?


Top
 Profile  
 
PostPosted: Mon Mar 22, 2021 12:59 am 
Offline
New Member

Joined: Fri Mar 12, 2021 3:53 pm
Posts: 9
Depth limited solving is important to creating a strong and robust agent, and there are currently two approaches. The goal is to solve until some depth limit (usually the end of the round), and get values for each hand given both players range at that point instead of having to solve until the end of the game. The first approach trained counterfactual value nets. Agents like Deepstack, Supremus, and Rebel use this type of approach. The second approach is to use multiivalued states. You first create a set of 'blueprint' strategies to choose from at the depth limit. The choice of this strategy is essentially just another node in the game tree, and is treated identically to a spot where you are making a decision between folding/call/raise. The choice is made on each iteration (so each strategy has a regret), and the choice is made separately for each hand. This is used by Modicum and Pluribus.

Multivalued states far outperform counterfactual value nets when you have limited resources. You can train a superhuman headsup agent on only a few gigabytes of ram. 6max is much more expensive, but can still be done on consumer hardware.

Monte carlo tree search is not currently used in any top agent. They usually just solve until the end of the round (pluribus may only solve until something like the second raise in the round, the specifics are in the paper).


Top
 Profile  
 
PostPosted: Tue Mar 23, 2021 1:29 am 
Offline
Junior Member

Joined: Sat Mar 13, 2021 3:43 pm
Posts: 23
llllllllll wrote:
Monte carlo tree search is not currently used in any top agent. They usually just solve until the end of the round (pluribus may only solve until something like the second raise in the round, the specifics are in the paper).


Oh I see, thanks for your input! I'm still confused about the process and looking into it. I guess the tree is too big to search through so you need to limit the depth.

Very promising to hear that this can be done on consumer hardware!


Top
 Profile  
 
PostPosted: Tue Oct 17, 2023 11:05 am 
Offline
New Member

Joined: Tue Oct 17, 2023 11:05 am
Posts: 1
Building a bot using Monte Carlo tree search and counterfactual regret minimization is definitely an exciting challenge.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
cron
Powered by phpBB® Forum Software © phpBB Group