Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Sat Mar 28, 2020 3:25 pm

All times are UTC




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: River buckets
PostPosted: Sat Dec 14, 2019 2:41 am 
Offline
New Member

Joined: Sat Dec 14, 2019 2:30 am
Posts: 4
I'm trying to create 10,000 flop, turn, and river buckets (30,000 in total) by following the potential-aware imperfect recall paper (https://www.cs.cmu.edu/~sandholm/potent ... aaai14.pdf)

First step is to create the abstraction for the river. I'm trying to use k means to cluster hands with similar equity(vs range of all hands). I've calculated and saved the equity as a double for all 1326 hands on all 2598960 5 card boards. I'm having trouble initializing 10,000 unique and non empty buckets. I tried initializing the buckets to existing values, but its looking like there may not even be enough. The paper on Libratus said that they had >1 million river buckets, so I'm feeling quite stupid right now. Do I have a major misunderstanding or is it likely a problem with my code? I'm pretty confident that my equities are correct.

I haven't checked how many unique equities there are on the river, just basing this off the speed that it was initializing


Top
 Profile  
 
 Post subject: Re: River buckets
PostPosted: Sat Dec 14, 2019 6:16 am 
Offline
New Member

Joined: Sat Dec 14, 2019 6:16 am
Posts: 1
hey good luck with an answer


Top
 Profile  
 
 Post subject: Re: River buckets
PostPosted: Thu Dec 19, 2019 5:28 pm 
Offline
New Member

Joined: Sat Dec 14, 2019 2:30 am
Posts: 4
So river buckets aren't based on equity,

see viewtopic.php?f=25&t=2381&start=0


Top
 Profile  
 
 Post subject: Re: River buckets
PostPosted: Fri Dec 20, 2019 7:01 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 613
I never had a lot of faith in this type of abstraction and I never went down that route, so can't comment knowledgeably. I just want to make you aware if you weren't already that recent efforts seem to be moving away from this type of abstraction as a complete solution. I recommend watching the DeepStack video and reading the DeepStack and Pluribus papers. There is some doubt about how good DeepStack actually is but I think there is merit in the limited depth / value function approach. IIRC Libratus used a lot of resources.


Top
 Profile  
 
 Post subject: Re: River buckets
PostPosted: Fri Dec 20, 2019 2:40 pm 
Offline
New Member

Joined: Sat Dec 14, 2019 2:30 am
Posts: 4
My current understanding is that Modicum used these sources for to make its buckets
https://www.cs.cmu.edu/~sandholm/hierar ... amas15.pdf
https://www.cs.cmu.edu/~sandholm/potent ... aaai14.pdf
http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf
steps in this order:
paper 3: evaluating state space abstraction
cluster turn(earth movers distance over equity distribution on all rivers) and river (clustering histograms of equity vs 8 ranges on the river by using k means and euclidean distance)

paper 2: earth movers distance
bucket flops by clustering histograms of how likely they are to transition into bucket n on the turn (keeping same turn/river buckets as paper 3)

paper 1: hierarchical abstraction
cluster public flop boards (not including our hole cards) by clustering histograms of how likely flop board plus holecard transitions into bucket n on the flop(using the previously computed abstraction from paper 2)
I haven't gotten through how they then cluster public+holecards on the flop/turn/river and I may not be entirely correct on this part

I was under the impression that pluribus used the same buckets as modicum, but I could look into it


Last edited by t_steu on Fri Dec 20, 2019 6:00 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: River buckets
PostPosted: Fri Dec 20, 2019 5:04 pm 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 613
You may well be right, but my point was they seem to be moving away from this type of abstraction as a complete solution. In Pluribus the bucketed abstraction produced the relatively small blueprint solution which was used as a starting point for a more detailed local solution at run time. I commented on the resources Libratus used because you had commented on the number of buckets. So you don't have to worry about the huge number of buckets Libratus used if you use the Pluribus technique of a blueprint solution enhanced at runtime.

Thanks for mentioning Modicum. I didn't know about that.


Top
 Profile  
 
 Post subject: Re: River buckets
PostPosted: Fri Dec 20, 2019 5:52 pm 
Offline
New Member

Joined: Sat Dec 14, 2019 2:30 am
Posts: 4
Thanks for the responses,

I believe Modicum and Pluribus are very similar (weak blueprint strategy and realtime depth limited solving). I mentioned the number of buckets libratus used because I didn't understand how they were able to create >1m buckets on the river when there are <2,000 unique equities. I now have a better idea of how those papers work together. I'm actually interested in seeing if I can use lossy abstraction for a postflop solver to get an estimate ev values and then use depth limited solving to get the exploitability down. All of that in hopefully less time/resources that a traditional lossless approach would take.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group