Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 2:25 pm

All times are UTC




Post new topic Reply to topic  [ 41 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
PostPosted: Wed Apr 24, 2013 9:51 pm 
Offline
Senior Member

Joined: Mon Mar 11, 2013 10:24 pm
Posts: 216
somehomelessguy wrote:
proud2bBot wrote:
no, one action/bucket combo needs 2 doubles = 16 byte. 3 (actions) * 20000 (buckets) * 16 byte = 960000 byte = 0,915MB. Multiply that with the numer of decisions and I'd be surprised if your notebook can handle it... Regarding the tree building: for smaller trees its fine, for bigger ones you might want to generate it programmatically.


i.e. 500.

you made the same calculation he did, all though you used some weird definition of mega :ugeek:


OP estimated "this makes about 10M decisions nodes" - this is a little bit different... And the "weird definition" is pretty standard when talking about storage capacity ;)


Top
 Profile  
 
PostPosted: Tue Apr 30, 2013 10:18 am 
Offline
Regular Member

Joined: Sun Mar 03, 2013 11:55 am
Posts: 64
The size of a megabyte generally depends on whether you're buying or selling it.

Any looking at optimal bots, I always find you need more RAM than you think.

8 core 32GB machine, here we come!


Top
 Profile  
 
PostPosted: Sat May 04, 2013 7:43 am 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
I'm now starting the implementation, I will first go with external sampling, from the paper Monte Carlo Sampling for Regret Minimization in Extensive Games
However they seem to assume everywhere that their game is perfect recall, but they say at the end:
Quote:
Lastly, it seems like this work could be naturally extended to cases where we don’t assume perfect recall. Imperfect recall could be used as a mechanism for abstraction over actions, where information sets are grouped by important partial sequences rather than their full sequences.

Can I use their work as is? Or do I need to pay extra attention to the fact that my game is played on an (acyclic) graph rather than on a tree?


Top
 Profile  
 
PostPosted: Sat May 04, 2013 8:06 am 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
Might this be a problem?
Quote:
Let ZI be the subset of all terminal histories where a prefix of the history is in the set I; for
z ∈ ZI let z[I] be that prefix. Since we are restricting ourselves to perfect recall games z[I] is
unique. Define counterfactual value vi(σ, I) as [...]


Top
 Profile  
 
PostPosted: Mon May 06, 2013 7:32 pm 
Offline
New Member

Joined: Sat May 04, 2013 9:04 pm
Posts: 1
About the hs and hs^2 metric, from what I understand this is simply the probability of winning the hand against a random hand. Wouldn't it make more sense to use hs or hs^2 (or hs^numberfromopponentmodel?) against the hands the opponent is most likely holding?

The idea of using more natural buckets sounds like using 'features' instead of buckets, creating some huge hyperdimensional feature space. You'll probably want to use some clustering algorithm to reduce dimensionality again (which may or may not bring you close to the hs^2 metric?).

How should one handle bet sizes for NL? Adding different pot%s to the histories seems straightforward but won't that make the tree grow exponentially? What memory would be needed (I have 2 16G machines and willing to upgrade to 128 :lol: )?

Has anyone here already posted code (c#?) or a good paper for a basic CFRM algorithm for HUNL? It sure would be helpful for us newbies :)


Top
 Profile  
 
PostPosted: Sat May 11, 2013 7:15 pm 
Offline
Site Admin
User avatar

Joined: Thu Feb 28, 2013 5:24 pm
Posts: 230
Check the poker papers section here and especially in the archives for all important poker papers on CFRM and abstraction techniques. All of your questions have been discussed in those basically.

_________________
Cheers.


Top
 Profile  
 
PostPosted: Thu May 16, 2013 1:25 pm 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
betwin, i think i saw an implementation in c# for rhode island, but i don't remember where

My implementation is now working for basic preflop games, now onto the hard work!
  • external sampling doesn't seem to converge very rapidly, even after 1 billion iteration, strategies for simple 3bet-4bet-5bet games have some (small) regions of hands where the strategy isn't very clear. According to you, what should i improve next?
  • the strategies sometimes have undefined or non-zero raising frequencies for, for instance, 5betting, while the frequency is 0 for 3betting. This looks normal because a node can be trained in the beginning even though it becomes non reachable after. How can I safely modify the cumulative strategies so that for each hand, frequency(3bet) equals 0 implies that frequency(5bet) equals 0? I mean, doing a simple traversal and setting everything to 0 might not be safe, because it could happen that the frequency of 3betting gets higher than 0 at some point?
  • for postflop, if I use the "intuistionistic" hand strength, then I'll have to precompute LUT for determining the equity of going all-in for any match up and for any street, right? Basically that's like for preflop when we're using a table giving us the equity of QJs vs ATo etc.


Also, how legal do you think this is (for big sites like ps or ftp)? I guess computing the strategies and looking at them/learning them is fine, but not while playing right? It's like looking at push/fold nash equilibrium tables while playing, probably forbidden too isn't it?


Top
 Profile  
 
PostPosted: Thu May 16, 2013 2:56 pm 
Offline
Junior Member

Joined: Mon Apr 22, 2013 11:46 am
Posts: 34
febbriz wrote:

Also, how legal do you think this is (for big sites like ps or ftp)? I guess computing the strategies and looking at them/learning them is fine, but not while playing right? It's like looking at push/fold nash equilibrium tables while playing, probably forbidden too isn't it?


just keep the solution on another computer? oh, and make sure you put some cloth over your webcam if you have one. and tinfoil the walls in your grinding room.


Top
 Profile  
 
PostPosted: Thu May 16, 2013 5:14 pm 
Offline
Junior Member

Joined: Sun Mar 10, 2013 10:30 am
Posts: 20
febbriz wrote:
betwin, i think i saw an implementation in c# for rhode island, but i don't remember where


It is here : http://poker-ai.org/archive/www.pokerai.org/pf3/viewtopicba91.html?f=64&t=2922&st=0&sk=t&sd=a&start=40


Top
 Profile  
 
PostPosted: Tue May 28, 2013 8:39 am 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
thanks

for external sampling, i thought i needed to have lookup tables storing the probability of a bucket winning against another bucket when going all-in, and this for each street, but actually i may not need those

at each iteration, i should just draw 9 random cards, check in which bucket they are, and if the hand goes all-in on the flop, give the whole pot to the winner, right?

my program is in c++, which hand evaluator should i use to determine the winner? (something which is not very memory demanding)
and what is the most efficient way to draw 9 random cards? right now i just have an array of bool storing which cards were picked and then pick a random card until i have 9 (distinct ones)


Top
 Profile  
 
PostPosted: Fri Jul 05, 2013 9:01 am 
Offline
New Member

Joined: Fri Jul 05, 2013 8:58 am
Posts: 3
Location: hefeifeicuilu
However I'm still having trouble understanding the EHS and EHS2 history bucketing thing

_________________
I have been a FunForFreedom survivor since 7/2/12!Runescape 2007 Gold
diablo 3 items Runescape Gold
Supporting Leon X Zara! <3
I have a blog!


Top
 Profile  
 
PostPosted: Sat Jul 20, 2013 7:22 pm 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
My implementation looks ok now, but it can only make 5k iterations/second while there was a thread where people were claiming more than 1M ips. Even though it was on different machines, I guess I still have a lot of room for improvement.

Right now, at each iteration, my algorithm draws 9 random cards, and computes the winner using a c++ hand evaluator as well as the homemade hand buckets with a homemade function. Instead of this last homemade function, i should probably have a huge look-up table, right? I guess it could fit in memory if the LUT contains about 40k*169 entries (for all boards*holecards) storing an index presenting the buckets (at most 20 bits)

Where else should I get LUTs to improve performance?


Top
 Profile  
 
PostPosted: Sat Jul 20, 2013 9:57 pm 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
febbriz wrote:
My implementation looks ok now, but it can only make 5k iterations/second while there was a thread where people were claiming more than 1M ips. Even though it was on different machines, I guess I still have a lot of room for improvement.

Right now, at each iteration, my algorithm draws 9 random cards, and computes the winner using a c++ hand evaluator as well as the homemade hand buckets with a homemade function. Instead of this last homemade function, i should probably have a huge look-up table, right? I guess it could fit in memory if the LUT contains about 40k*169 entries (for all boards*holecards) storing an index presenting the buckets (at most 20 bits)

Where else should I get LUTs to improve performance?


There are 16432 turns and 1127 holecards and lots of info on LUTS if you search. The next big benefit comes from removing recursion.
http://www.poker-ai.org/archive/www.pok ... view=print


Top
 Profile  
 
PostPosted: Sun Jul 21, 2013 9:59 am 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
Yes thanks that's the topic I was referring to
I think I'll make directly LUTs for the river, by using the ideas in
http://www.poker-ai.org/archive/www.pok ... f=3&t=1989

But just to be sure before starting, I need a LUT : holecards + board -> "number given by some hand evaluator", and a LUT : holecards + board -> "homemade buckets", right?

thanks


Top
 Profile  
 
PostPosted: Sun Jul 21, 2013 12:40 pm 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
febbriz wrote:
I think I'll make directly LUTs for the river, by using the ideas in
http://www.poker-ai.org/archive/www.pok ... f=3&t=1989

Your ref is good for flop and turn. A better approach for river is viewtopic.php?f=24&t=2485&view=previous

febbriz wrote:
I need a LUT : holecards + board -> "number given by some hand evaluator", and a LUT : holecards + board -> "homemade buckets", right?

I think you just need the second, but I don't know for sure.


Top
 Profile  
 
PostPosted: Sun Jul 21, 2013 2:08 pm 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
Ok thanks for the link

If I just have the buckets, how can I deduce the utility function?
I'd need a LUT telling me for each pair of flops buckets (resp. turn buckets, resp. river buckets), what is the EV

But if I know the winner for this iteration, I can just say that for this round, player X wins the whole pot


Top
 Profile  
 
PostPosted: Sun Jul 21, 2013 2:26 pm 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
febbriz wrote:
Ok thanks for the link

If I just have the buckets, how can I deduce the utility function?
I'd need a LUT telling me for each pair of flops buckets (resp. turn buckets, resp. river buckets), what is the EV

But if I know the winner for this iteration, I can just say that for this round, player X wins the whole pot

Just follow the papers and amax's nice code posted here http://poker-ai.org/archive/www.pokerai ... 335#p40335


Top
 Profile  
 
PostPosted: Sun Jul 21, 2013 3:10 pm 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
Ok thanks

They also use a LUT for hand rank in the code; it's going to be Huge!


Top
 Profile  
 
PostPosted: Sun Jul 21, 2013 5:50 pm 
Offline
Junior Member

Joined: Sat Apr 06, 2013 1:21 pm
Posts: 18
Is there a full LUT freely available for: (holecards, board) -> strength?
And a function which takes 7 cards and computes the canonical form (there should be 1127*42769 possible canonical forms is that right?) that the LUT uses as keys


Top
 Profile  
 
PostPosted: Sun Jul 21, 2013 8:45 pm 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
febbriz wrote:
They also use a LUT for hand rank in the code; it's going to be Huge!
You haven't seen anything yet

febbriz wrote:
Is there a full LUT freely available for: (holecards, board) -> strength?
And a function which takes 7 cards and computes the canonical form that the LUT uses as keys

No. Don't be lazy

febbriz wrote:
there should be 1127*42769 possible canonical forms is that right?
A little less I think


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 41 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group