| Poker-AI.org http://poker-ai.org/phpbb/ |
|
| The memory dilemma http://poker-ai.org/phpbb/viewtopic.php?f=24&t=2385 |
Page 1 of 1 |
| Author: | proud2bBot [ Wed Mar 13, 2013 8:52 pm ] |
| Post subject: | The memory dilemma |
Lets assume we want to solve 25bb NLH HU games using CFRM. Even with this small stack size, we still have to apply tons of abstractions in the following areas to keep the tree size managable: 1 - Holecard-Board abstraction (how do we model our hand [and the board] = size of a node) 2 - Betting abstraction (how many bet sizes and raising depth do we allow = depth of the tree) I'm currently experimenting with varying in both areas, but facing two major problems: A) evaluating takes a long time as we need to calculate a model until convergence and play the 2 models against each other B) the factors may be interdependent (e.g. the effect of allowing more betting options might be minimal if we have few hole card buckets but significant if we have a fine-grained holecard/board abstraction) Does anyone have an idea how we can evaluate these things better? If not - and you are solving a similar game - how do your abstractions look like? Mine is as follows: - 169 preflop buckets and 200 buckets on flop/turn/river based on EHS2 - allowing 4 different bet sizes and all-ins, with maximum of 5 raises per game (over all streets) I'm changing the card/board abstraction however to a non-EHS2 based one, which should be way better given we have the same number of buckets (at least the first results for the flop indicate it, as now on a board like Td8d5s, hands like Jd9d, AsAh and 8h5h are all in the same bucket, while EHS2 doesn't value monster draws too much). |
|
| Author: | cantina [ Wed Mar 13, 2013 11:50 pm ] |
| Post subject: | Re: The memory dilemma |
I've experimented with some unorthodox approaches, like using NNs to 'store' the cumulative regrets. It converged, to a point, but nowhere near anything useful. I had other unorthodox ideas as well, though I'm not as motivated to spend time implementing them. Spears is working on an approach along the same lines, with more success. As far as my abstractions, I'm working on one where the betting nodes are generated by player data. |
|
| Author: | proud2bBot [ Thu Mar 14, 2013 2:10 am ] |
| Post subject: | Re: The memory dilemma |
Nasher wrote: As far as my abstractions, I'm working on one where the betting nodes are generated by player data. What do you mean by this? Are you creating nodes based on mined "typical" bet sizes for different histories or do you restrict actions based on an opponent model (i.e., most villains only play x%, so we force them to fold 100-x%)? |
|
| Author: | cantina [ Thu Mar 14, 2013 10:34 pm ] |
| Post subject: | Re: The memory dilemma |
I have player "clones" and I'm using the various nodes that they reached through play to build the abstraction. 16 bet sizes in total as % of pot. |
|
| Page 1 of 1 | All times are UTC |
| Powered by phpBB® Forum Software © phpBB Group http://www.phpbb.com/ |
|