Poker-AI.org

More theoretical stuff here I'm afraid.

If we've got some optimal convergence algorithm that uses chance based sampling and has multiple bet amounts as possible actions, is that likely to skew the bot towards being too aggressive due to variance? In my head, it is, if not, why not?

Here's my thought process
Let's say we have our possible actions as Fold, Check/Call and then 100000 different bet sizes (extreme example here), due to variance, on marginal hands the chances are one of of those bet actions will skew towards being more +EV than the passive actions as there are so many of them.

If this is the case, can we measure it? What can we do about it?

yes , maybe with 100000 different bet sizes, if the best action is checking, the bot will bet 1.01 or 1.05 blind instead of checking but it is not a big problem because the difference is not great if the pot is big.

Wouldn't each action EV average out over time, assuming you keep a history?

Maybe you could do something like Public Chance Sampling?

This answer is theoretical too since I don't really know


Are you saying that for a particular bet amount there are too few iterations to achieve convergence?


There is surely a relationship between sample size, variance and error. Maybe you could get a estimate of that from examining the history of convergence of more frequently visited nodes.


- increase the number of iterations
- reduce the number of bet actions
- concentrate the iterations on the area of interest
- smooth the results over "nearby" points using some sort of machine learning. Depending on your abstraction, "nearby" might be hard to define. Since performance is likely to be an issue maybe something like this http://webdocs.cs.ualberta.ca/~sutton/b ... ode88.html

For a particular bet amount there are too few iterations to eliminate variance - obviously we'd need infinite iterations for that.

This is a possibility, although you still get variance when measuring variance. Therefore it still couldn't be eliminated.

This would work, but is a large performance hit on the speed of convergence. Especially if we increased it to the required infinity.

This would help, although makes us more exploitable.

Already doing this

Again, we've got performance issues.

It sounds like it's the usual speed vs quality of result problem... It also sounds like there's not a huge amount that can be done about it.

Does your abstraction allow you to define "nearby"? If not, maybe a solution is to find an abstraction that does.

I can get nearby without too much difficulty, but to do so would increase memory consumption as I'll need to store parent pointers back through the tree. Which obviously isn't ideal when trying to conserve memory.

To be honest, I'm not massively concerned - as neither I, not anyone else is going to generate a perfect 6 max solution (probably in my lifetime), it's just something I wanted to be aware of.

I'm storing floats using 8 bits for gods sake... I really shouldn't be worrying about precision!

How are you managing that?

All EVs on a single node are usually in a fairly small range, so I use an byte multiplier and offset on the node, and the store each float as byte, applying the multiplier and offset when I need to convert it into a workable number.

A bit like having a shared exponent between multiple float values I suppose.

Don't you lose a great deal of decimal accuracy in that situation?

Not as much as I'd lose if I had to quarter the size of my game tree.

How do you know that for sure?

I don't

In my brief experiments, game tree increase at the cost of decimal reduction didn't improve performance.

Of course, that was with CFRM, where the gradient topology was described via those decimals.