Poker-AI.org

Re: Sampling, multiple bet amounts and variance...?

2013-08-05T11:17:24+00:00

In my brief experiments, game tree increase at the cost of decimal reduction didn't improve performance.

Of course, that was with CFRM, where the gradient topology was described via those decimals.

Statistics: Posted by cantina — Mon Aug 05, 2013 11:17 am

Re: Sampling, multiple bet amounts and variance...?

2013-08-05T11:07:39+00:00

Nasher wrote:

How do you know that for sure?

I don't

Statistics: Posted by OneDayItllWork — Mon Aug 05, 2013 11:07 am

Re: Sampling, multiple bet amounts and variance...?

2013-08-05T10:37:28+00:00

How do you know that for sure?

Statistics: Posted by cantina — Mon Aug 05, 2013 10:37 am

Re: Sampling, multiple bet amounts and variance...?

2013-08-05T09:37:14+00:00

Nasher wrote:

Don't you lose a great deal of decimal accuracy in that situation?

Not as much as I'd lose if I had to quarter the size of my game tree.

Statistics: Posted by OneDayItllWork — Mon Aug 05, 2013 9:37 am

Re: Sampling, multiple bet amounts and variance...?

2013-08-04T22:21:45+00:00

Don't you lose a great deal of decimal accuracy in that situation?

Statistics: Posted by cantina — Sun Aug 04, 2013 10:21 pm

Re: Sampling, multiple bet amounts and variance...?

2013-08-04T20:31:15+00:00

Nasher wrote:

OneDayItllWork wrote:

I'm storing floats using 8 bits for gods sake... I really shouldn't be worrying about precision!

How are you managing that?

All EVs on a single node are usually in a fairly small range, so I use an byte multiplier and offset on the node, and the store each float as byte, applying the multiplier and offset when I need to convert it into a workable number.

A bit like having a shared exponent between multiple float values I suppose.

Statistics: Posted by OneDayItllWork — Sun Aug 04, 2013 8:31 pm

Re: Sampling, multiple bet amounts and variance...?

2013-08-03T12:36:47+00:00

OneDayItllWork wrote:

I'm storing floats using 8 bits for gods sake... I really shouldn't be worrying about precision!

How are you managing that?

Statistics: Posted by cantina — Sat Aug 03, 2013 12:36 pm

Re: Sampling, multiple bet amounts and variance...?

2013-08-02T14:36:07+00:00

I can get nearby without too much difficulty, but to do so would increase memory consumption as I'll need to store parent pointers back through the tree. Which obviously isn't ideal when trying to conserve memory.

To be honest, I'm not massively concerned - as neither I, not anyone else is going to generate a perfect 6 max solution (probably in my lifetime), it's just something I wanted to be aware of.

I'm storing floats using 8 bits for gods sake... I really shouldn't be worrying about precision!

Statistics: Posted by OneDayItllWork — Fri Aug 02, 2013 2:36 pm

Re: Sampling, multiple bet amounts and variance...?

2013-08-02T11:19:04+00:00

Does your abstraction allow you to define "nearby"? If not, maybe a solution is to find an abstraction that does.

Statistics: Posted by spears — Fri Aug 02, 2013 11:19 am

Re: Sampling, multiple bet amounts and variance...?

2013-08-02T10:30:59+00:00

spears wrote:

Are you saying that for a particular bet amount there are too few iterations to achieve convergence?

For a particular bet amount there are too few iterations to eliminate variance - obviously we'd need infinite iterations for that.

spears wrote:

There is surely a relationship between sample size, variance and error. Maybe you could get a estimate of that from examining the history of convergence of more frequently visited nodes.

This is a possibility, although you still get variance when measuring variance. Therefore it still couldn't be eliminated.

Quote:

- increase the number of iterations

This would work, but is a large performance hit on the speed of convergence. Especially if we increased it to the required infinity.

Quote:

- reduce the number of bet actions

This would help, although makes us more exploitable.

Quote:

- concentrate the iterations on the area of interest

Already doing this

Quote:

- smooth the results over "nearby" points using some sort of machine learning. Depending on your abstraction, "nearby" might be hard to define. Since performance is likely to be an issue maybe something like this http://webdocs.cs.ualberta.ca/~sutton/b ... ode88.html

Again, we've got performance issues.

It sounds like it's the usual speed vs quality of result problem... It also sounds like there's not a huge amount that can be done about it.

Statistics: Posted by OneDayItllWork — Fri Aug 02, 2013 10:30 am

Re: Sampling, multiple bet amounts and variance...?

2013-08-01T10:42:04+00:00

OneDayItllWork wrote:

More theoretical stuff here I'm afraid.

This answer is theoretical too since I don't really know

OneDayItllWork wrote:

If we've got some optimal convergence algorithm that uses chance based sampling and has multiple bet amounts as possible actions, is that likely to skew the bot towards being too aggressive due to variance? In my head, it is, if not, why not? Let's say we have our possible actions as Fold, Check/Call and then 100000 different bet sizes (extreme example here), due to variance, on marginal hands the chances are one of of those bet actions will skew towards being more +EV than the passive actions as there are so many of them.

Are you saying that for a particular bet amount there are too few iterations to achieve convergence?

OneDayItllWork wrote:

If this is the case, can we measure it?

There is surely a relationship between sample size, variance and error. Maybe you could get a estimate of that from examining the history of convergence of more frequently visited nodes.

OneDayItllWork wrote:

What can we do about it?

- increase the number of iterations
- reduce the number of bet actions
- concentrate the iterations on the area of interest
- smooth the results over "nearby" points using some sort of machine learning. Depending on your abstraction, "nearby" might be hard to define. Since performance is likely to be an issue maybe something like this http://webdocs.cs.ualberta.ca/~sutton/b ... ode88.html

Statistics: Posted by spears — Thu Aug 01, 2013 10:42 am

Re: Sampling, multiple bet amounts and variance...?

2013-08-01T10:27:13+00:00

Wouldn't each action EV average out over time, assuming you keep a history?

Maybe you could do something like Public Chance Sampling?

Statistics: Posted by cantina — Thu Aug 01, 2013 10:27 am

Re: Sampling, multiple bet amounts and variance...?

2013-07-31T18:29:02+00:00

yes , maybe with 100000 different bet sizes, if the best action is checking, the bot will bet 1.01 or 1.05 blind instead of checking but it is not a big problem because the difference is not great if the pot is big.

Statistics: Posted by Isildur11 — Wed Jul 31, 2013 6:29 pm

Sampling, multiple bet amounts and variance...?

2013-07-31T15:30:56+00:00

More theoretical stuff here I'm afraid.

If we've got some optimal convergence algorithm that uses chance based sampling and has multiple bet amounts as possible actions, is that likely to skew the bot towards being too aggressive due to variance? In my head, it is, if not, why not?

Here's my thought process
Let's say we have our possible actions as Fold, Check/Call and then 100000 different bet sizes (extreme example here), due to variance, on marginal hands the chances are one of of those bet actions will skew towards being more +EV than the passive actions as there are so many of them.

If this is the case, can we measure it? What can we do about it?

Statistics: Posted by OneDayItllWork — Wed Jul 31, 2013 3:30 pm