Poker-AI.org
http://poker-ai.org/phpbb/

Over-fitting?
http://poker-ai.org/phpbb/viewtopic.php?f=24&t=2887		 Page 1 of 1
Author:  Alan Napier [ Tue Mar 03, 2015 11:03 pm ]
Post subject:  Over-fitting?
I made a simple bot (Bot A)
card abstraction: 169-5000-5000-5000
algorithm: PCFR
clustering: k-means
type: distribution aware (50 bins), on river OCHS
iterations: 2.081 billions

Then I changed it's flop abstraction from 5000 to 10000 clusters. (Bot B), and matched it against Bot A.
1 000 000 hands, same rng seed. Here is the result.

Code:
iterations (billions)| result (mbb/hand)   |
+--------------------+---------------------+
|     1.538          |       -12.4895      |
+--------------------+---------------------+
|     2.536          |        -3.8870      |
+--------------------+---------------------+
|     3.129          |        -2.6320      |
+--------------------+---------------------+
|     3.402          |        -2.8180      |   
+--------------------+---------------------+
|     3.715          |        -5.1305      |
+--------------------+---------------------+


I expected, that Bot B needs more iterations to converge, but after 3.129 billion iterations it results start to drop, and it never beats Bot A. I have to add, that i run the flop clustering in both case (5000 and 10000 clusters) only one time. I used k-means++ initialization which has a certain luck factor. It could explain, the poor result (because bad clustering), but not the decreasing tendency.

Have you ever experimented something like this?
Author:  spears [ Wed Mar 04, 2015 6:40 am ]
Post subject:  Re: Over-fitting?
http://poker.cs.ualberta.ca/publication ... sponse.pdf figure 6
Author:  nonpareil [ Wed Mar 04, 2015 12:16 pm ]
Post subject:  Re: Over-fitting?
My first instinct is to suspect a bug, especially since it started to perform much worse. I've trained strategies for 100+ billion iterations and haven't seen behavior like that. I agree it should take longer for the larger abstraction to train. That said, just going from 5k to 10k flop buckets in (limit?) hold'em, I wouldn't expect a huge improvement, just a few milli big blinds. Make sure your testing is accurate, doing like 10+ million duplicate hands. It also helps if you can get some stats on the strategy... e.g. compare their preflop opening ranges, postflop aggression factor, etc.

There's some randomness with k-means++ seeding but the clustering quality doesn't seem to vary that widely. You can test by calculating the average distance squared from each point to its center, and with 10k buckets I expect you'll always get a smaller value than with 5k buckets, which is an improvement.
Author:  algonoob [ Wed Mar 04, 2015 6:12 pm ]
Post subject:  Re: Over-fitting?
there's definitely a bug
check your flop abstraction code
Author:  Alan Napier [ Sat Mar 07, 2015 11:01 am ]
Post subject:  Re: Over-fitting?
Thank you all for the reply. After running more matches, and make them DUPLICATE, I have some whole different result:

Image

Legend:
A: iterations (billion)
B: Match A result (mbb/hand)
C: Match B result (mbb/hand)
D: B + C

Does it look more normal? The best version of Bot B beats for only 13.38 mbb/hand (or 1.34 BB/100) Bot A.
I would expect more performance boost after dubbeling the number of flop clusters.

The graph seems simular to that one, what was linked by spears. (only opposit)
Have you experienced the same drop after a peak? I would expect a constans increase with increasing the number of iterations.
Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/