Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 12:29 pm

All times are UTC




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: Over-fitting?
PostPosted: Tue Mar 03, 2015 11:03 pm 
Offline
Junior Member

Joined: Wed Oct 01, 2014 6:59 pm
Posts: 11
I made a simple bot (Bot A)
card abstraction: 169-5000-5000-5000
algorithm: PCFR
clustering: k-means
type: distribution aware (50 bins), on river OCHS
iterations: 2.081 billions

Then I changed it's flop abstraction from 5000 to 10000 clusters. (Bot B), and matched it against Bot A.
1 000 000 hands, same rng seed. Here is the result.

Code:
iterations (billions)| result (mbb/hand)   |
+--------------------+---------------------+
|     1.538          |       -12.4895      |
+--------------------+---------------------+
|     2.536          |        -3.8870      |
+--------------------+---------------------+
|     3.129          |        -2.6320      |
+--------------------+---------------------+
|     3.402          |        -2.8180      |   
+--------------------+---------------------+
|     3.715          |        -5.1305      |
+--------------------+---------------------+


I expected, that Bot B needs more iterations to converge, but after 3.129 billion iterations it results start to drop, and it never beats Bot A. I have to add, that i run the flop clustering in both case (5000 and 10000 clusters) only one time. I used k-means++ initialization which has a certain luck factor. It could explain, the poor result (because bad clustering), but not the decreasing tendency.

Have you ever experimented something like this?


Top
 Profile  
 
 Post subject: Re: Over-fitting?
PostPosted: Wed Mar 04, 2015 6:40 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
http://poker.cs.ualberta.ca/publication ... sponse.pdf figure 6


Top
 Profile  
 
 Post subject: Re: Over-fitting?
PostPosted: Wed Mar 04, 2015 12:16 pm 
Offline
Junior Member

Joined: Wed Jul 17, 2013 10:52 pm
Posts: 21
My first instinct is to suspect a bug, especially since it started to perform much worse. I've trained strategies for 100+ billion iterations and haven't seen behavior like that. I agree it should take longer for the larger abstraction to train. That said, just going from 5k to 10k flop buckets in (limit?) hold'em, I wouldn't expect a huge improvement, just a few milli big blinds. Make sure your testing is accurate, doing like 10+ million duplicate hands. It also helps if you can get some stats on the strategy... e.g. compare their preflop opening ranges, postflop aggression factor, etc.

There's some randomness with k-means++ seeding but the clustering quality doesn't seem to vary that widely. You can test by calculating the average distance squared from each point to its center, and with 10k buckets I expect you'll always get a smaller value than with 5k buckets, which is an improvement.


Top
 Profile  
 
 Post subject: Re: Over-fitting?
PostPosted: Wed Mar 04, 2015 6:12 pm 
Offline
Junior Member

Joined: Thu May 23, 2013 11:35 pm
Posts: 23
there's definitely a bug
check your flop abstraction code


Top
 Profile  
 
 Post subject: Re: Over-fitting?
PostPosted: Sat Mar 07, 2015 11:01 am 
Offline
Junior Member

Joined: Wed Oct 01, 2014 6:59 pm
Posts: 11
Thank you all for the reply. After running more matches, and make them DUPLICATE, I have some whole different result:

Image

Legend:
A: iterations (billion)
B: Match A result (mbb/hand)
C: Match B result (mbb/hand)
D: B + C

Does it look more normal? The best version of Bot B beats for only 13.38 mbb/hand (or 1.34 BB/100) Bot A.
I would expect more performance boost after dubbeling the number of flop clusters.

The graph seems simular to that one, what was linked by spears. (only opposit)
Have you experienced the same drop after a peak? I would expect a constans increase with increasing the number of iterations.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
cron
Powered by phpBB® Forum Software © phpBB Group