Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 1:38 pm

All times are UTC




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: Rating system
PostPosted: Thu Aug 07, 2014 1:39 am 
Offline
New Member

Joined: Mon Aug 04, 2014 11:42 pm
Posts: 6
Has anyone figured out some objective way to measure the relative playing strength of bots? The ELO rating system works pretty well for games like chess and table tennis, but it doesn't map onto poker very well for obvious reasons. ROI (total bets / total winnings) is another simple measure, but I see problems there, too. Any research papers on this topic?


Top
 Profile  
 
 Post subject: Re: Rating system
PostPosted: Thu Aug 07, 2014 7:50 am 
Online
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
I haven't come across much about it. Most of the research is centred around NE bots and the rating is the win rate. They typically play zillions of hands, duplicate, to get statistically valid results, and even then quote error margins.

Personally, I think poker research would be much more interesting and relevant if the competitions were run like skillbet http://en.wikipedia.org/wiki/Duplicate_ ... lBet_poker. I suggested it to UoA and got blown off


Top
 Profile  
 
 Post subject: Re: Rating system
PostPosted: Fri Aug 08, 2014 2:42 pm 
Offline
New Member

Joined: Mon Aug 04, 2014 11:42 pm
Posts: 6
spears wrote:
I haven't come across much about it. Most of the research is centred around NE bots and the rating is the win rate.

It's a tricky problem, for sure. Win rate doesn't seem like a good rating for the simple reason that the goal of poker isn't to win a lot of hands, but rather to win the most money. Of course a lot would depend on event structure, i.e., freezeout tournament or ring game? heads-up or multiway? limit or no-limit? ... and so on.
spears wrote:
They typically play zillions of hands, duplicate, to get statistically valid results, and even then quote error margins.

Duplicate matches require memory wipes of all bots in between matches, right? But how do event organizers verify the wipe? Do they inspect the source code of each bot before and after each match? Do they use the honor system?
spears wrote:
Personally, I think poker research would be much more interesting and relevant if the competitions were run like skillbet http://en.wikipedia.org/wiki/Duplicate_ ... lBet_poker. I suggested it to UoA and got blown off

Makes sense. Any interest in organizing an alternate competition?


Top
 Profile  
 
 Post subject: Re: Rating system
PostPosted: Fri Aug 08, 2014 3:14 pm 
Online
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
Sailor Roberts wrote:
It's a tricky problem, for sure. Win rate doesn't seem like a good rating for the simple reason that the goal of poker isn't to win a lot of hands, but rather to win the most money.
I meant the amount won per zillion hands.

Sailor Roberts wrote:
Duplicate matches require memory wipes of all bots in between matches, right? But how do event organizers verify the wipe? Do they inspect the source code of each bot before and after each match? Do they use the honor system?

I think they reload a VM image and the bot software from CD

Sailor Roberts wrote:
Any interest in organizing an alternate competition?
Interested but have no time. Its other great advantage is that you don't have to hand over your code.


Top
 Profile  
 
 Post subject: Re: Rating system
PostPosted: Fri Aug 15, 2014 11:40 pm 
Offline
New Member

Joined: Mon Aug 04, 2014 11:42 pm
Posts: 6
spears wrote:
Its other great advantage is that you don't have to hand over your code.


Exactly. Hmmm


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group