| Poker-AI.org http://poker-ai.org/phpbb/ |
|
| Rating system http://poker-ai.org/phpbb/viewtopic.php?f=22&t=2792 |
Page 1 of 1 |
| Author: | Sailor Roberts [ Thu Aug 07, 2014 1:39 am ] |
| Post subject: | Rating system |
Has anyone figured out some objective way to measure the relative playing strength of bots? The ELO rating system works pretty well for games like chess and table tennis, but it doesn't map onto poker very well for obvious reasons. ROI (total bets / total winnings) is another simple measure, but I see problems there, too. Any research papers on this topic? |
|
| Author: | spears [ Thu Aug 07, 2014 7:50 am ] |
| Post subject: | Re: Rating system |
I haven't come across much about it. Most of the research is centred around NE bots and the rating is the win rate. They typically play zillions of hands, duplicate, to get statistically valid results, and even then quote error margins. Personally, I think poker research would be much more interesting and relevant if the competitions were run like skillbet http://en.wikipedia.org/wiki/Duplicate_ ... lBet_poker. I suggested it to UoA and got blown off |
|
| Author: | Sailor Roberts [ Fri Aug 08, 2014 2:42 pm ] |
| Post subject: | Re: Rating system |
spears wrote: I haven't come across much about it. Most of the research is centred around NE bots and the rating is the win rate. It's a tricky problem, for sure. Win rate doesn't seem like a good rating for the simple reason that the goal of poker isn't to win a lot of hands, but rather to win the most money. Of course a lot would depend on event structure, i.e., freezeout tournament or ring game? heads-up or multiway? limit or no-limit? ... and so on. spears wrote: They typically play zillions of hands, duplicate, to get statistically valid results, and even then quote error margins. Duplicate matches require memory wipes of all bots in between matches, right? But how do event organizers verify the wipe? Do they inspect the source code of each bot before and after each match? Do they use the honor system? spears wrote: Personally, I think poker research would be much more interesting and relevant if the competitions were run like skillbet http://en.wikipedia.org/wiki/Duplicate_ ... lBet_poker. I suggested it to UoA and got blown off Makes sense. Any interest in organizing an alternate competition? |
|
| Author: | spears [ Fri Aug 08, 2014 3:14 pm ] |
| Post subject: | Re: Rating system |
Sailor Roberts wrote: It's a tricky problem, for sure. Win rate doesn't seem like a good rating for the simple reason that the goal of poker isn't to win a lot of hands, but rather to win the most money. I meant the amount won per zillion hands.Sailor Roberts wrote: Duplicate matches require memory wipes of all bots in between matches, right? But how do event organizers verify the wipe? Do they inspect the source code of each bot before and after each match? Do they use the honor system? I think they reload a VM image and the bot software from CD Sailor Roberts wrote: Any interest in organizing an alternate competition? Interested but have no time. Its other great advantage is that you don't have to hand over your code.
|
|
| Author: | Sailor Roberts [ Fri Aug 15, 2014 11:40 pm ] |
| Post subject: | Re: Rating system |
spears wrote: Its other great advantage is that you don't have to hand over your code. Exactly. Hmmm |
|
| Page 1 of 1 | All times are UTC |
| Powered by phpBB® Forum Software © phpBB Group http://www.phpbb.com/ |
|