Poker-AI.org

Re: Rating system

2014-08-15T23:40:46+00:00

spears wrote:

Its other great advantage is that you don't have to hand over your code.

Exactly. Hmmm

Statistics: Posted by Sailor Roberts — Fri Aug 15, 2014 11:40 pm

Re: Rating system

2014-08-08T15:14:52+00:00

Sailor Roberts wrote:

It's a tricky problem, for sure. Win rate doesn't seem like a good rating for the simple reason that the goal of poker isn't to win a lot of hands, but rather to win the most money.

I meant the amount won per zillion hands.

Sailor Roberts wrote:

Duplicate matches require memory wipes of all bots in between matches, right? But how do event organizers verify the wipe? Do they inspect the source code of each bot before and after each match? Do they use the honor system?

I think they reload a VM image and the bot software from CD

Sailor Roberts wrote:

Any interest in organizing an alternate competition?

Interested but have no time. Its other great advantage is that you don't have to hand over your code.

Statistics: Posted by spears — Fri Aug 08, 2014 3:14 pm

Re: Rating system

2014-08-08T14:42:44+00:00

spears wrote:

I haven't come across much about it. Most of the research is centred around NE bots and the rating is the win rate.

It's a tricky problem, for sure. Win rate doesn't seem like a good rating for the simple reason that the goal of poker isn't to win a lot of hands, but rather to win the most money. Of course a lot would depend on event structure, i.e., freezeout tournament or ring game? heads-up or multiway? limit or no-limit? ... and so on.

spears wrote:

They typically play zillions of hands, duplicate, to get statistically valid results, and even then quote error margins.

spears wrote:

Personally, I think poker research would be much more interesting and relevant if the competitions were run like skillbet http://en.wikipedia.org/wiki/Duplicate_ ... lBet_poker. I suggested it to UoA and got blown off

Makes sense. Any interest in organizing an alternate competition?

Statistics: Posted by Sailor Roberts — Fri Aug 08, 2014 2:42 pm

Re: Rating system

2014-08-07T07:50:11+00:00

I haven't come across much about it. Most of the research is centred around NE bots and the rating is the win rate. They typically play zillions of hands, duplicate, to get statistically valid results, and even then quote error margins.

Personally, I think poker research would be much more interesting and relevant if the competitions were run like skillbet http://en.wikipedia.org/wiki/Duplicate_ ... lBet_poker. I suggested it to UoA and got blown off

Statistics: Posted by spears — Thu Aug 07, 2014 7:50 am

Rating system

2014-08-07T01:39:29+00:00

Has anyone figured out some objective way to measure the relative playing strength of bots? The ELO rating system works pretty well for games like chess and table tennis, but it doesn't map onto poker very well for obvious reasons. ROI (total bets / total winnings) is another simple measure, but I see problems there, too. Any research papers on this topic?

Statistics: Posted by Sailor Roberts — Thu Aug 07, 2014 1:39 am