Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Thu Nov 14, 2019 3:19 am

All times are UTC




Post new topic Reply to topic  [ 10 posts ] 
Author Message
PostPosted: Fri Jul 12, 2019 7:05 am 
Offline
Junior Member

Joined: Tue Dec 13, 2016 4:11 am
Posts: 13
Abstract
In recent years there have been great strides in artificial intelligence (AI), with games often serving as challenge problems, benchmarks, and milestones for progress. Poker has served for decades as such a challenge problem. Past successes in such benchmarks, including poker, have been limited to two-player games. However, poker in particular is traditionally played with more than two players. Multiplayer games present fundamental additional issues beyond those in two-player games, and multiplayer poker is a recognized AI milestone. In this paper we present Pluribus, an AI that we show is stronger than top human professionals in six-player no-limit Texas hold’em poker, the most popular form of poker played by humans.

https://science.sciencemag.org/content/ ... ce.aay2400


Top
 Profile  
 
PostPosted: Sat Jul 13, 2019 12:58 pm 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 600
Looks like Noam Brown has been busy. https://www.cs.cmu.edu/~noamb/research.html


Top
 Profile  
 
PostPosted: Sat Jul 13, 2019 5:37 pm 
Offline
New Member

Joined: Tue Jul 03, 2018 12:57 am
Posts: 1
Very interesting


Top
 Profile  
 
PostPosted: Sun Jul 14, 2019 7:33 pm 
Offline
Senior Member

Joined: Fri Nov 25, 2016 10:42 pm
Posts: 122
Sample size only 10k hands. Any poker player knows that 10k hands is way too small sample to make any conclusions.
The best player in the world could be losing after 10k hands, and bad player could be also winning after 10k hands.
100k would be appropriate sample size.
Also their explanations about hands played are super fishy, obviously guy who was writting it is amateur for poker.
In science results are often manipulated, i am wondering if it is the case here.


Top
 Profile  
 
PostPosted: Mon Jul 15, 2019 9:10 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 600
Quote:
Sample size only 10k hands. Any poker player knows that 10k hands is way too small sample to make any conclusions.


And the author knows this too, which is why the quoted results are those resulting from AIVAT variance reduction technique.


Quote:
In science results are often manipulated, i am wondering if it is the case here
.
I am wondering if this is sour grapes from a sore loser


Top
 Profile  
 
PostPosted: Mon Jul 15, 2019 9:19 pm 
Offline
Senior Member

Joined: Fri Nov 25, 2016 10:42 pm
Posts: 122
Spears, You don't seem to have an argument, except to insult someone who has opinion other than yours.

It doesn't matter if author is aware of 10k sample. There are so many news bot beats best humans in poker.
10k sample is far away of a proof of being better.

Their comment about hand JQ bot vs QT human, both hitting top pair, and claiming bot checks flop (without preflop initiative) to trap opponent, then again on turn it traps opponent and again on river traps opponent (totally fishy comments). Hands played by bot are fine, i am referring to their commenting on hands played is ridicilous.


Top
 Profile  
 
PostPosted: Tue Jul 16, 2019 5:46 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 600
Quote:
Spears, You don't seem to have an argument, except to insult someone who has opinion other than yours.

You can dish it out to Noam Brown
Quote:
In science results are often manipulated, i am wondering if it is the case here
but you can't take it.

Quote:
Sample size only 10k hands. Any poker player knows that 10k hands is way too small sample to make any conclusions.
The best player in the world could be losing after 10k hands, and bad player could be also winning after 10k hands.
100k would be appropriate sample size.

You missed the reference to AIVAT on your first reading, which was a bit dumb, so I highlighted it for you.

Quote:
It doesn't matter if author is aware of 10k sample. There are so many news bot beats best humans in poker.
10k sample is far away of a proof of being better.

And still you didn't read it, which makes you look utterly ridiculous.

It's been fun discussing this but I'm a bit short of time to continue further. Any more dumb or obnoxious comments are likely to be deleted.


Top
 Profile  
 
PostPosted: Tue Jul 16, 2019 10:20 am 
Offline
New Member

Joined: Mon May 20, 2019 2:21 pm
Posts: 5
mlatinjo wrote:
Spears, You don't seem to have an argument, except to insult someone who has opinion other than yours.

It doesn't matter if author is aware of 10k sample. There are so many news bot beats best humans in poker.
10k sample is far away of a proof of being better.

Their comment about hand JQ bot vs QT human, both hitting top pair, and claiming bot checks flop (without preflop initiative) to trap opponent, then again on turn it traps opponent and again on river traps opponent (totally fishy comments). Hands played by bot are fine, i am referring to their commenting on hands played is ridicilous.


It's reasonable to be extra cautious about some published papers as you often get the impression the result of a paper was a foregone conclusion and statistics etc. are just staged. However, in this case, the paper was written by Noam Brown who has an excellent reputation and has been very active in the ACPC since it's very start.
I think this is an excellent paper as it's very well written and is a fun read for even for non-scientists.

You have a point regarding sample size though. AIVAT is an excellent metric to compare bot vs bot performance because you can play duplicate hands with known strategies. I think even despite using AIVAT the sample size in this format is a little shallow considering they can't know the strategy distribution of the humans and they couldn't deal duplicate hands.

On the other hand, it's still an impressive result and a great paper with every information transparently included.


Top
 Profile  
 
PostPosted: Tue Jul 16, 2019 10:25 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 600
Quote:
despite using AIVAT the sample size in this format is a little shallow


Maybe, but still a lot better than 10k hands. IIRC UofA put the effective improvement due to AIVAT at a factor of 10 for bot v human.


Top
 Profile  
 
PostPosted: Tue Jul 16, 2019 10:54 am 
Offline
New Member

Joined: Mon May 20, 2019 2:21 pm
Posts: 5
spears wrote:
Quote:
despite using AIVAT the sample size in this format is a little shallow


Maybe, but still a lot better than 10k hands. IIRC UofA put the effective improvement due to AIVAT at a factor of 10 for bot v human.


Indeed, that's what they claim. However, the amount of variance reduction also depends on how well the value functions estimate the true expected value. I would assume that the value functions are even less precise in 6-max as opposed to heads-up. Also, if you can't factor in card distribution because you don't deal duplicate hands, you can't account for "card luck". Theoretically, Pluribus could have been dealt strong hands disproportionately often. That's why the sample size is still on the low side IMO.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
cron
Powered by phpBB® Forum Software © phpBB Group