Poker-AI.org

viewtopic.php?f=25&t=3009
The latest paper shows that they have developed an HUNL AI with deep neural networks instead of the abstraction. It shows a huge edge against some decent human players.
We had some HU AI based on abstraction method with only ~6bb/100 on PS low stakes without opponents knowing they are playing against AI. This AI's 45+bb/100 edge is very surprising.

Very interesting but 45bb/100 just seems astronomical and too good to be true. I am not convinced that good players are playing with such a disadvantage vs a defensive/GTO style. If DeepStack is putting the human to an indifferent decision at all times I cannot see the human making the wrong play often enough to achieve that kind of win rate..but maybe.

It could be more difficult for the human then I thought. In general there is not much vacuum play (as most good players are all exploit based strategies) going on so maybe they are seeing exploitation when there isn't any and it just compounds the problem when they do face a true GTO opponent long term.

It's a very surprising result but it does seem to be a very strong AI.

The pros they played against weren't HU specialists but 450mbb/h is a huge margin. I guess you might reduce that by 100mbb/h against the best.

The paper says that previous best HUNL were analysed to be exploitable by over 3000mbb/h. One of the best, Claudico, beat pros by 91mmb/h last year.

I have been thinking about this and 45bb is definitely possible. My 6max PLO bot crushes HU at 96bb/100 but its way easier to make a mistake at PLO then Holdem for recreational players I think. Everything we know about GTO/balanced play I think is all theories at this point but a good chunk of it is just basic math and abstractions of perfect play is fine 99.9% of the time imo.

I think a defensive strategy is exploitative by nature just in the fact that the human cannot do the math. The player may think he is playing balanced but the reality is he is not over all board types. So there lies the weakness and the mistakes just pile up as more hands are played. It really does not take many mistakes to go heavy -EV (especially mistakes made on the river or all-ins)

There are some situations that are very difficult for the human to calculate that come up quite often. One such situation is a low SPR 3 way situation. When you have various stack sizes, side pots and ranges there is just no way to estimate your EV with any real degree of accuracy long term. For a bot its no problem.

So I think DeepStacks results are completely plausible.

Just watch what the best HUNL poker pro thinks about that bot
https://www.youtube.com/watch?v=gz9FJfe2YGE

I think that it is not realistic that bot has 45bb/100 winrate against pros. Last time the bot played vs humans, humans won with 9bb/100 winrate, which is a huge winrate in poker, but they called it statistical draw which doesn't make sense. Anyway, it also depends on the number of hands they played it. Humans had to play like 9 hours a day vs bot and were super annoyed and bored, got tired etc but still crashed the bot last time.

96bb/100 is totally unrealistic in poker. If you have a winrate of 10bb/100 hands it is considered to be a huge winrate. 3bb/100 would be standard, 5bb solid, 10bb is crash. 96bb/100 is simply unrealistic even at micros. You should only observe winrate on sample > 100k hands. On sample like <10k variance is sick.

The stats are correct. I am only getting a higher return due to wealth of poor players at micro limits. I have a 500k sample of bot data and still crush HU at 63bb at PLO100. Over all number of players its 6bb which is standard.

I am using 6 max data here so I cannot really compare to all HU. But if the bot reaches the flop HU its game over for the villain in general. So just disregard this data as I cannot really compare 6max to true HU play

Doug Polk was ranting about the Carnegie Mellon (CMU) bot Claudico which he and his team beat 96mbb/h in 2015. CMU are fielding another bot, named Libratus, against Jimmy Chou, Dong Kim, Jason Les and Daniel McAulay right now. https://www.inverse.com/article/26168-p ... against-ai

university of Alberta (UoA) played their bot called DeepStack against pros in December 2016. It works on quite different principles to Claudico.

When you say "I think that it is not realistic that bot has 45bb/100 winrate against pros", are you saying that the UoA team are lying?

I trust the numbers in UoA papers. But I don't believe a Nash model can have such a big edge against decent players from a poker player's experience.

Is it possible that the players are not treating this test seriously? Many of them didn't even have the patience to finish their 3000 hands and some may gamble for the top 3 prizes. The results will be more convincing and accurate if they are risking their own money. Having said that, I still believe DeepStack is a very strong AI and should be able to beat the best human players in HU.

I had not even considered that. They have to be risking something or its just a play money like attitude. That is a huge factor in this test. With nothing to lose they will not play well or have a reason too.

The other factor which has to be overcome is the rake factor. Online this AI will be much less effective. I have seen rake as high as 20bb/100 depending on the stakes played.

The test was not done in the best way, only top 3 of 30 players went itm so the humans were encouraged to gamble and not to play their real A-game in a cash table. Anyway 45bb is truly amazing.

I would try to reproduce the deep-stack algorithm. The bulk of the cost is to reproduce 10M size training set of random situation solved by cfr for training the network with 3500 hidden units. They've runned 6144 cpu for 11 days. I've estimated that would cost 50k euro. I can at best do 1M samples with 5k of investement so i was thinking to start solving some poker game less deep, like husng were i can reproduce 3-5M samples or start doing some data expansion using multiple examples from the same solved game but it's inappropriate for deep-stack resolving mechanism.

I would suggest starting with short stack HU as well. I have some small servers myself and may help a bit on the computing power if you need.