Poker-AI.org

Heads-up limit hold’em poker is solved
by: Michael Bowling, Neil Burch, Michael Johanson, Oskari Tammelin

Abstract
Poker is a family of games that exhibit imperfect information, where players do not have full knowledge of past events. Whereas many perfect-information games have been solved (e.g., Connect Four and checkers), no nontrivial imperfect-information game played competitively by humans has previously been solved. Here, we announce that heads-up limit Texas hold’em is now essentially weakly solved. Furthermore, this computation formally proves the common wisdom that the dealer in the game holds a substantial advantage. This result was enabled by a new algorithm, CFR+, which is capable of solving extensive-form games orders of magnitude larger than previously possible.

http://www.sciencemag.org/content/347/6218/145.abstract

Also noticed this just now: http://poker.cs.ualberta.ca/open_cfr_plus.html

Oskari Tammelin's web page also has some brief notes / pseudocode: http://jeskola.net/ (click CFR at top)

"Actions that have appeared poor (with less than zero regret for not having been played) will be chosen immediately after proving useful."



Brilliant. Hats off, gentlemen.

Where did you find this quote?

Here's a brief paper by Oskari on CFR+ with pseudocode: http://arxiv.org/pdf/1407.5042v1

He says that CFR+ is meant for "vector-form" updates, and cites http://poker.cs.ualberta.ca/publication ... 12-pcs.pdf with more info about that. The pseudocode from both papers are very similar...

My current interpretation is that CFR+ works with public chance sampling (go through all hands for each player, but just one board). My concern is if it means you have to do Vanilla CFR iterations and go through all boards as well.

I'm curious to hear of any attempts to implement this. It'll take time to get some old PCS code working again

Cool stuff!



PCS should work better than monte carlo, but there will still be actions that "prove useful" due to variance and not because they're actually useful. Using PCS and only using CFR+ on the river could work though (no variance there).

Maybe it's possible to use some negative treshold instead of 0 to provide some buffer against variance in sampling versions.

i.e.

Thanks for the response. I tried Pure CFR with several different negative thresholds, but it didn't converge faster in my tests.

I also tried a form of PCS that only uses the current strategy when training. It also performed worse when using a 0 threshold on the river compared to not using one at all. When using a card abstraction (and imperfect recall) there can be several board rundowns that result in the same bucket, so there is still variance on the river I think.

Has anyone seen improvement when trying something like CFR+ but with sampling? I feel like there should be some compromise between Vanilla CFR with no abstraction and one with buckets and sampling cards and actions.

Of course, there could be bugs in my implementations, my tests were not run for very long, maybe it does better with larger card abstractions, etc. I did get CFR+ working in toy games just fine though, based off Oskari's demo code.

Yes.

You can play against Cepheus the solved Heads-up limit hold’em PokerBot here

http://poker.srv.ualberta.ca/

I have a couple of questions regarding optimization

[1] In http://webdocs.cs.ualberta.ca/~bowling/papers/15science.pdf they state


So I modified the pseudocode provided in http://arxiv.org/pdf/1407.5042.pdf as I understand it. (Attached to this post - See bottom). Do you agree with that?

[2] In http://webdocs.cs.ualberta.ca/~bowling/papers/15science.pdf they explain

What might that scaling factor be? 1,000? 10,000? 100,000?

[3] The pseudocode uses a non-declared/ non-preinitialized vector m. What might that be? A temporary variable?

[4] The entire strategy is ~524 TB in size:
- Since the average strategy is not used -> 524 / 2 = 262 TB required
- Since we are using 4 Byte Int32 instead of 8 Byte Doubles -> 262 / 2 = 131 TB required
- Since storage allows a compression of factor 13 -> 131 / 13 = ~10 TB required

Correct? Thanks ahead

I couldn't get the thresholding to work either. I did found sort of an improvement though (which could possibly be enhanced).

Weighing the regrets has been discussed here before and gives a slight boost. This seems to be related to the cfr+ phenomena, because weighing only the negative regrets gives faster convergence than weighing all regrets.

The boost is nothing like cfr+ over vanilla cfr however.

Using weight=iterations+1 in CSCFR and weighing negative regrets and avg strategy, it reached <5mbb/g exploitability 20% faster than weighing all regrets and avg strategy, and 36% faster than not weighing anything (in the flop subgame I tested).

The weight will likely depend on what sampling is used, and linear isn't necessarily optimal.

Actions that have appeared poor (with less than zero regret for not having been played) will be chosen immediately after proving useful.

I was looking for poker but I came across some mathematicians. Wtf? Card counting ISN'T illegal. (Unless you use a ""device"" such as a computer to do so in Nevada, which is a felony under Nevada law.) Card counting is frowned upon because it violates the UNWRITTEN ""law"" (that the casino is supposed to have the edge ). So the choice is up to you.
_________________________________________________________
https://www.score88poker.bid/mobile.php
Life, like poker, has an element of risk. It shouldn’t be avoided. It should be faced. ” ~ Edward Norton