Poker-AI.org

If you saw my last topic, you know I already tried pretty naively to build a CS-CFRM library ( https://github.com/PierreMardon/gametheory ) where you just have to implement a game and the CFRM will run on it. It works but my model doesn't allow to make other CFRM algorithms run on those games. And it is slow because I didn't optimize it enough and because of my model.

So I want to build a new model as generic as possible to :
- allow any CFRM algorithm implementation
- work on zer-sum with incomplete information sequential games (where the chances infoset doesn't affect the possible players actions and only affect terminal nodes)

So what I try to figure out what are the required data a game must provide to achieve this and from it build a model that can be efficiently used in CFRM implementations.

In Vanilla CFRM ( http://poker.cs.ualberta.ca/publications/NIPS07-cfr.pdf ), the counterfactual utility of a player (p) node of infoset I in an iteration is :

So what we need is :
- the probability to reach each h in I : the players action probabilities contribution is given by the current state of the execution and the chances by their distribution
- for each possible action and each h, recursively getting the expected utility of this action (for h).

So my hypothetical model would need to have access to all this efficiently. And I think it should be enough to write any CFRM variant on it.

Of course, the first move is to separate the players actions sequential tree from the chances.
For this purpose, we can consider that each player node is in a "round", exactly as in HE poker.
A player node pn of info set I is defined by the path in the sequence tree and the knowledge that the player has of the chances (its chances info set).
All the histories h in I are defined by the rounds chances path that can lead to it and the sequence tree parent node.
The rounds chances path is the player's known ones and the probability of each combination of other players chances info set.

The chances informations that we need can be huge because for each chance info set we need to know other players possible chances.
Of course we can first reduce them by grouping game exact equivalency and remembering the ponderation.
Then we can use bucketing to reduce the number of info sets but it doesn't reduce our needed knowledge of the distribution of the current player's possible exact chances histories in this info set coupled to the other players possible exact chances histories.

Assuming many people already wrote a Vanilla CFRM algorithm on bucketed HE, maybe I missed a point about bucketing, if anyone can confirm or infirm my last assertion it would definitely help me.

So I'm stucked there for now because I don't fully understand how chances can be handled programmatically, and if I need a full round chances tree or could use something lighter.

That's exactly why it's so smooth to implement CS-CFRM to make a generic CFRM engine

Any contribution is welcome