Poker-AI.org :: View topic - CFRM in non-zero-sum-games?

To create an aggressive agent, we used the Counterfactual Regret Minimization technique from
Chapter 3, but with one change. On every terminal node of the game tree, we gave a 7% bonus in
utility to the winning player, while the losing player did not pay any extra cost. This means that the
game is no longer zero sum, as more money is being introduced on each hand. The important beneﬁt
of this approach is that the agent always thinks it has better odds than it actually does — it is more
willing to ﬁght for marginal pots. For every $1 it invests in a bet, it thinks it will receive $1.07 in
return even if the opponent immediately folds, and $2.14 if the opponent calls the bet and the agent
wins the showdown. This extra return on investment encourages it to bet in more situations where it
might otherwise call or fold. However, since it learns its strategy by playing against an agent trying
to exploit it, it learns to express this extra aggression in a balanced way that still effectively hides
information.

Author:	proud2bBot [ Fri Mar 22, 2013 9:18 pm ]
Post subject:	CFRM in non-zero-sum-games?
CFRM has been proven to result in nash-equilibrium approximations for zero-sum games. I wonder how to incorporate the rake in. For instance, in some games a optimal strategy might be to see a lot of flops by limping for example w/o considering rake, while minRaising might be way better if we consider rake. Obviously, we could add a rake model and change the values in all terminal nodes correspondingly. However, this would change the game as its not a zero-sum game anymore. Does anyone know if we still get a good approximation? Or how do you factor in rake in cash games?

Author:	cantina [ Fri Mar 22, 2013 9:34 pm ]
Post subject:	Re: CFRM in non-zero-sum-games?
Isn't it still zero-sum if both players pay the same amount of rake (assuming a two player game)?

Author:	proud2bBot [ Fri Mar 22, 2013 10:03 pm ]
Post subject:	Re: CFRM in non-zero-sum-games?
first the assumption might be wrong as it depends on the playing style, second I think even if its right - as far as I understood the GTO basics - its not a zero sum game anymore.

Author:	Magnum [ Fri Mar 22, 2013 11:24 pm ]
Post subject:	Re: CFRM in non-zero-sum-games?
Like this? Quote: To create an aggressive agent, we used the Counterfactual Regret Minimization technique from Chapter 3, but with one change. On every terminal node of the game tree, we gave a 7% bonus in utility to the winning player, while the losing player did not pay any extra cost. This means that the game is no longer zero sum, as more money is being introduced on each hand. The important beneﬁt of this approach is that the agent always thinks it has better odds than it actually does — it is more willing to ﬁght for marginal pots. For every $1 it invests in a bet, it thinks it will receive $1.07 in return even if the opponent immediately folds, and $2.14 if the opponent calls the bet and the agent wins the showdown. This extra return on investment encourages it to bet in more situations where it might otherwise call or fold. However, since it learns its strategy by playing against an agent trying to exploit it, it learns to express this extra aggression in a balanced way that still effectively hides information. source: http://poker.cs.ualberta.ca/publications/johanson.msc.pdf, pg.82 So instead of adding a 7% bonus to pots, you would subtract x% for rake.

Author:	proud2bBot [ Sat Mar 23, 2013 12:03 am ]
Post subject:	Re: CFRM in non-zero-sum-games?
ahhh, I remember they were doing something like that and they converged. Thanks magnum!

Poker-AI.org http://poker-ai.org/phpbb/

CFRM in non-zero-sum-games? http://poker-ai.org/phpbb/viewtopic.php?f=24&t=2423	Page 1 of 1

Author:	Coffee4tw [ Mon Mar 25, 2013 11:11 pm ]
Post subject:	Re: CFRM in non-zero-sum-games?
This assumes a constant rake though. Most poker sites cap their rake somewhere which would probably have interesting effects on strategies. But since it is still symmetrical, I don't see why it shouldn't converge.

Author:	proud2bBot [ Mon Mar 25, 2013 11:28 pm ]
Post subject:	Re: CFRM in non-zero-sum-games?
I'm planning on just adding a "ValueChanger" to each showdown node, which gets the node as input and outputs the factor for the winner/loser. Default it returns always 1. But then we can return another factor to make our bot more agressive or to incorporate rake. The later can be implemented as we have the node as input, hence we know which round are we at and can apply the rake formula of our target site.

Author:	fraction [ Wed Dec 11, 2013 4:29 pm ]
Post subject:	Re: CFRM in non-zero-sum-games?
If we would minus some percent for rake (say 5), but could add some for aggression (say 5) then would a strategy generated with neither simply be overaggressive (and not nash) when played in raked games?

Page 1 of 1	All times are UTC
Powered by phpBB® Forum Software © phpBB Group http://www.phpbb.com/