Poker-AI.org Poker AI and Botting Discussion Forum 2017-06-19T15:34:51+00:00 http://poker-ai.org/phpbb/feed.php?f=24&t=3062 2017-06-19T15:34:51+00:00 2017-06-19T15:34:51+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7292#p7292 <![CDATA[Re: Max utility or average utility for action tree]]> mlatinjo wrote:

Hi I would recommend you reading first book Application of no limit holdem from Janda. It is very easy to understand gto theory, for beginners.
Once you understand it advanced one is Expert Heads Up No Limit Hold'em from Will Tipton. It will show you how you can make approximations to calculate approximated gto ranges.

You have several gto softwares based on that theory, like pio solver, CREV etc. Keep in mind that such calculations of approximated gto ranges take like few minutes (depends on PC), and if you want to implement bot that makes decision in few seconds then you need even more approximations.

Regarding how to estimate villain range. You can off course apply hero ranges for them, that would be ok solution. But if you would implement solid ranges for hero, such ranges would be actually wrong for micro stake players, especially for fish, because they don't play solid, they make a lot of overplays with value hands, silly bluffs, or mostly not bluffing often enough. I personally use rule based ranges for villain in combination with their player stats. E.g nobody folds full hause, set, two pair on specific board (no flush possible) etc.


Those look like poker books when what I'm after are poker AI books. I think I understand the concept of GTO enough to make what I'm making.

Those softwares are the ones I'm trying to emulate and yes it would be ok for it to take a few minutes (although that's longer than I thought it would be)

I will have separate ranges for villain (BB defence vs BTN open for example) for each position. I will likely make some microstakes adjustments to my software at some point too, although right now I'm just focused on getting a working program.

Statistics: Posted by mediacalc — Mon Jun 19, 2017 3:34 pm


]]>
2017-06-19T15:17:35+00:00 2017-06-19T15:17:35+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7291#p7291 <![CDATA[Re: Max utility or average utility for action tree]]> Once you understand it advanced one is Expert Heads Up No Limit Hold'em from Will Tipton. It will show you how you can make approximations to calculate approximated gto ranges.

You have several gto softwares based on that theory, like pio solver, CREV etc. Keep in mind that such calculations of approximated gto ranges take like few minutes (depends on PC), and if you want to implement bot that makes decision in few seconds then you need even more approximations.

Regarding how to estimate villain range. You can off course apply hero ranges for them, that would be ok solution. But if you would implement solid ranges for hero, such ranges would be actually wrong for micro stake players, especially for fish, because they don't play solid, they make a lot of overplays with value hands, silly bluffs, or mostly not bluffing often enough. I personally use rule based ranges for villain in combination with their player stats. E.g nobody folds full hause, set, two pair on specific board (no flush possible) etc.

Statistics: Posted by mlatinjo — Mon Jun 19, 2017 3:17 pm


]]>
2017-06-19T02:37:03+00:00 2017-06-19T02:37:03+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7290#p7290 <![CDATA[Re: Max utility or average utility for action tree]]> mlatinjo wrote:

hi it is average over all run outs.

Do you really plan to implement it? Just imagine how many operations would you need to perform, in order to evaluate the whole game tree. How many possible boards you have, how many possible actions and opponent actions. The reason why gto is not solved in poker is because game tree is huge, so there are only approximations of game tree which yield solid results.


What are these approximations that they make, I'm just so confused right now. My method was going to be to just process a tree with capped actions and sizes for hand vs hand, monte-carlo the possible board run-outs, have a training period of 100,000 boards with uniform strategy/probabilities and then start using CFR algorithm on the regrets after this training period. And that should have given me some sort of result like 70% bet, 30% check this flop with this hand.

jackrabbit wrote:

Actually implementing something like this will require some extraordinary amount of cleverness if you want to arrive at anything useful. As mentioned, the game tree in poker can get huge, so some form of size-reduction (game abstraction, pruning, limited-lookahead) has to be implemented whenever you work with a real decision tree.
In reality, you should probably reconsider the way you implement this EV-estimation you're trying to do.


Yep I'm already beginning to reconsider it after reading your two replies. The only problem is I don't know what else to try really. Wouldn't an abstracted game just limit the number of boards we look at, how would it actually shrink the tree itself (depth-wise)? It would just reduce the number of trees (one per board?)?

My aim was just to have a simple solid approximation of the best strategy given a hand on a board vs a range.

Statistics: Posted by mediacalc — Mon Jun 19, 2017 2:37 am


]]>
2017-06-18T22:54:44+00:00 2017-06-18T22:54:44+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7289#p7289 <![CDATA[Re: Max utility or average utility for action tree]]> mediacalc wrote:

jackrabbit wrote:

Seems pretty straightforward (maybe I'm misreading?). At every point in the game, EV is determined the weighted average of all possible future events.
In the example, the EV is the average of all possible events on all turn-river rollouts. This means you have to also take into account the opponent's decisions are any given decision-node.


I see I see. My model was just opponent always calls at 100% frequency (or checks if checked to). So I'd have to work out how often opponent takes some action and therefore how often we traverse a certain part of the tree and then use that to weight the EV.

Actually have no idea how I'd plug in an opponent strategy other than run the "hero-code" twice on itself. Anyway, so for my case all of them would be weighted the same so it would be a standard average?


Actually implementing something like this will require some extraordinary amount of cleverness if you want to arrive at anything useful. As mentioned, the game tree in poker can get huge, so some form of size-reduction (game abstraction, pruning, limited-lookahead) has to be implemented whenever you work with a real decision tree.
In reality, you should probably reconsider the way you implement this EV-estimation you're trying to do.

Statistics: Posted by jackrabbit — Sun Jun 18, 2017 10:54 pm


]]>
2017-06-18T20:52:00+00:00 2017-06-18T20:52:00+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7288#p7288 <![CDATA[Re: Max utility or average utility for action tree]]>
Do you really plan to implement it? Just imagine how many operations would you need to perform, in order to evaluate the whole game tree. How many possible boards you have, how many possible actions and opponent actions. The reason why gto is not solved in poker is because game tree is huge, so there are only approximations of game tree which yield solid results.

Statistics: Posted by mlatinjo — Sun Jun 18, 2017 8:52 pm


]]>
2017-06-18T02:13:08+00:00 2017-06-18T02:13:08+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7287#p7287 <![CDATA[Re: Max utility or average utility for action tree]]> jackrabbit wrote:

Seems pretty straightforward (maybe I'm misreading?). At every point in the game, EV is determined the weighted average of all possible future events.
In the example, the EV is the average of all possible events on all turn-river rollouts. This means you have to also take into account the opponent's decisions are any given decision-node.


I see I see. My model was just opponent always calls at 100% frequency (or checks if checked to). So I'd have to work out how often opponent takes some action and therefore how often we traverse a certain part of the tree and then use that to weight the EV.

Actually have no idea how I'd plug in an opponent strategy other than run the "hero-code" twice on itself. Anyway, so for my case all of them would be weighted the same so it would be a standard average?

Statistics: Posted by mediacalc — Sun Jun 18, 2017 2:13 am


]]>
2017-06-18T22:51:54+00:00 2017-06-17T22:46:51+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7285#p7285 <![CDATA[Re: Max utility or average utility for action tree]]> mediacalc wrote:

So for betting 1/3 on flop, should the expected EV of this action be the average of all turn EVs or the maximum of all turn EVs?


Seems pretty straightforward (maybe I'm misreading?). At every point in the game, EV is determined the weighted average of all possible future events.
In the example, the EV is the average of all possible events on all turn-river rollouts. This means you have to also take into account the opponent's decisions on any given decision-node.

Statistics: Posted by jackrabbit — Sat Jun 17, 2017 10:46 pm


]]>
2017-06-17T18:38:02+00:00 2017-06-17T18:38:02+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3062&p=7284#p7284 <![CDATA[Max utility or average utility for action tree]]>
So for betting 1/3 on flop, should the expected EV of this action be the average of all turn EVs or the maximum of all turn EVs?

Visual representation of tree for betting 1/3 on flop (assuming opponent always calls or checks):

Image

Statistics: Posted by mediacalc — Sat Jun 17, 2017 6:38 pm


]]>