Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 5:17 pm

All times are UTC




Post new topic Reply to topic  [ 8 posts ] 
Author Message
PostPosted: Sat Jun 17, 2017 6:38 pm 
Offline
Junior Member

Joined: Sat May 27, 2017 2:35 pm
Posts: 33
We take some action out of 5 possible actions on the flop, for each action we simulate the game forward from the turn and for each action on the turn we simulate the game on the river.

So for betting 1/3 on flop, should the expected EV of this action be the average of all turn EVs or the maximum of all turn EVs?

Visual representation of tree for betting 1/3 on flop (assuming opponent always calls or checks):

Image


Top
 Profile  
 
PostPosted: Sat Jun 17, 2017 10:46 pm 
Offline
Junior Member

Joined: Thu May 11, 2017 8:59 pm
Posts: 13
mediacalc wrote:
So for betting 1/3 on flop, should the expected EV of this action be the average of all turn EVs or the maximum of all turn EVs?


Seems pretty straightforward (maybe I'm misreading?). At every point in the game, EV is determined the weighted average of all possible future events.
In the example, the EV is the average of all possible events on all turn-river rollouts. This means you have to also take into account the opponent's decisions on any given decision-node.


Last edited by jackrabbit on Sun Jun 18, 2017 10:51 pm, edited 1 time in total.

Top
 Profile  
 
PostPosted: Sun Jun 18, 2017 2:13 am 
Offline
Junior Member

Joined: Sat May 27, 2017 2:35 pm
Posts: 33
jackrabbit wrote:

Seems pretty straightforward (maybe I'm misreading?). At every point in the game, EV is determined the weighted average of all possible future events.
In the example, the EV is the average of all possible events on all turn-river rollouts. This means you have to also take into account the opponent's decisions are any given decision-node.


I see I see. My model was just opponent always calls at 100% frequency (or checks if checked to). So I'd have to work out how often opponent takes some action and therefore how often we traverse a certain part of the tree and then use that to weight the EV.

Actually have no idea how I'd plug in an opponent strategy other than run the "hero-code" twice on itself. Anyway, so for my case all of them would be weighted the same so it would be a standard average?


Top
 Profile  
 
PostPosted: Sun Jun 18, 2017 8:52 pm 
Offline
Senior Member

Joined: Fri Nov 25, 2016 10:42 pm
Posts: 122
hi it is average over all run outs.

Do you really plan to implement it? Just imagine how many operations would you need to perform, in order to evaluate the whole game tree. How many possible boards you have, how many possible actions and opponent actions. The reason why gto is not solved in poker is because game tree is huge, so there are only approximations of game tree which yield solid results.


Top
 Profile  
 
PostPosted: Sun Jun 18, 2017 10:54 pm 
Offline
Junior Member

Joined: Thu May 11, 2017 8:59 pm
Posts: 13
mediacalc wrote:
jackrabbit wrote:

Seems pretty straightforward (maybe I'm misreading?). At every point in the game, EV is determined the weighted average of all possible future events.
In the example, the EV is the average of all possible events on all turn-river rollouts. This means you have to also take into account the opponent's decisions are any given decision-node.


I see I see. My model was just opponent always calls at 100% frequency (or checks if checked to). So I'd have to work out how often opponent takes some action and therefore how often we traverse a certain part of the tree and then use that to weight the EV.

Actually have no idea how I'd plug in an opponent strategy other than run the "hero-code" twice on itself. Anyway, so for my case all of them would be weighted the same so it would be a standard average?


Actually implementing something like this will require some extraordinary amount of cleverness if you want to arrive at anything useful. As mentioned, the game tree in poker can get huge, so some form of size-reduction (game abstraction, pruning, limited-lookahead) has to be implemented whenever you work with a real decision tree.
In reality, you should probably reconsider the way you implement this EV-estimation you're trying to do.


Top
 Profile  
 
PostPosted: Mon Jun 19, 2017 2:37 am 
Offline
Junior Member

Joined: Sat May 27, 2017 2:35 pm
Posts: 33
mlatinjo wrote:
hi it is average over all run outs.

Do you really plan to implement it? Just imagine how many operations would you need to perform, in order to evaluate the whole game tree. How many possible boards you have, how many possible actions and opponent actions. The reason why gto is not solved in poker is because game tree is huge, so there are only approximations of game tree which yield solid results.


What are these approximations that they make, I'm just so confused right now. My method was going to be to just process a tree with capped actions and sizes for hand vs hand, monte-carlo the possible board run-outs, have a training period of 100,000 boards with uniform strategy/probabilities and then start using CFR algorithm on the regrets after this training period. And that should have given me some sort of result like 70% bet, 30% check this flop with this hand.

jackrabbit wrote:

Actually implementing something like this will require some extraordinary amount of cleverness if you want to arrive at anything useful. As mentioned, the game tree in poker can get huge, so some form of size-reduction (game abstraction, pruning, limited-lookahead) has to be implemented whenever you work with a real decision tree.
In reality, you should probably reconsider the way you implement this EV-estimation you're trying to do.


Yep I'm already beginning to reconsider it after reading your two replies. The only problem is I don't know what else to try really. Wouldn't an abstracted game just limit the number of boards we look at, how would it actually shrink the tree itself (depth-wise)? It would just reduce the number of trees (one per board?)?

My aim was just to have a simple solid approximation of the best strategy given a hand on a board vs a range.


Top
 Profile  
 
PostPosted: Mon Jun 19, 2017 3:17 pm 
Offline
Senior Member

Joined: Fri Nov 25, 2016 10:42 pm
Posts: 122
Hi I would recommend you reading first book Application of no limit holdem from Janda. It is very easy to understand gto theory, for beginners.
Once you understand it advanced one is Expert Heads Up No Limit Hold'em from Will Tipton. It will show you how you can make approximations to calculate approximated gto ranges.

You have several gto softwares based on that theory, like pio solver, CREV etc. Keep in mind that such calculations of approximated gto ranges take like few minutes (depends on PC), and if you want to implement bot that makes decision in few seconds then you need even more approximations.

Regarding how to estimate villain range. You can off course apply hero ranges for them, that would be ok solution. But if you would implement solid ranges for hero, such ranges would be actually wrong for micro stake players, especially for fish, because they don't play solid, they make a lot of overplays with value hands, silly bluffs, or mostly not bluffing often enough. I personally use rule based ranges for villain in combination with their player stats. E.g nobody folds full hause, set, two pair on specific board (no flush possible) etc.


Top
 Profile  
 
PostPosted: Mon Jun 19, 2017 3:34 pm 
Offline
Junior Member

Joined: Sat May 27, 2017 2:35 pm
Posts: 33
mlatinjo wrote:
Hi I would recommend you reading first book Application of no limit holdem from Janda. It is very easy to understand gto theory, for beginners.
Once you understand it advanced one is Expert Heads Up No Limit Hold'em from Will Tipton. It will show you how you can make approximations to calculate approximated gto ranges.

You have several gto softwares based on that theory, like pio solver, CREV etc. Keep in mind that such calculations of approximated gto ranges take like few minutes (depends on PC), and if you want to implement bot that makes decision in few seconds then you need even more approximations.

Regarding how to estimate villain range. You can off course apply hero ranges for them, that would be ok solution. But if you would implement solid ranges for hero, such ranges would be actually wrong for micro stake players, especially for fish, because they don't play solid, they make a lot of overplays with value hands, silly bluffs, or mostly not bluffing often enough. I personally use rule based ranges for villain in combination with their player stats. E.g nobody folds full hause, set, two pair on specific board (no flush possible) etc.


Those look like poker books when what I'm after are poker AI books. I think I understand the concept of GTO enough to make what I'm making.

Those softwares are the ones I'm trying to emulate and yes it would be ok for it to take a few minutes (although that's longer than I thought it would be)

I will have separate ranges for villain (BB defence vs BTN open for example) for each position. I will likely make some microstakes adjustments to my software at some point too, although right now I'm just focused on getting a working program.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group