Poker-AI.org

I touched on this in the previoues AHS vs EHS thread but I found this very interesting blog post over on propokertools.com:
http://pokercoder.blogspot.com/2006/07/ ... hands.html

It discusses a way to get a more meaningful hand ranking than ranking against a 100% range AND removes the guessing part ("oh let's just rank against the top 10% of hands").



Now the part he is talking about from "The Challenge of Poker" seems to be this one 5.1.2: Iterated Roll-out Simulations.

A great idea for 10 handed but I want to take it down to HU and make it multi-street, to account for card removal effects.
So instead of just allowing one bet preflop, I want to allow a bet on all streets and keep track of "good hands" per board.

Thoughts?

_________________
Cheers.

So to explain this a little more, we can do either of two things:

• Each street separately
  
A player puts in a bet of 1 if they have a good hand. If the other player has a good hand as well he matches the bet and we roll it out.
I believe we need some sort of pot already built to ensure we don't reduce the good hands to the single best hand so I'd always assume the pot is 2 before any betting.
Preflop and all boards are evaluated separately without connection between each other.


• Full hand tree
  
Here we are using a full betting structure tree. Assume blinds are 2.
If SB doesn't have a good hand he folds and pot is awarded to BB.
If SB does have a good hand he puts in 1, if doesn't have a good hand BB folds and pot is awarded to SB.
If SB does have a good hand he puts in 1, if BB does have a good hand he puts in 1 and we go to the flop.
Now we repeat this after evaluating if the hands are now good hands given the flop.
Same on turn and river.

At the end we'll get profitabilities associated with each hand and then make the ones that are +EV good hands and repeat the process.
Sounds like ProPokertools and UofA was using 10 evolutions with 50k iterations each but I don't see why we can't just roll-out all possible combinations for each evolution. Probably gonna take a while but it's gonna be 100% accurate without the instability that they are talking about.

The first approach is easier to implement and guarantees that we pick the best hands possible given a board but doesn't account for card removal effects.
The second approach is maybe a little trickier to implement but has the benefit that we don't choose good hands on a certain board that are unlikely to get there based on action on previous streets.

_________________
Cheers.

I guess it all depends on where you are using the ranking: if you try to make a decision on the river with a MC bot and want to assess villains range after you raises preflop, bet the flop and the turn, the second approach will give you the better answer. However, in a HU match where it was limped pre and checked on the flop, approach 1 might be better. In general, I think the idea has some merit to outperform EHS2

IMO, for bot vs human poker, it's difficult to calculate an "accurate" EV because you never know for sure your opponent(s) cards are, or if they are bluffing. Obviously we can do *some* opponent modelling but it will only give us a rough idea of what they *might* be doing. For HU SNG, I have a MC evaluator that basically runs my hand vs opponent range and returns the EV. The main problem is that it assumes the game will go to the river, so is essentially an all-in calculator, but I guess it could be adapted to only deal a flop or turn as well.

Getting an opponent range is a major task in itself. I've taken a simple approach and give the opponent a "top X%" range based on the number of times they pay to see a flop. With a lack of information, I assume the stats compiled from the HH's of known "good" players (i.e. top 10 Sharkscopers) at effective stack > 10, which is top 78% from the SB and top 52% from the BB. I then merge these with observed stats as I gather more information. My hand range lookup table was compiled from the Chubukov rankings which is a fairly standard 32o - AA table.

I've never liked the hand evaluators that assume your opponent plays 100% as it's a very unrealistic abstraction. I'd take fictitious play every time.

_________________
www.bespokebots.com

Well this essentially is trying to obtain which hands exactly should be in that TOP-X% range.
For HU preflop all-in equity, this is a fairly simple task to get right. Chubokov is nice as it assumes your opponent is calling perfectly against you.

However, I am more concerned about deepstack play and postflop decisions, which changes rankings dramatically.

Just a quick example to illustrate my point:
We raise, villain calls preflop. Flop comes 347r. Villain bets.
Which ranking should we pick to get an accurate picture of our equity against the opponent? Remember, this is supposed to be GTO without an opponent model against a specific opponent.
Let's assume it's optimal for villain to call with the top 40% preflop and he leads out with good hands rather than weak hands on the flop.

Traditional models such as AHS, EHS and EHS2 will now roll-out our hand against all possible holdings for our opponent to figure out how we stand on this flop.
My approach (#2) would roll-out our hand against the "good hands" on this flop, which are restricted by the "good hands" preflop.
My approach (#1) would roll-out our hand against the "good hands" on this flop, unrestricted of the "good hands" preflop (aka a 100% range leading up to the current street).

Betting Revision
I think the bet 1 bet on all streets is a bad metric. A half pot bet would probably have better results.
This means:
Blinds: 1+1
Preflop bets: 1+1
Flop bets: 2+2
Turn bets: 4+4
River bets: 8+8
Final pot: 32

Although some experimentation on this would be interesting.

Profitability Threshold
The original approach assumes all hands that are +EV to be the "good hands" now with multiple streets of betting and somewhat arbitrary bet sizes this will possibly need to be revised as well. Gonna have to wait for the first evolution to finish before deciding that. In theory this should still be true but I wonder how tight those ranges are going to get ...

Now I only need to find some time to actually implement this ...

_________________
Cheers.

I understand what you are trying to do, but I can't help wondering if you are overcomplicating the issue. No matter how much you try to calculate an exact value, there's always the possibility that an opponent is bluffing or playing "incorrectly". I actually use the TOP-X% range from the Chubukov table all the way to the river because the opponent doesn't change his cards half way. Obviously that range can then be pruned as we gain more info on specific opponents, but meanwhile assuming that this player has the same range as a known "good" player can never be a bad thing.

Having said all that, with Turbo and Super Turbo HU SNG's, you only have to hold position for 10 - 20 hands and then start shoving Nash, so I've not spent a great deal of time trying to put opponents on specific hands or clusters.

_________________
www.bespokebots.com