Poker-AI.org

OK, I've thought of another way poker and physics guy could work together. Physics guy calculates equity and variance (see above). Poker guy provides the rules for actions based on equity, variance, position, pot, stack size etc, but without any consideration of actual cards. This is much simpler for physics guy and gives poker guy more to do than in my MCTS suggestion.

This is a bit of an understatement. I think it depends on what you're aiming for - beating the micros maybe, but a winning SSNL rules based bot would be extremely difficult to construct even for someone with the domain knowledge to beat that level.

That said, I do like your approach:



Have you checked out blah and inprova on deucescracked? They've got some great stuff on perceived ranges.

Hey Spears, thank you for your kind help in this.
What you are suggesting is more or less what we are already doing. So I guess I was terrible at explaining it, I'm so sorry, it must be my terrible english plus my approximate usage of the "bot language", so to speak.
When I asked about putting a rule based core in a Montecarlo AI was talking more or less about this, my bad, I'm totally new to the bot's world.
Will get better, I promise, I'm a fast learner



Well, I see what you are saying here as we had a lot of interesting conversations about this, but eventually we both got convinced that given that our goal is *not* to totally solve the game in GT, we could use some shortcuts, or rules, to get the AI past all the things we already know as reasonably "correct" (e.g.: rule says to open 2.5x with a determined range from UTG instead of trying all the combinations of amount to open + any two cards and see in the tree what is profitable and what is not - which is what MCTS does at the moment). Plus, the poker player checks on frequencies, bluffs, slowplay et cetera, exactly as you suggest.

I have to say that the physicist totally agrees with me that the advantage we have, should we have one, on the normal approach to a poker AI is that we have a good insight on both the two worlds, science of probabilities and poker, so we do believe we should use each others' knowledge to get an AI which takes the best of the two approaches.

By the way I already got MCTS and simplebot playing each other on testbed, but I noticed that Holdem Manager won't read the hand history generated, not even if I reformat the HH in xml or whatever format I know. I can see the two guys playing via a universal replayer, which is fast and so I can easily understand what they are doing, but I can't access that with the tracking software.
Is the poker academy crack HH readable by holdem manager? or should I pass to pokertracker, of which I heard is more elastic with HHs?



As for this, I can't wait to see what my friend says. In general, for all the interesting technical stuff pointed out in this thread, I already told the physicist to come here and interact. As I said he's currently traveling for work, hope he gets back here soon.

Thank you everybody.

To Shalako, who seems very experienced in the field so probably doesn't need any suggestions by me, I would like to suggest not to be overvaluing the importance of polarization. Polarizing is a very very important concept but it can also be tricky, for example you can't be always polarized on rivers as it makes pretty easy for attentive players to bluff catch if you don't bet also your mid range of value, as you end up with too few value combos to bet with. Another example can be the 3bet preflop, where you don't want to be polarized OOP vs a high caller (>35%) who's not too prone to folding to cbets in 3bet pots (fcb3b<40%).

Wow. I must be going soft. Civil interactions with two rules guys in 24 hours

Well it has not been easy and in the last year I had to totally hit the books/videos/blogs in order to get a better understanding of NL game theory as most of my bot experience up to that point was mostly FL although I did write a decent 9max NL50 Zoom bot when it first came out. Nasher got me into HU which started me thinking on how very little I knew about game theory. All my NL experience is live MTT play and very little cash. I think videos are the best learning tool.

My aim at this point is not very high..maybe 100NL tops. I believe the power of the bot is its ability to play multiple tables without error so I am looking to win a small amount and make up the difference by putting in a massive amount of hands per hour.



I have only polarized preflop and the flop and not the turn and river for just the reasons you mention. So instead of polarizing the turn the one thing I did do was run sims to make sure I was not over folding the turn which is a common leak when facing a bet. Luckily it was perfect and I didn't have to make any changes. I still have work to do on the river but generally I am happy with it. I do need to work on using more hand combinatorics for river betsizing to try to extract maximum value. Like if I have the nuts and he has bluffcatchers in his range he would be more likely to call a pot or 2x pot bet vs a half pot value bet because it looks like a bluff. I think...I am not really sure if I am doing this correctly yet...

Interesting you point out that 35% number which is critical one for calling OOP. I think ideally its 38% but you want to just be nearly over calling by just a smidge. I have only polarized its baseline preflop strategy but its changing quite a bit based on the players stats adding value or air where needed. I think the toughest part of preflop is getting the villains 3 bet bluffing range right. That range can vary widely with the player and I struggle with it. I settled on what I think is the best compromise after consulting many online players on how they handle it. One thing I keep wanting to do is use showdown information to verify I am ranging correctly. Its on the list...

I like all the suggestions you can give me! All of this is a learning experience and the more I study and ask questions the better and more fun this is.

spears: we are all working on the same goal so I really enjoy your input on simulation related stuff which to me is way above my head! I stick with rules because they make sense to me and have worked for me in the past but I am always open to anything new. I also like total control of the bots actions so when I see an error I can fix it easily and fast. I have heard that is not so easy on simulation based bots or ?

No I haven't but I will check it out. I put in a considerable amount of code dealing with this so I think its pretty close to being right but I will read those articles and see if I can pick up anything new. Thanks!

Basically what I did was run sims to get my bots stats..so the same stats that the villains HUD would see to form the perceived range. After that it gets a bit more complicated but it works. The problem is that it caused a big surge in cpu usage on a decision by adding these routines.

fisherking. here is the thread on bluffing and perceived ranges:

https://poker-ai.org/phpbb/viewtopic.php?f=24&t=2566

look over it and see if you see anything that I was missing. I am still a bit confused on what formulas I should be using. I have seen several and I am not sure which one to use.

Agreed. This was part of the motivation for my second suggested architecture.

A bit more on the second suggestion...

Separation of concerns http://en.wikipedia.org/wiki/Separation_of_concerns is an important concept in software development, and I suspect in scientific thought in general. And this is a problem for you as well as physics guy. When you are composing rules you are engaged in software development. Condensing the cards down to two parameters (equity and variance) allows everyone involved to separate that aspect from all the others like position, pot size, stack size, etc. It should make your rules an order of magnitude simpler. It may be that equity and variance are not enough to describe the cards. If that is the case I would love to hear what you perceive the problems to be.

The more I think about it the more I think this architecture is a good platform for future development. Over time, you will see opportunities to replace human generated rules with rules generated from the data. MCTS has not been a huge success so far, and I don't think it is commonly understood why that is the case.

yep, I take it with the same attitude, it is a big interesting challenge, let's see where me and my friend will end up to, I'm very excited at the moment also because I know for a fact that the physicist is a very talented mind in the field of statistics; I hope to add to the process the little bit I can to speed it up (other than offering my insight on level of bot's play, players, games to play, rooms to attack etc., as I know quite a bit about the poker world).




I checked that out, and it seems you are getting it quite right, I mean, it all makes sense to me, but for the moment I don't think I could give you any useful suggestions, as I have not yet approached the problem in a botter perspective, so to speak. As soon as I'll reach that point in the making of the AI and I discuss this with my friend I'll get back to you, hopefully being of some better help.

Right now I would only say what you already know: that according to Sklansky in a vacuum you should bluff a percentage similar to the odds you are giving to your opponent with the bet, and such too generic things. But I repeat, it all sounded just about right, minimum 10 representable combos, plus that 30ish% of times you bluff (given that you bet more than 3/4 pot) sounds ok for sure. Obviously every opponent has his tendencies thou, so it is important to also look at specific stats, like how much he bet-folds river in general and possibly his frequencies on some given board textures (draw completed, draw missed, A-K fallen on the riv as overcards et cetera), how much he bluffcatches and so on. At midlowstakes I see very different bet-fold river %, for example, ranging from 30 to 70+ over decent samples. I see it as a pretty difficult problem to address, in any case.

Gotta love your suggestion, sounds pretty interesting, and again, can't wait to see what the statistics guy says about this, as I have heard him saying that MCTS is potentially very very powerful and that he could help him getting dramatically better also with my help, which I hope I'll be able to supply.
Thank you again.

P.s.: you are obviously talking about rethinking the thing from scratch, I am just hoping that a viable solution could also be to heavily review MCTS optimizing it more than doing a new one from zero.

sounds wery beautiful, but it is imposible (not near 1000 years).
poker fsm has more then millions of millions of trillions posible states. And in each state your model should do something. How many data you need to generate this LUT by statistic?

Only if you make some assumptions. But on what are they will be based? Only on human model. example
Here we come to the start

If the problem were really this hard then humans couldn't solve it either, and the current Nash Equilibrium solutions wouldn't exist. Or do you believe humans have magical properties? The key to solving the problem is in using good abstractions that reduce the trillions of of states to more manageable numbers. Sonia already solved this problem years ago.

Good luck with that.

If the villains stats show tendencies to over fold I have it disregarding combos and perceived ranges and just leading out when it has nothing just because of the fact I will always get value from my bluffs. What I didnt ever consider was what you stated above about the villain folding more or less on certain boards/cards fall although I have considered rivers in which all the draws missed and bricks hit. But..range equity will already reflect that to a large degree wont it? I mean whether certain cards help or hurt his range would be auto reflected in the range equity calc.

This is assuming your assigning the correct range which is not easy to get right. I spent a month working on a complicated range equity finder based on how pros put people on hands. It is surprisingly accurate but it does completely get it wrong quite often (like his actual hand was not in the bots range). With that being said its not often enough for me to change it yet however. Generally the problems lie with the 3B bluff range and the villains slowplaying range (although I have worked that out to some degree). Some people believe that unknown hands should be weighted and never be removed which is something I really need to consider.

I would like to work on this river problem you see as being difficult. I think I may have all the pieces to solve it but maybe we can work on a certain hand example and come up with solutions.

Ok..question..do calculators like Pro Poker Tools and others (like Timmys) use MCTS to figure out equity? If not how does MCTS differ from those?

If they are the same or similar that is exactly what I am doing. I use Timmys numbers to decipher what my absolute hand strength is (ie top pair, str8, nuts etc) then I write rules based on exactly what you say above, position, bet to call etc. I use both absolute and range equity together to come up with a decision.

I haven't looked at Poker Pro Tools or Timmy's work in any detail but the answer is almost certainly no. I think Timmy stuff, Poker Pro Tools, PokerStove use Monte Carlo Simulation to determine equity for one hand versus another, or one hand versus a range, or one hand versus several ranges.

MCTS is a much more ambitious undertaking. It contains models of how opponents act given an equity, position, pot etc. Then, for a given hand and board, it attempts to calculate your optimum action given your hand, position, pot etc. It does this by doing a Monte Carlo Simulation of the entire hand from the current situation to the end of the hand. By this I mean simulating a sample of the future board cards, how opponents will play, and how you will play.

And just before you scream how insane all that sounds, I have my doubts too, which is why I'm not doing it.

I have been thinking about this all day and its not really that difficult. If your ranging correctly then you have his complete hand range which also includes the true fold equity. So it really does not matter how often the guy folds the river because your only bluffing when you have nothing with high fold equity and the other times your checking down weaker pairs or betting for value. I think it all averages out generally but lets look at players on the extreme ends:

If the guy has a high river folding % (like 60+) then you might as well bet every river whether its for value or not as you would crush him on fold equity alone. So that's pretty straightforward. There are probably some other factors I am not considering but it sounds good..

Ok..now for the guy that over calls the river or something like 30% folding. Bluffing him would not work unless his fold equity is really high so in this spot so I think you would want to bet for value much thinner then normal and check down just about everything else. So anything above 50% range equity could be bet for value. Depending on the range this could be as far down as 3rd pair or what would be the bots bluffcatching range roughly.

I dunno..these are just thoughts at this point.

Well, from what you say it seems to me that you have pretty much figured the problem out in the right terms, I think it might only be a question of fine tuning for you now.
I still think it is a pretty complex problem because especially on the river that decision is the result of all the process the bot is gone thru (assigning a range pre, deciding what part of the range the guy is carrying on on the flop, how much he folds, how much he herocalls, what should our bluff freq. be and so on), and this seems so far away for me as we are just starting the project, while you seem to be at a very good point in rule based decision making.

I ask seriously, was that ironic? I think so, and I like it in some ways lol.
As I said, this is what I think me and the science guy decided, and I think I heard him saying Yes, we can do it no problem. I can be, and probably am, *very* wrong.
Time will tell!

Yeah. I think my second suggestion (card free rules system) is much better than my first (modify MCTS). Right now, MCTS looks like the quicker and easier option to you, but I'm wondering if you will still be thinking that in 2 months time. If I had a poker guy I'd build a card free rules system. It's not hard.