Poker-AI.org

Re: ruled based + montecarlo, is it a viable option?

2014-02-20T13:35:55+00:00

Hey guys, we are still working on the thing and now I am getting a programmer to work on the "stealth" part of the project.

He is suggesting that for our setup we should use two different physical machines, with one of the two grabbing poker data from the client via OCR, but he says that the OCR inaccuracy in number recognition can be a big problem (words, not much so as we have a very small vocabulary in poker).

I have noticed thou that this isn't accounted as a big problem in general by the botting community, how is that so? Am I getting something wrong here? Maybe that is because most people do the DLL injection thing and or similar and not the OCR trick?
Thank you!

Statistics: Posted by fisherking — Thu Feb 20, 2014 1:35 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-08T20:30:59+00:00

Quote:

Well, from what you say it seems to me that you have pretty much figured the problem out in the right terms, I think it might only be a question of fine tuning for you now.

Yeah the problem is that the fine tuning takes the longest to do as the little things make the biggest difference.

Quote:

I still think it is a pretty complex problem because especially on the river that decision is the result of all the process the bot is gone thru (assigning a range pre, deciding what part of the range the guy is carrying on on the flop, how much he folds, how much he herocalls, what should our bluff freq. be and so on), and this seems so far away for me as we are just starting the project, while you seem to be at a very good point in rule based decision making.

As a whole its not really complex (not like MCTS). Getting the range correct pre for me is the biggest hurdle as it obviously affects the rest of the hand and if your wrong it could really cost some big EV. As I mentioned previously the biggest problem is the bluffing ranges of the villain. Villains can have identical stats but completely different bluff ranges which has caused me some big headaches. Generally it seems most decent players will 3B bluff hands that can take a call and not just total trash. So SC's, blockers, etc.

Post flop is fairly straightforward and it comes down to basic hand combinatorics and range reduction. So of his range what % is value, air, draws. And if he is not leading the betting then what part of his range can be called etc. On each street its going to get tighter and tighter as the range he can bet for value gets smaller and smaller until your left with a very small number of hands that he could possibly reach the river with. So on the river your left with value, bluff catchers and missed draws. So now its just a matter of running his range vs hand and its gets easier to figure out what to do..either to bluff catch, bluff or value bet. But..its not just a matter of equity but also a matter of hand combinatorics to figure out how to extract max value. Like if his range is mostly missed draws you would be way better off letting him try to bluff at you vs leading out. You get the idea..

Statistics: Posted by shalako — Sun Dec 08, 2013 8:30 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-06T17:12:12+00:00

Nose wrote:

shalako wrote:

How does MC, MCTS, NN etc handle ranges? I never hear anything about that which I do not understand at all. My whole HUNL bot revolves around it. [...]

simulation based bots aren't aware of the concept or ranges. the range (or better probability distribution) in a given spot is more or less given by the probabilities for all hands to reach that spot. these probabilities are balanced (preventing exploitability) and take into consideration the probability distribution of the best possible opponent

EDIT: yay, that post made me a regular member

this was enlightening for me! ty, regular member!

Statistics: Posted by fisherking — Fri Dec 06, 2013 5:12 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-06T13:15:25+00:00

spears wrote:

Quote:

I ask seriously, was that ironic?

Yeah. I think my second suggestion (card free rules system) is much better than my first (modify MCTS). Right now, MCTS looks like the quicker and easier option to you, but I'm wondering if you will still be thinking that in 2 months time. If I had a poker guy I'd build a card free rules system. It's not hard.

That highly depends on the knowledge of the poker guy and his ability to generalize his thoughts. Too much knowledge might actually be hindering - even if the edge was greater

Statistics: Posted by Nose — Fri Dec 06, 2013 1:15 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-06T13:06:24+00:00

Quote:

I ask seriously, was that ironic?

Statistics: Posted by spears — Fri Dec 06, 2013 1:06 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-06T11:37:40+00:00

spears wrote:

Quote:

P.s.: you are obviously talking about rethinking the thing from scratch, I am just hoping that a viable solution could also be to heavily review MCTS optimizing it more than doing a new one from zero.

Good luck with that.

I ask seriously, was that ironic? I think so, and I like it in some ways lol.
As I said, this is what I think me and the science guy decided, and I think I heard him saying Yes, we can do it no problem. I can be, and probably am, *very* wrong.
Time will tell!

Statistics: Posted by fisherking — Fri Dec 06, 2013 11:37 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-06T11:34:09+00:00

shalako wrote:

I have been thinking about this all day and... CUT

Well, from what you say it seems to me that you have pretty much figured the problem out in the right terms, I think it might only be a question of fine tuning for you now.
I still think it is a pretty complex problem because especially on the river that decision is the result of all the process the bot is gone thru (assigning a range pre, deciding what part of the range the guy is carrying on on the flop, how much he folds, how much he herocalls, what should our bluff freq. be and so on), and this seems so far away for me as we are just starting the project, while you seem to be at a very good point in rule based decision making.

Statistics: Posted by fisherking — Fri Dec 06, 2013 11:34 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T22:39:17+00:00

Quote:

Obviously every opponent has his tendencies thou, so it is important to also look at specific stats, like how much he bet-folds river in general and possibly his frequencies on some given board textures (draw completed, draw missed, A-K fallen on the riv as overcards et cetera), how much he bluffcatches and so on. At midlowstakes I see very different bet-fold river %, for example, ranging from 30 to 70+ over decent samples. I see it as a pretty difficult problem to address, in any case.

I have been thinking about this all day and its not really that difficult. If your ranging correctly then you have his complete hand range which also includes the true fold equity. So it really does not matter how often the guy folds the river because your only bluffing when you have nothing with high fold equity and the other times your checking down weaker pairs or betting for value. I think it all averages out generally but lets look at players on the extreme ends:

If the guy has a high river folding % (like 60+) then you might as well bet every river whether its for value or not as you would crush him on fold equity alone. So that's pretty straightforward. There are probably some other factors I am not considering but it sounds good..

Ok..now for the guy that over calls the river or something like 30% folding. Bluffing him would not work unless his fold equity is really high so in this spot so I think you would want to bet for value much thinner then normal and check down just about everything else. So anything above 50% range equity could be bet for value. Depending on the range this could be as far down as 3rd pair or what would be the bots bluffcatching range roughly.

I dunno..these are just thoughts at this point.

Statistics: Posted by shalako — Thu Dec 05, 2013 10:39 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T18:16:28+00:00

Quote:

Ok..question..do calculators like Pro Poker Tools and others (like Timmys) use MCTS to figure out equity? If not how does MCTS differ from those?

I haven't looked at Poker Pro Tools or Timmy's work in any detail but the answer is almost certainly no. I think Timmy stuff, Poker Pro Tools, PokerStove use Monte Carlo Simulation to determine equity for one hand versus another, or one hand versus a range, or one hand versus several ranges.

MCTS is a much more ambitious undertaking. It contains models of how opponents act given an equity, position, pot etc. Then, for a given hand and board, it attempts to calculate your optimum action given your hand, position, pot etc. It does this by doing a Monte Carlo Simulation of the entire hand from the current situation to the end of the hand. By this I mean simulating a sample of the future board cards, how opponents will play, and how you will play.

And just before you scream how insane all that sounds, I have my doubts too, which is why I'm not doing it.

Statistics: Posted by spears — Thu Dec 05, 2013 6:16 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T17:51:23+00:00

spears wrote:

OK, I've thought of another way poker and physics guy could work together. Physics guy calculates equity and variance (see above). Poker guy provides the rules for actions based on equity, variance, position, pot, stack size etc, but without any consideration of actual cards. This is much simpler for physics guy and gives poker guy more to do than in my MCTS suggestion.

Ok..question..do calculators like Pro Poker Tools and others (like Timmys) use MCTS to figure out equity? If not how does MCTS differ from those?

If they are the same or similar that is exactly what I am doing. I use Timmys numbers to decipher what my absolute hand strength is (ie top pair, str8, nuts etc) then I write rules based on exactly what you say above, position, bet to call etc. I use both absolute and range equity together to come up with a decision.

Statistics: Posted by shalako — Thu Dec 05, 2013 5:51 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T17:16:09+00:00

Quote:

If the villains stats show tendencies to over fold I have it disregarding combos and perceived ranges and just leading out when it has nothing just because of the fact I will always get value from my bluffs. What I didnt ever consider was what you stated above about the villain folding more or less on certain boards/cards fall although I have considered rivers in which all the draws missed and bricks hit. But..range equity will already reflect that to a large degree wont it? I mean whether certain cards help or hurt his range would be auto reflected in the range equity calc.

This is assuming your assigning the correct range which is not easy to get right. I spent a month working on a complicated range equity finder based on how pros put people on hands. It is surprisingly accurate but it does completely get it wrong quite often (like his actual hand was not in the bots range). With that being said its not often enough for me to change it yet however. Generally the problems lie with the 3B bluff range and the villains slowplaying range (although I have worked that out to some degree). Some people believe that unknown hands should be weighted and never be removed which is something I really need to consider.

I would like to work on this river problem you see as being difficult. I think I may have all the pieces to solve it but maybe we can work on a certain hand example and come up with solutions.

Statistics: Posted by shalako — Thu Dec 05, 2013 5:16 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T16:54:59+00:00

Quote:

Good luck with that.

Statistics: Posted by spears — Thu Dec 05, 2013 4:54 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T16:51:51+00:00

Quote:

If the problem were really this hard then humans couldn't solve it either, and the current Nash Equilibrium solutions wouldn't exist. Or do you believe humans have magical properties? The key to solving the problem is in using good abstractions that reduce the trillions of of states to more manageable numbers. Sonia already solved this problem years ago.

Statistics: Posted by spears — Thu Dec 05, 2013 4:51 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T15:12:54+00:00

spears wrote:

Over time, you will see opportunities to replace human generated rules with rules generated from the data.

sounds wery beautiful, but it is imposible (not near 1000 years).
poker fsm has more then millions of millions of trillions posible states. And in each state your model should do something. How many data you need to generate this LUT by statistic?

Only if you make some assumptions. But on what are they will be based? Only on human model. example
Here we come to the start

Statistics: Posted by nefton — Thu Dec 05, 2013 3:12 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T11:41:35+00:00

spears wrote:

Quote:

I also like total control of the bots actions so when I see an error I can fix it easily and fast. I have heard that is not so easy on simulation based bots or ?

Agreed. This was part of the motivation for my second suggested architecture.

A bit more on the second suggestion...

Separation of concerns http://en.wikipedia.org/wiki/Separation_of_concerns is an important concept in software development, and I suspect in scientific thought in general. And this is a problem for you as well as physics guy. When you are composing rules you are engaged in software development. Condensing the cards down to two parameters (equity and variance) allows everyone involved to separate that aspect from all the others like position, pot size, stack size, etc. It should make your rules an order of magnitude simpler. It may be that equity and variance are not enough to describe the cards. If that is the case I would love to hear what you perceive the problems to be.

The more I think about it the more I think this architecture is a good platform for future development. Over time, you will see opportunities to replace human generated rules with rules generated from the data. MCTS has not been a huge success so far, and I don't think it is commonly understood why that is the case.

Gotta love your suggestion, sounds pretty interesting, and again, can't wait to see what the statistics guy says about this, as I have heard him saying that MCTS is potentially very very powerful and that he could help him getting dramatically better also with my help, which I hope I'll be able to supply.
Thank you again.

P.s.: you are obviously talking about rethinking the thing from scratch, I am just hoping that a viable solution could also be to heavily review MCTS optimizing it more than doing a new one from zero.

Statistics: Posted by fisherking — Thu Dec 05, 2013 11:41 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T11:35:59+00:00

shalako wrote:

All of this is a learning experience and the more I study and ask questions the better and more fun this is.

yep, I take it with the same attitude, it is a big interesting challenge, let's see where me and my friend will end up to, I'm very excited at the moment also because I know for a fact that the physicist is a very talented mind in the field of statistics; I hope to add to the process the little bit I can to speed it up (other than offering my insight on level of bot's play, players, games to play, rooms to attack etc., as I know quite a bit about the poker world).

shalako wrote:

fisherking. here is the thread on bluffing and perceived ranges:

http://poker-ai.org/phpbb/viewtopic.php?f=24&t=2566

look over it and see if you see anything that I was missing. I am still a bit confused on what formulas I should be using. I have seen several and I am not sure which one to use.

I checked that out, and it seems you are getting it quite right, I mean, it all makes sense to me, but for the moment I don't think I could give you any useful suggestions, as I have not yet approached the problem in a botter perspective, so to speak. As soon as I'll reach that point in the making of the AI and I discuss this with my friend I'll get back to you, hopefully being of some better help.

Right now I would only say what you already know: that according to Sklansky in a vacuum you should bluff a percentage similar to the odds you are giving to your opponent with the bet, and such too generic things. But I repeat, it all sounded just about right, minimum 10 representable combos, plus that 30ish% of times you bluff (given that you bet more than 3/4 pot) sounds ok for sure. Obviously every opponent has his tendencies thou, so it is important to also look at specific stats, like how much he bet-folds river in general and possibly his frequencies on some given board textures (draw completed, draw missed, A-K fallen on the riv as overcards et cetera), how much he bluffcatches and so on. At midlowstakes I see very different bet-fold river %, for example, ranging from 30 to 70+ over decent samples. I see it as a pretty difficult problem to address, in any case.

Statistics: Posted by fisherking — Thu Dec 05, 2013 11:35 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-05T10:16:05+00:00

Quote:

I also like total control of the bots actions so when I see an error I can fix it easily and fast. I have heard that is not so easy on simulation based bots or ?

Statistics: Posted by spears — Thu Dec 05, 2013 10:16 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T19:49:19+00:00

Statistics: Posted by shalako — Wed Dec 04, 2013 7:49 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T18:29:17+00:00

Quote:

Have you checked out blah and inprova on deucescracked? They've got some great stuff on perceived ranges.

No I haven't but I will check it out. I put in a considerable amount of code dealing with this so I think its pretty close to being right but I will read those articles and see if I can pick up anything new. Thanks!

Basically what I did was run sims to get my bots stats..so the same stats that the villains HUD would see to form the perceived range. After that it gets a bit more complicated but it works. The problem is that it caused a big surge in cpu usage on a decision by adding these routines.

Statistics: Posted by shalako — Wed Dec 04, 2013 6:29 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T18:21:04+00:00

Quote:

This is a bit of an understatement. I think it depends on what you're aiming for - beating the micros maybe, but a winning SSNL rules based bot would be extremely difficult to construct even for someone with the domain knowledge to beat that level.

Well it has not been easy and in the last year I had to totally hit the books/videos/blogs in order to get a better understanding of NL game theory as most of my bot experience up to that point was mostly FL although I did write a decent 9max NL50 Zoom bot when it first came out. Nasher got me into HU which started me thinking on how very little I knew about game theory. All my NL experience is live MTT play and very little cash. I think videos are the best learning tool.

My aim at this point is not very high..maybe 100NL tops. I believe the power of the bot is its ability to play multiple tables without error so I am looking to win a small amount and make up the difference by putting in a massive amount of hands per hour.

Code:

To Shalako, who seems very experienced in the field so probably doesn't need any suggestions by me, I would like to suggest not to be overvaluing the importance of polarization. Polarizing is a very very important concept but it can also be tricky, for example you can't be always polarized on rivers as it makes pretty easy for attentive players to bluff catch if you don't bet also your mid range of value, as you end up with too few value combos to bet with. Another example can be the 3bet preflop, where you don't want to be polarized OOP vs a high caller (>35%) who's not too prone to folding to cbets in 3bet pots (fcb3b<40%).

I have only polarized preflop and the flop and not the turn and river for just the reasons you mention. So instead of polarizing the turn the one thing I did do was run sims to make sure I was not over folding the turn which is a common leak when facing a bet. Luckily it was perfect and I didn't have to make any changes. I still have work to do on the river but generally I am happy with it. I do need to work on using more hand combinatorics for river betsizing to try to extract maximum value. Like if I have the nuts and he has bluffcatchers in his range he would be more likely to call a pot or 2x pot bet vs a half pot value bet because it looks like a bluff. I think...I am not really sure if I am doing this correctly yet...

Interesting you point out that 35% number which is critical one for calling OOP. I think ideally its 38% but you want to just be nearly over calling by just a smidge. I have only polarized its baseline preflop strategy but its changing quite a bit based on the players stats adding value or air where needed. I think the toughest part of preflop is getting the villains 3 bet bluffing range right. That range can vary widely with the player and I struggle with it. I settled on what I think is the best compromise after consulting many online players on how they handle it. One thing I keep wanting to do is use showdown information to verify I am ranging correctly. Its on the list...

I like all the suggestions you can give me! All of this is a learning experience and the more I study and ask questions the better and more fun this is.

spears: we are all working on the same goal so I really enjoy your input on simulation related stuff which to me is way above my head! I stick with rules because they make sense to me and have worked for me in the past but I am always open to anything new. I also like total control of the bots actions so when I see an error I can fix it easily and fast. I have heard that is not so easy on simulation based bots or ?

Statistics: Posted by shalako — Wed Dec 04, 2013 6:21 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T13:00:36+00:00

Wow. I must be going soft. Civil interactions with two rules guys in 24 hours

Statistics: Posted by spears — Wed Dec 04, 2013 1:00 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T11:13:23+00:00

spears wrote:

I've just thought of a productive way you guys could work together that has quite good long term potential. There is some java code for a MCTS bot somewhere. It almost certainly needs some poker value judgements to make it work well so you could do that. IIRC it doesn't know how to bluff or slow play so you could introduce that. Get a crack of PokerAcademy to test it and to spew out diagnostics to help you tune it.

Hey Spears, thank you for your kind help in this.
What you are suggesting is more or less what we are already doing. So I guess I was terrible at explaining it, I'm so sorry, it must be my terrible english plus my approximate usage of the "bot language", so to speak.
When I asked about putting a rule based core in a Montecarlo AI was talking more or less about this, my bad, I'm totally new to the bot's world.
Will get better, I promise, I'm a fast learner

spears wrote:

The tension between those advocating rules based approaches and pure mathematical approach is quite deep and quite common. I'm surprised fisherking can get physics guy to play along.

Well, I see what you are saying here as we had a lot of interesting conversations about this, but eventually we both got convinced that given that our goal is *not* to totally solve the game in GT, we could use some shortcuts, or rules, to get the AI past all the things we already know as reasonably "correct" (e.g.: rule says to open 2.5x with a determined range from UTG instead of trying all the combinations of amount to open + any two cards and see in the tree what is profitable and what is not - which is what MCTS does at the moment). Plus, the poker player checks on frequencies, bluffs, slowplay et cetera, exactly as you suggest.

I have to say that the physicist totally agrees with me that the advantage we have, should we have one, on the normal approach to a poker AI is that we have a good insight on both the two worlds, science of probabilities and poker, so we do believe we should use each others' knowledge to get an AI which takes the best of the two approaches.

By the way I already got MCTS and simplebot playing each other on testbed, but I noticed that Holdem Manager won't read the hand history generated, not even if I reformat the HH in xml or whatever format I know. I can see the two guys playing via a universal replayer, which is fast and so I can easily understand what they are doing, but I can't access that with the tracking software.
Is the poker academy crack HH readable by holdem manager? or should I pass to pokertracker, of which I heard is more elastic with HHs?

spears wrote:

As for this, I can't wait to see what my friend says. In general, for all the interesting technical stuff pointed out in this thread, I already told the physicist to come here and interact. As I said he's currently traveling for work, hope he gets back here soon.

Thank you everybody.

To Shalako, who seems very experienced in the field so probably doesn't need any suggestions by me, I would like to suggest not to be overvaluing the importance of polarization. Polarizing is a very very important concept but it can also be tricky, for example you can't be always polarized on rivers as it makes pretty easy for attentive players to bluff catch if you don't bet also your mid range of value, as you end up with too few value combos to bet with. Another example can be the 3bet preflop, where you don't want to be polarized OOP vs a high caller (>35%) who's not too prone to folding to cbets in 3bet pots (fcb3b<40%).

Statistics: Posted by fisherking — Wed Dec 04, 2013 11:13 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T09:58:20+00:00

shalako wrote:

Quote:

Even if I was interested in writing a rules based based system it's not an option for me because I don't know the rules.

Learning the rules is not easy I must admit. I have adopted poker coaching sites in order to understand game theory. One guy in particular I have focused on.

Quote:

Well that sounds like a good combination to me. Bluffing and slowplaying are fairly easy to figure out using hand combinatorics.

Slowplaying is rather straightforward and the frequency should be balanced with your air ideally. What I mean by this is that if your betting air 25% of the time you should be check back value 25% of the time as well to remain balanced. To find the frequency just run sims to see how often you have TP+ on the flop which is what I did. I cannot remember what mine turned out to be.

Bluffing is a bit more complicated in the fact that your perceived range is very important. You have to sell the the bluff and to do that your actions up to that point need to coincide with the hand your trying to represent. My bot will bluff when it has no showdown value and can represent at least 10 value combos (I have a thread here on how to figure that out which is a pain) based on its actions and there is enough fold equity in the villains range to pull it off.

Alot of this goes out the window if the villain is nitty or passive so you have to step up the pressure if he checks or folds too often in certain spots. Against those types I have incorporated more floats with complete air in order to take the pot away on a later street because they are less likely to bet the turn unless they really have the goods. Generally I only float in position but a check raise is a powerfull tool too...

Have you checked out blah and inprova on deucescracked? They've got some great stuff on perceived ranges.

Statistics: Posted by fraction — Wed Dec 04, 2013 9:58 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T08:10:42+00:00

Statistics: Posted by spears — Wed Dec 04, 2013 8:10 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-04T00:39:01+00:00

Quote:

Even if I was interested in writing a rules based based system it's not an option for me because I don't know the rules.

Learning the rules is not easy I must admit. I have adopted poker coaching sites in order to understand game theory. One guy in particular I have focused on.

Code:

I thought the approach I advocated for fisherking had some benefits both for him and physics guy. It would be an extensible platform that would allow poker guy quite a lot of discretion in the things that MCTS is bad at - like bluffing and slow play frequencies and opponent assessment. Anyway, that's enough shouting across no-man's land for tonight: I do enough of this in the day job as it is.

Statistics: Posted by shalako — Wed Dec 04, 2013 12:39 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T23:36:53+00:00

Quote:

I have nearly no idea what this means, and no clue how to get a number out of it. In my terms I think you are saying "call hands with high equity but medium variance, raise hands with low equity but high variance"? But that isn't a new parameter, it's just a (reasonable) rule based on my existing two parameters.

Right..that is basically what I am saying..but..you have have to polarize it a bit so your not doing the same thing all the time. But in general yes..you should be raising/betting your lower equity/high variance draws more often then your higher equity/lower variance draws because of equity realization. I am pretty sure that is good GTO advice but dont quote me on it! Some of this GTO stuff I am still trying to learn but it does make sense.

Statistics: Posted by shalako — Tue Dec 03, 2013 11:36 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T22:41:45+00:00

Quote:

I can see how this controversy between rule guys and sim guys can run deep. For one thing I think solving 100bb+ is way off like you say so rules seems like the better choice at the moment. I think Nasher is going to run simulations until rapture and by then it wont matter!

Not all math approaches are the same. Nasher is working on Nash Equilibrium, but MCTS is an exploitative approach. There are quite a few variants of both of these approaches. Personally I think Nash Equilibrium is close for nearly all games, but actually isn't a worthwhile goal because it doesn't maximise income and is easily detectable. What little time I have for development these days is devoted to a hybrid approach. Even if I was interested in writing a rules based based system it's not an option for me because I don't know the rules.

I thought the approach I advocated for fisherking had some benefits both for him and physics guy. It would be an extensible platform that would allow poker guy quite a lot of discretion in the things that MCTS is bad at - like bluffing and slow play frequencies and opponent assessment. Anyway, that's enough shouting across no-man's land for tonight: I do enough of this in the day job as it is.

Statistics: Posted by spears — Tue Dec 03, 2013 10:41 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T22:23:41+00:00

Quote:

Well I have adopted several "low equity" plays into my bot that most pros use. Anything with 3-5 outs is what they are. Gutshots, Pair and backdoor draw, backdoor draw and overcard, and the worst being backdoor str8 and flush draw combo (3 outs). All these plays do generally is make the bot more aggressive on the flop and provide more turn barreling opportunities. But..the real value of these hands come from being very disguised so when you hit you get more value out of them vs a standard open draw. Kick in any fold equity you have on the turn and these plays start to look pretty good.

Generally (I use this loosely because of predictability) you want to raise lower equity draws and call higher equity hands like open draws. It sounds counter-intuitive but you want to "realize your equity". This basically means you don't want to get blown off your good draws. But you don't always want to do that so you have to check back/call many of your good draws too. Many people make the mistake of raising the nut flush draw more often then not. The problem with this is that you will possibly push a weaker draw off the hand which is losing value.

Statistics: Posted by spears — Tue Dec 03, 2013 10:23 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T22:12:25+00:00

Poker Paradime was a site featuring a good exploitative limit bot that you could play against amongst other things.

https://www.google.co.uk/search?q=site% ... 2&ie=UTF-8

https://www.google.co.uk/search?q=site% ... 2&ie=UTF-8

https://www.google.co.uk/search?q=site% ... .com+sonia

Statistics: Posted by spears — Tue Dec 03, 2013 10:12 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T21:53:56+00:00

Quote:

So in my model the plethora of cases that you describe (like a pair and backdoor FD, or a backdoor FD and Str8) is reduced to 2 parameters. Maybe two parameters is too few, so if you have some suggestions for some other parameters I'm listening.

Quote:

The tension between those advocating rules based approaches and pure mathematical approach is quite deep and quite common. I'm surprised fisherking can get physics guy to play along. Ultimately (and that might be a long way off), the math approach will win. Arguably it already did at Poker Paradime.

What is/happened at Poker Paradime?

I can see how this controversy between rule guys and sim guys can run deep. For one thing I think solving 100bb+ is way off like you say so rules seems like the better choice at the moment. I think Nasher is going to run simulations until rapture and by then it wont matter!

Statistics: Posted by shalako — Tue Dec 03, 2013 9:53 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T20:19:50+00:00

@shalako

They adapt. And that can be quite clever statistically. With the limited stats they have they find from their database of many other players how villain is likely to play in spots they have not yet encountered.

A LUT is just a fast way of calculating something, usually in this case the probability of one hand beating another. But I also use the variance (http://en.wikipedia.org/wiki/Variance) of pwin. IMO this captures the "drawiness" of the hand. (A draw has lots of drawiness) So in my model the plethora of cases that you describe (like a pair and backdoor FD, or a backdoor FD and Str8) is reduced to 2 parameters. Maybe two parameters is too few, so if you have some suggestions for some other parameters I'm listening.

The tension between those advocating rules based approaches and pure mathematical approach is quite deep and quite common. I'm surprised fisherking can get physics guy to play along. Ultimately (and that might be a long way off), the math approach will win. Arguably it already did at Poker Paradime.

Statistics: Posted by spears — Tue Dec 03, 2013 8:19 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T19:44:40+00:00

Quote:

Ah ok. I understand now.

So do they recognize weaknesses or do they just play "balanced" all the time? Like lets say the villain folds to 60% of 3 bets. My bot will start to 3 bet very wide until the guy adapts and it becomes unprofitable. Does the simulation bot adapt or just play the same way for all situations?

It should also adapt in the opposite manner too..meaning if the guy is 3 betting the hell out of me I start calling wider in position and 4 betting/shoving more until he gets the message its not going to be tolerated...

Statistics: Posted by shalako — Tue Dec 03, 2013 7:44 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T19:28:18+00:00

shalako wrote:

How does MC, MCTS, NN etc handle ranges? I never hear anything about that which I do not understand at all. My whole HUNL bot revolves around it. [...]

Statistics: Posted by Nose — Tue Dec 03, 2013 7:28 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T17:39:30+00:00

How does MC, MCTS, NN etc handle ranges? I never hear anything about that which I do not understand at all. My whole HUNL bot revolves around it. Not only do you have to get range accurate (not easy against a pro who polarizes) but also dissect that range using hand combinatorics and the board to reduce his range all the way to the river (and I wont even mention dissecting the bots own perceived range for bluffing). A basic strategy without ranging will lose so how does a LUT handle that?

Also..I think you do have to break down what your hand is instead of using a LUT. One thing that LUTS do not reflect is certain hand types (like a pair and backdoor FD, or a backdoor FD and Str8) give you more turn barreling opportunities. Mathematically they suck but by showing aggression on the flop and leading the turn many players will just give up marginal holdings. Even if you miss the turn you can use hand combinatorics to figure out the correct fold equity which might allow you to barrel again anyway. Any villain not holding TP+ will think twice about calling the turn.

Statistics: Posted by shalako — Tue Dec 03, 2013 5:39 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T16:10:49+00:00

spears wrote:

There is some java code for a MCTS bot somewhere.

In the opentestbad there is a full MCTS bot framework: https://github.com/corintio/opentestbed .
There are some discussions about it and how to customize it in the archived forums.

Statistics: Posted by corintio — Tue Dec 03, 2013 4:10 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T15:40:45+00:00

Statistics: Posted by spears — Tue Dec 03, 2013 3:40 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T15:10:28+00:00

Quote:

Sorry, this must be a semantic problem as this is the poker player writing. I meant to ask if you can do a decent rule based bot and then let a montecarlo and NN based AI perfect its strategies. Or, in general, if this is a viable approach or not.

Still makes no sense to me. How do the rules and MC approaches play together? Could you give a simple example?

Quote:

Should we proceed with the idea we have now, we'll set some simple rules of engagement, so to speak. For example the bot will raise cbets on a class of strategically similar flops when his holding is in an assigned rank (TPTK+, MidPair + backdoor FD etc.) and the opener is in a steal position and his cbet % is a certain value and the bot is IP or OOP and so on.

OK. Here is something I can comment on. Instead of some long winded description of "strategically similar" such as (TPTK+, MidPair + backdoor FD etc.) you could reduce this to a small number of parameters. For example "probability of win" and "standard deviation of probability of win". Draws have high st dev. And then calculation of these quantities can be very quick if you use precalculated lookup tables. Search around the forum for LUT.

Statistics: Posted by spears — Tue Dec 03, 2013 3:10 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T11:49:54+00:00

shalako wrote:

CUT

We are *very* thankful for your advice, sir. Ty.
This is already helping our better understanding of the long way we have in front of us.

Statistics: Posted by fisherking — Tue Dec 03, 2013 11:49 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-03T11:53:03+00:00

spears wrote:

mainly I'd like to hear what you think about implementing a rule based core in a montecarlo AI

- What does this mean exactly?

spears wrote:

- Do you model the opponents?

We will, in the montecarlo phase. But maybe obv we could have to change this approach and implement oppos modeling in the rule based core.

spears wrote:

- Does chance play a part in the action you calculate?

You mean to ask if we are making the bot so that he will not always do the same things in the same situations? If so, yes. He will have a part of his range with which he'll do some times x and some times y.
If not, again I'm sorry, it must be my poor understanding of the matter in a strictly theoretical mathematical sense. I'll ask to the physicist and come back (he's doing conferences abroad at the moment).

spears wrote:

- Can describe what you propose to do with the board post flop

Statistics: Posted by fisherking — Tue Dec 03, 2013 11:46 am

Re: ruled based + montecarlo, is it a viable option?

2013-12-02T18:02:43+00:00

Quote:

I really can see how complex and long can be to put together a high level rules based bot with a complete strategy for every street of play, so this in my thoughts could be a good compromise between the two approaches, am I right in this?

Ok..its not difficult at all to put together a high level rules based bot. It really isn't. I have been coding rules based bots for 10 years and I still have not seen a reason to deviate to a simulation based approach even partially and I really considered it this year for HUNL. You would be surprised about how easy it really is once you start coding but it will take lots of time to fine tune it.

Once you code up a basic strategy then put it in play money to find leaks and make adjustments. Play money is good because you will see a wide range of player types and crazy bet sizes. Take notes on mistakes it made. Fix them and put it back in the ring until you think its ready. This process could take you months but I believe it is the best way to do it. Another method I use is to test it against itself to see how it handles a particular situation and to see what its stats would be to the villain. An example is checking to see how often it c bets the flop and what its range is composed of. This is key to check how polarized it is. Like you do not want your bot to heavily c bet only value hands. This would be a leak.

Use PT or write your own stats database for opponent modeling then write rules to adapt to various stats. I have found the villains folding stats to be the most important for adapting/exploiting (3BF, 4BF, Turn fold etc) and all the others for correct ranging (PFR, 3BC etc). This may seem obvious but its easy to overlook some of the stats that make a huge impact on how well the bot plays.

So..write a basic core strategy that does not adjust at all. Once that is done then write the rules to property range the villain (Not as easy as it sounds and probably the most difficult aspect of bot writing), then write code based on the villains stats to either adapt or exploit his weaknesses.

I have found it to be better to write a hyper aggressive bot and force the villain to adapt to you instead of vice versa. Passive bots are usually very predictable and thus are exploitable.

Statistics: Posted by shalako — Mon Dec 02, 2013 6:02 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-02T15:18:15+00:00

Quote:

mainly I'd like to hear what you think about implementing a rule based core in a montecarlo AI

- What does this mean exactly?
- Do you model the opponents?
- Does chance play a part in the action you calculate?
- Can describe what you propose to do with the board post flop

Statistics: Posted by spears — Mon Dec 02, 2013 3:18 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-02T13:16:03+00:00

Yes, infact I already had to do a very detailed legenda for the maths guy to get into the poker vocabulary, and I totally see that it might take some time for us to understand each other, but it is quite some time now that we are talking and working on it and things are getting at least started.

I was a little verbouse in my other post, and I beg your pardon for that; mainly I'd like to hear what you think about implementing a rule based core in a montecarlo AI, which was actually my idea because the scientist wanted to go thru all the decisions via more or less a GTO approach.

I really can see how complex and long can be to put together a high level rules based bot with a complete strategy for every street of play, so this in my thoughts could be a good compromise between the two approaches, am I right in this?

For example resolving the preflop play can be a really harsh and long task for a MC AI, while compiling a really valid, even high level preflop strategy in a rule based bot in not hard at all, takes like not more than 5 hours of work, I mean, I did a decent one in more or less that period of time, using a range chart that my friend compiled for the purpose and that the simplebot can read and use.

Thank you!

Statistics: Posted by fisherking — Mon Dec 02, 2013 1:16 pm

Re: ruled based + montecarlo, is it a viable option?

2013-12-02T12:31:51+00:00

Personally, I think a pure rules based bot is a quick solution for low stakes play, but has little potential for improvement. I think it is much harder than most people imagine to collate all the rules that would ever be required for a really top class player, even supposing you are one.

But I think cooperation between a scientific approach and a poker expert could work. For example, the scientific approach can easily calculate the odds of one hand beating another, but an expert could compose rules to recognize fish much quicker than a scientist. I wouldn't expect this cooperation to be easy, because you will be using a different vocabulary

When I think of more specific cases of how an expert could contribute I'll post them.

Statistics: Posted by spears — Mon Dec 02, 2013 12:31 pm

ruled based + montecarlo, is it a viable option?

2013-12-02T12:00:52+00:00

Hi,
we just presented ourselves in another section of this very interesting forum.

We are a team of two people, a very experienced poker player and a world class physicist specialized in stochastic and montecarlo simulations and all that jazz. The physicist knows nothing about poker, the poker player knows nothing serious about maths and programming.

We are working on an AI for poker, and our current idea is to tune an opensource already compiled poker AI with a heavy rule based strategy and then implementing the result into a more evoluted montecarlo-based AI with opponents modeling etc.
We wanted to know if you guys thought this was a viable path to a decent poker AI or not.

Basically what we are asking is also: do you think a rule based bot could be the core for a NN and/or montecarlo based AI? This given that our intent is not to gain scientific recognition, or to solve the game once for all, but "simply" to end up with a BEish bot for the low to the mid stakes (say up to 200 nlh 6max). And obviously the team already knows for a fact what is profitable and what is not in, say, preflop play, so we don't need for a montecarlo AI to go thru all the gigantic magma of implementing a basic solid and winning strategy - at least we don't need it to do that for some specific and almost already solved parts of the game, as preflop, or Cbetting or something in the like.

At the moment the poker player is giving the physicist all the "correct" opening ranges, flatting ranges, 3bet etc. per position vs specific positions, and the guy is implementing those in something like simplebot (maybe that is *exactly* it, sorry for the inaccuracy, the poker player is writing this).

What you guys think about it? Do you think we could make cooperate our different skills better?
And do you think a so composed team can do good things in the field? We think this could be a very good team in theory but maybe there are things we are overlooking at.

(We plan on hitting a real programmer with some money to do the "stealth" things when and if we have a decent AI).

Any suggestions would be very much appreciated!

Ty!

Statistics: Posted by fisherking — Mon Dec 02, 2013 12:00 pm