Poker-AI.org

Re: Expert/Rule vs Simulation based systems

2013-07-14T12:31:48+00:00

Hi,

On reducing CPU load - you can do some amazing things with pre-calculated tables. I used to have something like a Tb of precalculated tables - any one hand would only use a tiny fraction of that but it allowed calculations that would have been impossible if attempted in realtime.

On rule based vs simulation based - I always favored a hybrid approach. ie a mix of rule based decisions and simulations - and even rules based on metrics that came from simulations - or simulations that include rules to estimate parts of the tree that would be too expensive to compute otherwise.

- PeppaPig

Statistics: Posted by PeppaPig — Sun Jul 14, 2013 12:31 pm

Re: Expert/Rule vs Simulation based systems

2013-05-28T15:25:17+00:00

Yes I have an SSD but I found the RAM disk to work better obviously since there is no disk access at all. I do use multithreading but it does not seem to help in this instance as it is just very cpu intensive. I just need to think of a better way to code this and reduce the amount of processing time. The problem is no ideas are coming to mind! I cannot think of a better way to do it as of yet.

Statistics: Posted by shalako — Tue May 28, 2013 3:25 pm

Re: Expert/Rule vs Simulation based systems

2013-05-27T19:06:23+00:00

Do you use multithreading?

Statistics: Posted by HontoNiBaka — Mon May 27, 2013 7:06 pm

Re: Expert/Rule vs Simulation based systems

2013-05-25T16:28:06+00:00

If you're doing a lot of hard disk reading or writing, a SSD will improve response time significantly.

Statistics: Posted by birchy — Sat May 25, 2013 4:28 pm

Re: Expert/Rule vs Simulation based systems

2013-05-24T21:38:10+00:00

Quote:

FWIW, 3 seconds to make a decision is not abnormal. Your bot will have to wait some random time before clicking buttons anyway, so why not use that time for number crunching? You can also reduce a fair bit of CPU usage by using lookup tables for commonly used values. Things like opponent stats can be calculated on the fly and stored in a database rather than looping through hand histories, etc. I also use small tables where the AI can lookup some action or value based on position, effective stack, pot size, win EV, etc.

Well I like to play at least 8 tables at once (per computer) so having it take up to 3 seconds on each street post flop is not going to work at all. Speed was never an issue until I wrote this last complicated routine to put the villain on a hand and then run range equity. I have all the common values done in under a second (board and pot info, hole card info, win EV etc ). I have thought about other ways to do it (like lookup tables) but I have not come up with any solutions yet except for a faster computer. Using a RAM disk cut the speed in half but my 2.1ghz Intel Core 2 Duo processor is just not cutting it anymore.

Statistics: Posted by shalako — Fri May 24, 2013 9:38 pm

Re: Expert/Rule vs Simulation based systems

2013-05-24T19:35:50+00:00

shalako wrote:

It is working pretty well but is consuming too much cpu time in order to come up with the answer so I am going to have to get a new box as 3 seconds is just too slow.

Statistics: Posted by birchy — Fri May 24, 2013 7:35 pm

Re: Expert/Rule vs Simulation based systems

2013-05-24T16:39:06+00:00

Quote:

Yeah I started with FL. Two years of AI writing before it won decent money then FL vanished. I have a killer FL AI and its really not even worth using now as there is just not enough loose tables to make it worthwhile. I remember PokerPal..My first bot used Holdem Inspector before I finally wrote my own calculator.

Quote:

In the short and medium term there is no good reason to change, especially if you have expert knowledge. But I reckon that eventually a rigorous mathematically based approach will win, just like it has in most other games like chess and backgammon. Eventually might be a very long way off.

Ah ok..that makes sense. I have heard that 100BB is somewhat within reach but 200BB is way way down the road I guess. One pro I ran into said "200BB will not be solved in my lifetime".

Quote:

There is nothing really wrong with expert systems other than two things:
1) They take a lot of time to write down and lots of iterations to plug holes that you will inevitably create in your IF...ELSE statements.
2) You need an expert that knows what he is doing and is willing to share his knowledge.

In academia, the primary goal is to develop solutions completely disconnected from expert knowledge but instead based on game theory. The only thing you know is the rules of the game and now you're supposed to create a good (or perfect) strategy. That's where simulation, NNs and other approaches come into play.

Ok..well I will continue on my IF THEN journey for now then. I do have access to pro knowledge and have learned more about poker theory in the last 6 months then I have in the last 10 years. One particular bit of info changed my whole way of thinking of how poker should be played. The difficult part was programming the bot to come up with the same answer as the pro. It is working pretty well but is consuming too much cpu time in order to come up with the answer so I am going to have to get a new box as 3 seconds is just too slow. This is the complicated routine I have using to narrow the villains range based on his actions and board. I think what I need to do to fine tune it is run it against hand histories to determine its accuracy.

Thanks for everybodys insight as I have a better understanding of what simulation systems do.

Statistics: Posted by shalako — Fri May 24, 2013 4:39 pm

Re: Expert/Rule vs Simulation based systems

2013-05-24T03:02:38+00:00

shalako wrote:

Maybe I am already using a form of NN but do not even know it?

Your brain.

Statistics: Posted by cantina — Fri May 24, 2013 3:02 am

Re: Expert/Rule vs Simulation based systems

2013-05-23T22:32:45+00:00

shalako wrote:

Since these guys really know how to play why should I detour to a simulation based system? My bot already adapts both preflop and postflop (although not as well) so I am trying to understand why switching to NN or similar method would be of any benefit.

shalako wrote:

So..how do NN or CFRM (forgive me if do not know the correct terms) handle these types of situations? Is it just by pure EV or ?

CFRM is a technique for calculating an approximate Nash Equilibrium strategy. A NE strategy will hold you to a tie, but not exploit you.
A neural network is just one of many machine learning techniques. http://en.wikipedia.org/wiki/Machine_learning http://poker.cs.ualberta.ca/publications/ICAI00.pdf is an accessible account of using a NN to model an opponent.

http://www.cs.auckland.ac.nz/~jrub001/f ... intAIJ.pdf is a nice overview paper that should answer many of your questions

shalako wrote:

Maybe I am already using a form of NN but do not even know it?

Unlikely

Statistics: Posted by spears — Thu May 23, 2013 10:32 pm

Re: Expert/Rule vs Simulation based systems

2013-05-23T22:27:31+00:00

Statistics: Posted by Coffee4tw — Thu May 23, 2013 10:27 pm

Re: Expert/Rule vs Simulation based systems

2013-05-21T23:52:06+00:00

Sounds like we're in similar boats. I created my first ever poker bot about 9 or 10 years ago. It was a FL 7-Card Stud player using a rules based system and a commercial odds calculator called PokerPal. It was quite profitable but sadly, 7CS lost popularity as hold'em became more and more commercialised.

As a none-academic, game theory seems to be such a steep learning curve and, apart from a total lack of spare time, I'm not really THAT interested in poker. I still play around with rules based systems but try to use lookup tables as much as possible to reduce the never ending IF...ELSE stuff. Over the last couple of years, I've messed about with HU NL and am currently putting some stuff together for Turbos and/or Super Turbos but it's very much a hobby rather than an obsession.

Personally, I think rules based systems are good for playing against humans PROVIDING that you are (or have) an expert or winning player to advise you. I believe most of the Game Theory Optimal guys are more interested in the AI and the bot vs bot competitions, where finding that elusive "perfect" Nash solution is of utmost importance. For guys like us, who are only interested in making a few quid, "unexploitable" is not really the "optimal" solution vs humans. Having said that, an optimal strategy IS a "one size fits all" and will profit if the opponents are not also playing optimally...and they rarely will be. These GTO strategies certainly have some merit (particularly if you don't have a poker expert available) because even a bad player can create a winning strategy.
In theory...

At the end of the day, if it ain't broke, don't fix it.

Statistics: Posted by birchy — Tue May 21, 2013 11:52 pm

Expert/Rule vs Simulation based systems

2013-05-21T19:22:00+00:00

Hello,

I have been writing rule based bots for nearly a decade now and have up until this point resisted any type of simulation based system to determine the correct play. I have been writing AI entirely based on expert systems. Lately I have concentrated on HU using techniques pioneered by Phil Galfond and other players using the same style such as Jason Senti. Since these guys really know how to play why should I detour to a simulation based system? My bot already adapts both preflop and postflop (although not as well) so I am trying to understand why switching to NN or similar method would be of any benefit.

Much of the profit from NL comes from marginal holdings, bluffing, and floating. Since good hands do not come up often in HU I have found it necessary to really pound the opponent as often as possible putting the villain to a tough decision on every hand. I have polarized its entire range from preflop to the river to make it completely unpredictable. One thing that has really helped in NL was switching to range equity. Preflop range equity is straightforward but post flop range equity was a bit more difficult to determine and I am consuming some rather large cpu usage in order to put the villian on a narrow range based on his actions and board texture.

So..how do NN or CFRM (forgive me if do not know the correct terms) handle these types of situations? Is it just by pure EV or ? Maybe I am already using a form of NN but do not even know it? My whole bot AI is basically a giant IF THEN or CASE statement.

Any insight would be appreciated.

Statistics: Posted by shalako — Tue May 21, 2013 7:22 pm