Poker-AI.org
http://poker-ai.org/phpbb/

Realistic and working design for full ring bot?
http://poker-ai.org/phpbb/viewtopic.php?f=24&t=2577		 Page 1 of 1
Author:  Seikez [ Fri Sep 06, 2013 5:24 pm ]
Post subject:  Realistic and working design for full ring bot?
Hi guys!

I need to decide on the design of my bot. It should be able to play Holdem with 10 players per table.

Solver
How do I determine what action the bot should take in a given situation?
I know that you can use NN and other types of AI and fancy algorithms. But I have the impression that these types of solvers can only handle 2 players, and even with this limited problem uses like 12GB RAM or so just to hold the nets, rules and all the what-not they need. Reading between the lines in some papers, it seems that anything bigger than 2 players is impossible with these techniques. Am I right?

Please point me in the right direction. A reference with code would be great! Something to get me started, so I can get back to you with more stupid questions. :D

Opponent modeller
I also need to figure out how to do the opponent modeller. Should it be rules based (if opponent has VPIP > x and 3-bet on flop, then...), or is there some other (smarter, more efficient, more flexible, less exploitable) way of doing it?

I need the same type of kick start here as I need for the solver. Something to get me started. Preferably with code I can adapt for my needs.
Author:  MrNice [ Fri Sep 06, 2013 6:03 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
Hi Seikez,

If you don't want to use fancy algorithm your bot will be based on rules and you will have to fine tune each rules all the time...

Maybe you could try a bot with Monte Carlo tree search and expectimax...

Have a look on : http://poker-ai.org/archive/www.pokerai ... =64&t=2548

For opponent modeling he is using NN with clustering...

At the beggining I tought as well to create a bot rules based but I finally give to to try with CFRM... (take as much time as rules fine tuning :D)

Hope it helps;

Regards,
Author:  shalako [ Fri Sep 06, 2013 6:12 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
I would go with rules for each situation for now and adapt in other methods (like your NN stuff) as needed. Preflop your going to have to have rules for each seat (first in, raised pot, limped pot, open shove etc). For allins you could use pokerstove to figure out the EV for various situations (like raise from early, raise from button, raise from early with a caller, 3B pot etc) for all hands down to around 35% equity against AA. Your looking at maybe 6 different situations you will have to have equities for. You can just hard code them into an array for a simple LUT.

Post flop it gets more complicated. For a 10 player game its your going to have to have two rule strategies..one for multiway and one for HU. Then its going to get broken down into sub categories..like preflop raiser, 3B pot..bet into, nobody in etc. Then breakdown what you do based on your hand strength..like Top Pair, Nuts, Marginal Hands (like middle pair, top pair weak kicker etc), draws, air etc.

It will take time but for now to get started just do the easy hands/situations to get started (like just top 5% of hands pre and top pair+/open draws post). Once you do that then add in all the other stuff for a "baseline strategy". After that is complete then add in opp models to vary that strategy based on the opp stats, reads etc. For now you concentrate on getting a good baseline strategy coded. This could take you months or even years to get right.

Good luck.
Author:  PolarBear [ Fri Sep 06, 2013 9:21 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
MrNice wrote:
Maybe you could try a bot with Monte Carlo tree search and expectimax...



MCTS and everything based on opponent modelling is not a viable idea IMO - the models won't be able to properly capture less-frequent situations and they'll affect the algorithm's outcome. Tweaking and tuning the models can easily take more time than writing and tuning a set of rules (for rule-based bot) - and with rules, you at least have full control over your bot's play.
Author:  Seikez [ Sun Sep 08, 2013 9:31 am ]
Post subject:  Re: Realistic and working design for full ring bot?
shalako wrote:
I would go with rules for each situation for now and adapt in other methods (like your NN stuff) as needed.

This sounds like good advice! Thanks shalako! I will keep it simpel in the beginning, then begin experimenting with more advanced stuff when the bot actually plays poker.

One big drawback with rules though is that they will probably only be as good as the person defining them, which is me. Therefore, I am afraid my bot will play a sub standard game in the beginning. But at least with rules I can understand what my bot is doing, and I can always refine the rules should my own play improve.

MrNice wrote:
At the beginning I thought as well to create a bot rules based but I finally give to to try with CFRM... (take as much time as rules fine tuning )

I am currently reading the paper you linked to. Seems it took him close to a year to implement all that. And he probably had some support from his supervisor and other knowledgeable people at the university. Without the same type of support, I wonder if I could do it at all. I guess it will take me many months (hopefully not years) to develop a bot that works along the same principles as the one described in the paper.

(From the comments on his article, I understand he is employed somewhere and has to work hard for his living. If I had a profitable bot, I would not be working hard for someone else. I would be sitting in a beach chair somewhere, sipping on a drink with a funny little umbrella in it. So I wonder how strong a poker player his bot actually was? ;) )

Before I embark on such a big project, I would need to know for sure it is possible to get it working with reasonable resources and effort.

Do you have a working bot based on CFRM that can handle 10 players?

Does anyone else have a working +EV bot that handles 10 players which is based on ANN, CFRM, Descision Trees or other 'algorithmic' methods?
What is your experience? Was it worth the effort? Has it given you a better bot with higher EV than you would have with a rules based?

Can you point me to some code so I have something to start with?
Author:  spears [ Sun Sep 08, 2013 11:10 am ]
Post subject:  Re: Realistic and working design for full ring bot?
https://code.google.com/p/cspoker/sourc ... TSBot.java

You would need to read around this a bit. It definitely worked at one stage, and was for more than 2 players. AFAIK it was never ev+ at anything except play games or very low stakes. But it is a start and the code appears professional.
Author:  Seikez [ Sun Sep 08, 2013 7:26 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
spears wrote:
https://code.google.com/p/cspoker/source/browse/trunk/ai/bots/src/main/java/org/cspoker/ai/bots/bot/gametree/mcts/MCTSBot.java

You would need to read around this a bit. It definitely worked at one stage, and was for more than 2 players. AFAIK it was never ev+ at anything except play games or very low stakes. But it is a start and the code appears professional.

Thanks, I'll have a look at it.

spears, you have been doing this for many years now and obviously have a lot of experience. What is your advice, rules based or AI?

If I decide my first version should be rules based, do you know any code I can build on?
Author:  shadehs [ Sun Sep 08, 2013 8:17 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
For rule based there are a lot of examples and demo bots on OpenHoldem on how to get started.
Author:  spears [ Sun Sep 08, 2013 8:24 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
I don't like the rules based approach because:
- I'm interested in AI and a rules based approach isn't AI
- I can't play poker and don't know anyone who can so I don't know the rules
- In all other games I can think of, chess, scrabble, backgammon eventually a rules based approach has been beaten
- A numerical approach has the potential to be better than a human
- The best bots I know of were not rules based: Sonia, Polaris, and a few from Poker-AI

Having said all that, writing a rules based approach might be quite easy if you can play poker and just want to make some money quickly.

If I were writing a rules based approach I'd take a close look at Sartre and formal approaches to building rules based systems.
Author:  shalako [ Sun Sep 08, 2013 8:28 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
Quote:
One big drawback with rules though is that they will probably only be as good as the person defining them, which is me. Therefore, I am afraid my bot will play a sub standard game in the beginning. But at least with rules I can understand what my bot is doing, and I can always refine the rules should my own play improve.


Yes that is true. The only way to do it with rules is with the help of a pro or by learning poker yourself. I use a combination of both which is why I have stuck with rules. They are easier to debug and you always have total control as to what it is doing. Rules by their nature make the bot predictable so its very important to completely polarize its betting ranges or else things will become transparent very quickly. If you do this then rules can handle any situation effectively.

I have also found that its easier to make them adapt to you vs the bot adapting to them just because its hard for the bot to pick up/see tendencies that can be exploited. What I mean by this is that aggression is key. Run them over for as long as you can until they adapt..ie 3 and 4 bet them pre etc until they are forced to make a change or give up. Putting the villain to a tough decision on every opportunity until they fight back is my philosophy. Bots dont tilt but humans do...
Author:  Seikez [ Sun Sep 08, 2013 9:19 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
spears wrote:
- I can't play poker and don't know anyone who can so I don't know the rules

Surely, sir, you jest! :lol:

Quote:
- In all other games I can think of, chess, scrabble, backgammon eventually a rules based approach has been beaten
- A numerical approach has the potential to be better than a human

I am aware of that, that is why AI is one my list. The question is if it can be done for a full ring table.

Quote:
- The best bots I know of were not rules based: Sonia, Polaris, and a few from Poker-AI

Could any of these bots handle 10 players, or where they all HU bots?

Quote:
If I were writing a rules based approach I'd take a close look at Sartre...

Sartre? Jean-Paul? If I recall correctly, he said 'Think Exist'

In a way it is true I am trying to make my computer 'think' poker. But I didn't realize this botting thing was about existentialism. :o

What is Sartre?


shadehs wrote:
For rule based there are a lot of examples and demo bots on OpenHoldem on how to get started.

Not a bad idea at all. I will investigate this further. Thanks.
Author:  spears [ Sun Sep 08, 2013 10:01 pm ]
Post subject:  Re: Realistic and working design for full ring bot?
Quote:
Could any of these bots handle 10 players, or where they all HU bots?

Only Polaris HU. Sonia was up to 6 player

Quote:
What is Sartre?

http://bit.ly/15Kd5Uj
Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/