Image Image Image




Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: Integrating Opponent Models with Monte-Carlo Tree Search in Poke
PostPosted: Mon Jan 02, 2012 12:50 am 
Offline
Junior member
User avatar

Posts: 17
Favourite Bot: my own
Integrating Opponent Models with Monte-Carlo Tree Search in Poker


abstract:
In this paper we apply a Monte-Carlo Tree Search implementation that is boosted with domain knowledge to the game of poker. More specifically, we integrate an opponent model in the Monte-Carlo Tree Search algorithm to produce a strong poker playing program. Opponent models allow the search algorithm to focus on relevant parts of the game-tree. We use an opponent modelling approach that starts from a (learned) prior, i.e., general expectations about opponent behavior, and then learns a relational regression tree-function that adapts these priors to specific opponents. Our modelling approach can generate detailed game features or relations on-the-fly. Additionally, using a prior we can already make reasonable predictions even when limited experience is available for a particular player. We show that Monte-Carlo Tree Search with integrated opponent models performs well against stateof-the-art poker programs.

authors: Marc Ponsen, Geert Gerritsen and Guillaume Chaslot
uni: Department of Knowledge Engineering, Maastricht University, Netherlands

link: http://www.personeel.unimaas.nl/m-ponsen/pubs/Ponsen-MCTSMODEL-IDTGT10.pdf


Top
 Profile E-mail  
 
 Post subject: Re: Integrating Opponent Models with Monte-Carlo Tree Search in Poke
PostPosted: Tue Jul 17, 2012 10:02 am 
Offline
New member
User avatar

Posts: 7
Favourite Bot: none yet
This is the very first paper, I actually take some time to read through because MCTS seems to be discussed quite often in here.

Is that just me or what? This MCTS method just assume the opponent will only always employ single strategy (single opponent model), which is so unrealistic. And, it also ignore the possibility of counter strategy that the opponent will employ to the model.


Top
 Profile E-mail  
 
 Post subject: Re: Integrating Opponent Models with Monte-Carlo Tree Search in Poke
PostPosted: Tue Jul 17, 2012 11:17 am 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
abctata001 wrote:
This MCTS method just assume the opponent will only always employ single strategy (single opponent model), which is so unrealistic. And, it also ignore the possibility of counter strategy that the opponent will employ to the model.
Yes. This is what was done in this paper, and most other published work that uses opponent modelling. MCTS just calculates a response to a given opponent strategy (which might be a mixed strategy). In principle, there is nothing stopping you updating the opponent strategy every hand or even every action. That update could reflect recent actions, or even anticipate future opponent strategy. I think the failure to anticipate changes in opponent strategy is one of the chief reasons this method is not more successful.


Top
 Profile E-mail  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: