Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 5:35 pm

All times are UTC




Post new topic Reply to topic  [ 3 posts ] 
Author Message
PostPosted: Fri Oct 02, 2015 4:19 pm 
Offline
Veteran Member

Joined: Wed Mar 20, 2013 1:43 am
Posts: 267
There are a few decent AIs that were trained with neural network self play, as far as I know Snowie did it and also Neo used neural networks amongst other things.

I am not really sure, how that is done in principle though. So you have your NN and you have your features, you have to find a usedul representation of the poker game,
maybe the bucket your hand belongs to will be a feature, maybe also public buckets etc. but I don't understand how to progress from there.

I have used NNs several times in different domains, basically I had labeleld training examples and I tried to predict the class from the features, simple enough.

But what is the class label in self play? What do I try to fit or predict? I really have problems with the general idea of using NNs and self play.


Top
 Profile  
 
PostPosted: Fri Oct 02, 2015 5:21 pm 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
https://poker-ai.org/phpbb/viewtopic.ph ... 13&start=0

I've read a few accounts of using NNs to model opponent play. I think the idea is that you then use that model to determine a counter strategy. A NN would be a more compact representation of a strategy than a histogram and would therefore have some advantages. I've considered this but the disadvantages seem to outweigh the advantages.

There is also the remote possibility that NNs other than feed forward were used. http://karpathy.github.io/2015/05/21/rnn-effectiveness/


Top
 Profile  
 
PostPosted: Sat Oct 03, 2015 2:41 pm 
Offline
Junior Member

Joined: Sat Apr 26, 2014 7:29 am
Posts: 34
HontoNiBaka wrote:
There are a few decent AIs that were trained with neural network self play, as far as I know Snowie did it and also Neo used neural networks amongst other things.

I am not really sure, how that is done in principle though. So you have your NN and you have your features, you have to find a usedul representation of the poker game,
maybe the bucket your hand belongs to will be a feature, maybe also public buckets etc. but I don't understand how to progress from there.

I have used NNs several times in different domains, basically I had labeleld training examples and I tried to predict the class from the features, simple enough.

But what is the class label in self play? What do I try to fit or predict? I really have problems with the general idea of using NNs and self play.


I work on something similar.
My idea is to do self play and evaluate multiple possibilities (e.g. call, raise half pot, raise pot) and then for each action chose/learn the one that maximizes the return. [Edit2: But this does not work to learn good folds, it would mean folding any losing hand preflop]

The hard question for me is how to learn good folding.

There are many possibilities:
- if you lose the hand pick random action to be fold instead (maybe not for all lost hands, maybe chose this random too?)
- pick a action to be fold with heuristics (e.g. where Hand Strength/Potential is worst, based on pot odds,...)
- use another bot or hand histories and learn folding from that

[Edit: just to be clear: the NN should learn folding, the question is how I decide as the teacher what are good folds]

But I just started with that approach, maybe it will not work well...


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC


Who is online

Users browsing this forum: Google [Bot] and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group