Poker-AI.org
http://poker-ai.org/phpbb/

Thank you!
http://poker-ai.org/phpbb/viewtopic.php?f=22&t=2732
Page 1 of 1

Author:  ibot [ Fri Apr 04, 2014 4:45 pm ]
Post subject:  Thank you!

Hey all,

I'd just like to say a quick thank you to everyone who has contributed to the current or old PokerAi forums. I've recently handed in my fourth year project on improving opponent models though clustering in simulation based poker agents - this forum has provided me a huge amount of information and help!

If people are interested I should be able to add the paper at a later point, although there's nothing that innovative within it.. main conclusion was that agents using opponent models based on clustered data (from HandHQ) performed better than those that used a generic model, and even those that trained a unique opponent model for each player every 500 hands. This is most likely due to the fact that even after 20 thousand hands, the unique models don't cover enough in game situations to generate effective opponent models. The agents were tested over 20 thousand hands in 6-max nlhe using PA.

A lot of things I would've liked to add to the agent, but can only do so much in a year!

Thanks once more :)

Author:  Coffee4tw [ Fri Apr 04, 2014 6:08 pm ]
Post subject:  Re: Thank you!

Congrats on the finished project & paper.
Please do add the paper as soon as it's published. I think it's great to see and read about the work people have done that have actively contributed to this forum. It validates our efforts and shows why we run this forum and why we try to encourage open discussions and sharing of ideas.

Hope you are not dropping the project now and we'll see some more in the future.

Author:  spears [ Fri Apr 04, 2014 9:31 pm ]
Post subject:  Re: Thank you!

Image

Author:  cantina [ Sat Apr 05, 2014 7:55 am ]
Post subject:  Re: Thank you!

Congrats! Curious how they performed against the PA agents?/

Author:  Tom [ Mon Apr 07, 2014 3:59 pm ]
Post subject:  Re: Thank you!

Quote:
opponent models though clustering in simulation based poker agents.

If people are interested I should be able to add the paper at a later point

Yes, that would be very nice.

Author:  ibot [ Mon Apr 21, 2014 8:33 pm ]
Post subject:  Re: Thank you!

Thanks for the responses, I'll hopefully be able to upload the paper within the next month :)

The agent doesn't perform great in PA, I feel as though quite an important factor in it's current performance is it's preflop strategy which is quite weak (focused mostly on postflop and had to go for a quick preflop strategy in the end).

I made a 6-max table with 5 opponents with a range playing styles; lag, tag, lap, tap and 'strong'. The opponents were based on Xenbot in PA.

I've uploaded a table with the results of around 20k hands per agent.
- GEN uses a generic opponent model for each opponent
- 4k assigns each player to one of four opponent models based on clustered data
- 9k assigns each player to one of nine opponent models based on clustered data
- 4kU and 9kU are identical to above, however every 500 hands they train a unique opponent model for each player and use it instead of the clustered model.

As you can see, the agent loses in all cases.. however, I highlight the reasons for this in my discussion, and have a few simple ideas that should drastically improve it's performance.

Attachments:
agent_comp.png
agent_comp.png [ 38.44 KiB | Viewed 15351 times ]

Author:  Coffee4tw [ Tue Apr 22, 2014 1:21 am ]
Post subject:  Re: Thank you!

Am I reading this correctly, the loose and passive (LAP) player won the most in all the scenarios?
That's ... very surprising to be honest. Must be a very odd combination of players that leads to that.

Author:  ibot [ Tue Apr 22, 2014 10:39 am ]
Post subject:  Re: Thank you!

Correct in a sense yes, the LAP won the most against my agent. However, the winnings summed for all games were:
  • TAG + 3.49sb/h
  • LAG + 0.50sb/h
  • TAP + 3.19sb/h
  • LAP - 0.81sb/h
  • STR + 4.16sb/h

In the end, LAP performs the worst overall, with the strong player (tweaked tag) playing the best. The results against my agent could be explained by the opponent modelling performing the worst against loose and passive players - it was harder to predict actions and harder to estimate a good range against players with a wide range of starting hole cards and fewer aggressive actions. I think this lead to several big losses for my agent. There were several situations in which the LAP had essentially slow played their way into winning a large pot, at the same time as my agent being over aggressive towards them as they often folded.

Author:  ibot [ Tue Sep 09, 2014 8:42 pm ]
Post subject:  Re: Thank you!

...finally got around to posting it - so it's in the poker papers section now.

Feel free to scrutinise thoroughly and i'll try my best to answer any questions. There are many areas that it could be improved, i've hopefully covered at least some of them in my discussion.

Additionally, i'm leaning towards putting my code into a public repository. It's quite messy, and at the moment requires 3rd party software (PT4), however that's an easy fix that could be improved later on so that it's completely stand-alone. Now that i've been working for a few months, I also realise how rough my code is.. so maybe it's not a good thing to release it into the public :shock: :P

It's been a busy summer, but still hoping to get back into this project at some point!

Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/