Poker-AI.org

Hey all,

I'd just like to say a quick thank you to everyone who has contributed to the current or old PokerAi forums. I've recently handed in my fourth year project on improving opponent models though clustering in simulation based poker agents - this forum has provided me a huge amount of information and help!

If people are interested I should be able to add the paper at a later point, although there's nothing that innovative within it.. main conclusion was that agents using opponent models based on clustered data (from HandHQ) performed better than those that used a generic model, and even those that trained a unique opponent model for each player every 500 hands. This is most likely due to the fact that even after 20 thousand hands, the unique models don't cover enough in game situations to generate effective opponent models. The agents were tested over 20 thousand hands in 6-max nlhe using PA.

A lot of things I would've liked to add to the agent, but can only do so much in a year!

Thanks once more

Congrats on the finished project & paper.
Please do add the paper as soon as it's published. I think it's great to see and read about the work people have done that have actively contributed to this forum. It validates our efforts and shows why we run this forum and why we try to encourage open discussions and sharing of ideas.

Hope you are not dropping the project now and we'll see some more in the future.

_________________
Cheers.

Congrats! Curious how they performed against the PA agents?/

Yes, that would be very nice.

Thanks for the responses, I'll hopefully be able to upload the paper within the next month

The agent doesn't perform great in PA, I feel as though quite an important factor in it's current performance is it's preflop strategy which is quite weak (focused mostly on postflop and had to go for a quick preflop strategy in the end).

I made a 6-max table with 5 opponents with a range playing styles; lag, tag, lap, tap and 'strong'. The opponents were based on Xenbot in PA.

I've uploaded a table with the results of around 20k hands per agent.
- GEN uses a generic opponent model for each opponent
- 4k assigns each player to one of four opponent models based on clustered data
- 9k assigns each player to one of nine opponent models based on clustered data
- 4kU and 9kU are identical to above, however every 500 hands they train a unique opponent model for each player and use it instead of the clustered model.

As you can see, the agent loses in all cases.. however, I highlight the reasons for this in my discussion, and have a few simple ideas that should drastically improve it's performance.

Am I reading this correctly, the loose and passive (LAP) player won the most in all the scenarios?
That's ... very surprising to be honest. Must be a very odd combination of players that leads to that.

_________________
Cheers.

Correct in a sense yes, the LAP won the most against my agent. However, the winnings summed for all games were:

• TAG + 3.49sb/h
• LAG + 0.50sb/h
• TAP + 3.19sb/h
• LAP - 0.81sb/h
• STR + 4.16sb/h

In the end, LAP performs the worst overall, with the strong player (tweaked tag) playing the best. The results against my agent could be explained by the opponent modelling performing the worst against loose and passive players - it was harder to predict actions and harder to estimate a good range against players with a wide range of starting hole cards and fewer aggressive actions. I think this lead to several big losses for my agent. There were several situations in which the LAP had essentially slow played their way into winning a large pot, at the same time as my agent being over aggressive towards them as they often folded.

...finally got around to posting it - so it's in the poker papers section now.

Feel free to scrutinise thoroughly and i'll try my best to answer any questions. There are many areas that it could be improved, i've hopefully covered at least some of them in my discussion.

Additionally, i'm leaning towards putting my code into a public repository. It's quite messy, and at the moment requires 3rd party software (PT4), however that's an easy fix that could be improved later on so that it's completely stand-alone. Now that i've been working for a few months, I also realise how rough my code is.. so maybe it's not a good thing to release it into the public

It's been a busy summer, but still hoping to get back into this project at some point!