Poker-AI.org

Hi all,

I've released an implementation of DeepStack for NL texas hold'em:
https://github.com/happypepper/DeepHoldem

Cool, will have a look.
How did you play against Slumbot? I only see a website, do you just send http requests?

I wrote a selenium script to play on the site. If you look in the javascript, all the dealing code is in there. So I can extract the situation from there by modifying the JS slightly.

I think this is a really good contribution, I will try to fire up my Linux partition soon and reproduce your bot.
On GitHub there was a discussion about how abstraction influences the Huber loss of the networks, I have run a few tests myself and when you map the buckets back to cards and compute the loss then in an unabstracted way it definitely increased.
For actual online botting the thinking times might be a bit long, any ideas how to decrease them?

Yes, it is mainly the flop that is too slow.

There are a few ways it can be faster:

- Precalculate the flop call matrix (around 1.5 seconds savings)
- Support luaJIT or C++ implementation
- Use flop network in preflop for 20 iterations like they do in the paper, so the opponent CFVs can be looked up in the flop instead of recalculated. The downside is that the CFVs won't be very accurate
- Precalculate flop CFVs for all boards for common preflop situations
- Use less CFR iterations on the flop

Thanks for the contribution. I have a couple of questions on implementation details.

Firstly, what bucketing methods did you use? Did you compare different methods?

And secondly, did you check other loss functions for neural net training? I have also been experimenting with DeepStack for quite some time now and from my experience simple MSE works better (but I only tested it with small games like Rhode Island Holdem). I know that folks from UoA used Huber loss but there were no any explanations or comparisons in the paper.

Hey there,

The flop and turn were bucketed using k-means clustering with earth mover's distance metric. I didn't include the bucketing code used to generate the bucketing data files since they were ugly and unpolished. I can clean them up and release them if enough people want it though.

The river was bucketed using pair of (win%, tie%), assuming uniform opponent range. (similar to EHS)
I didn't experiment with different bucketing strategies.

That is an interesting suggestion to use MSE instead of huber loss. It's probably a good idea since outliers in poker are actually quite important. Nutted hands can sometimes have cfv of 30x pot size for certain range pairs and it's these cases that contain the greatest loss when trained with huber loss. IMO it's definitely worth running the experiment but I don't know if I'll have the time to do it in the near future.

Sadly I don't have time to have a really good look at this, but from what I see it seems like really good work. Thanks for the contribution. What computing resources did this require to create?

Thanks!

I used many of google cloud platform's nVidia Tesla K80s in parallel. Since the code is preemptible instance friendly, it is pretty cheap to achieve the results that I did. For me it took $3000, but my code now contains many optimizations that it didn't when I was generating the data. I was able to take advantage of many free GCP credits online, so it cost me $0.

If you wanted to 10x amount of data generated (like the paper did) it would probably cost ~$30,000.

Consider just for the river 'hand strength histogram homogeneity.' In my experimenting with HUNL CFRM strategies, that worked best in comparison. I'm not sure I have the code for it anymore, but I believe it involved calculating a inverse center-weighted skewness for the HS histogram. So, for nine slot HS histogram you would normalize it, multiply it by [5,4,3,2,1,2,3,4,5], then calculate the skewness for the weighted histogram and divide the skewness into x buckets. Having a better 'understanding' of the histogram distribution (as apposed to EHS) seemed to allow better strategic performance.

For flop and turn just do what you're doing.

Would the skewness just be 1 number then? Would it be used in combination with win%?

Glad I found this! Thanks for making it available, will take a deeper look as I move further into this subject

I liked this project but was struggling with the software stack used. That's why I converted it to a Python/PyTorch implementation:
https://github.com/lucky72s/dyypholdem