Poker-AI.org Poker AI and Botting Discussion Forum 2021-12-07T16:31:33+00:00 http://poker-ai.org/phpbb/feed.php?f=24&t=3144 2021-12-07T16:31:33+00:00 2021-12-07T16:31:33+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=8974#p8974 <![CDATA[Re: DeepStack implementation for texas holdem released]]> https://github.com/lucky72s/dyypholdem

Statistics: Posted by lucky72s — Tue Dec 07, 2021 4:31 pm


]]>
2021-03-13T21:00:47+00:00 2021-03-13T21:00:47+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=8635#p8635 <![CDATA[Re: DeepStack implementation for texas holdem released]]> Statistics: Posted by someone_else — Sat Mar 13, 2021 9:00 pm


]]>
2018-07-25T14:17:50+00:00 2018-07-25T14:17:50+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7682#p7682 <![CDATA[Re: DeepStack implementation for texas holdem released]]> Statistics: Posted by happypepper — Wed Jul 25, 2018 2:17 pm


]]>
2018-07-24T09:20:52+00:00 2018-07-24T09:20:52+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7678#p7678 <![CDATA[Re: DeepStack implementation for texas holdem released]]> happypepper wrote:

Hey there,

The flop and turn were bucketed using k-means clustering with earth mover's distance metric. I didn't include the bucketing code used to generate the bucketing data files since they were ugly and unpolished. I can clean them up and release them if enough people want it though.

The river was bucketed using pair of (win%, tie%), assuming uniform opponent range. (similar to EHS)
I didn't experiment with different bucketing strategies.

That is an interesting suggestion to use MSE instead of huber loss. It's probably a good idea since outliers in poker are actually quite important. Nutted hands can sometimes have cfv of 30x pot size for certain range pairs and it's these cases that contain the greatest loss when trained with huber loss. IMO it's definitely worth running the experiment but I don't know if I'll have the time to do it in the near future.


Consider just for the river 'hand strength histogram homogeneity.' In my experimenting with HUNL CFRM strategies, that worked best in comparison. I'm not sure I have the code for it anymore, but I believe it involved calculating a inverse center-weighted skewness for the HS histogram. So, for nine slot HS histogram you would normalize it, multiply it by [5,4,3,2,1,2,3,4,5], then calculate the skewness for the weighted histogram and divide the skewness into x buckets. Having a better 'understanding' of the histogram distribution (as apposed to EHS) seemed to allow better strategic performance.

For flop and turn just do what you're doing.

Statistics: Posted by cantina — Tue Jul 24, 2018 9:20 am


]]>
2018-06-07T22:57:51+00:00 2018-06-07T22:57:51+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7647#p7647 <![CDATA[Re: DeepStack implementation for texas holdem released]]>
I used many of google cloud platform's nVidia Tesla K80s in parallel. Since the code is preemptible instance friendly, it is pretty cheap to achieve the results that I did. For me it took $3000, but my code now contains many optimizations that it didn't when I was generating the data. I was able to take advantage of many free GCP credits online, so it cost me $0.

If you wanted to 10x amount of data generated (like the paper did) it would probably cost ~$30,000.

Statistics: Posted by happypepper — Thu Jun 07, 2018 10:57 pm


]]>
2018-06-07T20:30:49+00:00 2018-06-07T20:30:49+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7646#p7646 <![CDATA[Re: DeepStack implementation for texas holdem released]]> Statistics: Posted by spears — Thu Jun 07, 2018 8:30 pm


]]>
2018-06-06T23:56:35+00:00 2018-06-06T23:56:35+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7644#p7644 <![CDATA[Re: DeepStack implementation for texas holdem released]]>
The flop and turn were bucketed using k-means clustering with earth mover's distance metric. I didn't include the bucketing code used to generate the bucketing data files since they were ugly and unpolished. I can clean them up and release them if enough people want it though.

The river was bucketed using pair of (win%, tie%), assuming uniform opponent range. (similar to EHS)
I didn't experiment with different bucketing strategies.

That is an interesting suggestion to use MSE instead of huber loss. It's probably a good idea since outliers in poker are actually quite important. Nutted hands can sometimes have cfv of 30x pot size for certain range pairs and it's these cases that contain the greatest loss when trained with huber loss. IMO it's definitely worth running the experiment but I don't know if I'll have the time to do it in the near future.

Statistics: Posted by happypepper — Wed Jun 06, 2018 11:56 pm


]]>
2018-06-06T09:43:19+00:00 2018-06-06T09:43:19+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7643#p7643 <![CDATA[Re: DeepStack implementation for texas holdem released]]>
Firstly, what bucketing methods did you use? Did you compare different methods?

And secondly, did you check other loss functions for neural net training? I have also been experimenting with DeepStack for quite some time now and from my experience simple MSE works better (but I only tested it with small games like Rhode Island Holdem). I know that folks from UoA used Huber loss but there were no any explanations or comparisons in the paper.

Statistics: Posted by optimizer — Wed Jun 06, 2018 9:43 am


]]>
2018-06-06T04:46:35+00:00 2018-06-06T04:46:35+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7642#p7642 <![CDATA[Re: DeepStack implementation for texas holdem released]]>
There are a few ways it can be faster:

- Precalculate the flop call matrix (around 1.5 seconds savings)
- Support luaJIT or C++ implementation
- Use flop network in preflop for 20 iterations like they do in the paper, so the opponent CFVs can be looked up in the flop instead of recalculated. The downside is that the CFVs won't be very accurate
- Precalculate flop CFVs for all boards for common preflop situations
- Use less CFR iterations on the flop

Statistics: Posted by happypepper — Wed Jun 06, 2018 4:46 am


]]>
2018-06-06T00:03:13+00:00 2018-06-06T00:03:13+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7641#p7641 <![CDATA[Re: DeepStack implementation for texas holdem released]]> On GitHub there was a discussion about how abstraction influences the Huber loss of the networks, I have run a few tests myself and when you map the buckets back to cards and compute the loss then in an unabstracted way it definitely increased.
For actual online botting the thinking times might be a bit long, any ideas how to decrease them?

Statistics: Posted by HontoNiBaka — Wed Jun 06, 2018 12:03 am


]]>
2018-06-05T02:19:33+00:00 2018-06-05T02:19:33+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7640#p7640 <![CDATA[Re: DeepStack implementation for texas holdem released]]> Statistics: Posted by happypepper — Tue Jun 05, 2018 2:19 am


]]>
2018-06-04T23:16:18+00:00 2018-06-04T23:16:18+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7639#p7639 <![CDATA[Re: DeepStack implementation for texas holdem released]]> How did you play against Slumbot? I only see a website, do you just send http requests?

Statistics: Posted by HontoNiBaka — Mon Jun 04, 2018 11:16 pm


]]>
2018-06-03T23:58:51+00:00 2018-06-03T23:58:51+00:00 http://poker-ai.org/phpbb/viewtopic.php?t=3144&p=7638#p7638 <![CDATA[DeepStack implementation for texas holdem released]]>
I've released an implementation of DeepStack for NL texas hold'em:
https://github.com/happypepper/DeepHoldem

Statistics: Posted by happypepper — Sun Jun 03, 2018 11:58 pm


]]>