This work will be continued but I thought I'd briefly share information.
In a few words: The lstm + genetic algorithm approach of ASHE was reused (with some tweaks). It was expanded for a 6max sit and go setting. The bot developed is trained and tested against various weak bots. The report then follows with a qualitative analysis of the strategy developed. One contribution of this work is the bot itself (6max sng agents being less explored); the other contribution is to obtain information from the bot's developed strategy on how to beat another (human readable) beginner strategy.
The main insights for those considering to follow a similar path are:
-the genetic algorithm / neuroevolution is effectively learning. The number of games played before estimating the fitness of an agent must be high enough, otherwise an all-in-or-fold strategy develops. Heuristically, the bare minimum is to play four times 6 games (one for each position) at a table to get a reasonable estimate of the agent's fitness. Games last about a 150 hands.
-as the opponents are almost beaten to the maximum, the neural network architecture (and the lstm approach) has not been seriously put to test (yet). No serious conclusion can be drawn. Also adaptation is a tricky thing to measure and it is not possible to determine whether long term memory is used. To further test the capacity of the model and get a stronger bot, it should be confronted to stronger opponents.
-some hardware is required but it is not excessive. An aws ec2 instance c5.18xlarge is used, and the code is written to exploit multi-threading. To simulate 100'000 games it took between one and two hours. To train against one table (=set of opponents), 420'000 games are simulated. In the last implementation there are four different tables, thus the training took about 24 hours. Though a fast hand evaluator was used (OMPEval), calculating the equity is still the most time consuming element of training. Similarly, the training of ASHE (as presented in the paper above) would take roughly 5 hours.
Feel free to shoot questions / critics. The code is now being structured so that it is easier to use
A useful element to pursue this work would be to have one (or even multiple) strong sng bot to train / validate against. Is there one accessible that is considered a reference in that game type?ps: if an admin feels like this post should be located elsewhere, please tell me so
Statistics: Posted by csrev — Tue Oct 01, 2019 2:37 pm
]]>
... I assume it would get better at higher limits with rake decreasing (not sure by how much this effect would be negated by decreasing winrate though). Haven't tested anything higher than 50NL.
. Also about exploitability: the player has a pure strategy (deterministic w.r.t. its inputs), but since it is keeping in memory the past states, and using them as inputs, it will never be in the same state twice and it would be very difficult to predict its actions.