Poker-AI.org

I was wondering if somebody could recommend an easy to implement regression modeling package/API for .NET (2005)? I've currently tried Encog NNs, none of which seem to be able to accurately represent an entire HUNL games <= 40BBs. Whatever the package, it probably needs to be an iterative training algorithm.

Read: I have a terabyte of NE strategies for various stack sizes that I'm trying to encode into a few models.

Are you regressing on action probability?
Can you get acceptable solutions for selected slices of the entire problem eg a single stack size and player position?
Why does Encog fail? Is the problem just too big so it can't do enough iterations in a reasonable timeframe? I'm guessing given the size of this problem you are using some disk based technique.

Yes, action probabilities.

I tried breaking up the games into stacks <> 20bb, but it didn't seem to make a difference. The whole idea of regressing multiple stack sizes is to allow the NN to self-interpolate between them. Same goes for the various game states.

I think the problem is just too big. I wouldn't say Encog completely fails, but the best I can do loses to the underlying EQ by descent margin and performs poorly in practice (sometimes taking senseless actions). It might be that there's just too much detail being generalized away -- the combined strategies are over 250gb worth of data. For a game like HUL I think NNs can handle the entire strategy space, but with HUNL including various stack sizes, there may be too much detail, at least for single layer feed forward networks. Of course, it could very well just be my labeling (inputs), or my EQs just suck.

My thought was that maybe you could first build lots of models for slices. Then build models out of those models. But it sounds like the problem is that you can't build models of the history.

I think I see what you're saying. Building a model for each history in each game then building some sort of 'governor' to select amongst them would be interesting. I can easily build models of each history, but there are millions. The hard part would be the selection method.

No. Build a model of each slice. The model requires much less memory than the slice. So then you can build a model using the models you have already built.

What do you mean of each slice? Each stack size? How would I build a model of a model? Wouldn't there be some signal degradation in that, like making a copy of a copy?

I'm not sure if I completely understand what you're wanting to achieve, but I think you're trying to go from a huge model that maps game states to action probabilities and then model this information in a NN. Therefore decreasing the size ~250Gb to something far, far smaller, am I close?



Have you tried any other techniques for breaking up the games, even more? It sounds like there is a lot of information you're trying to encode into the NN. I feel like the only way you could get a decent model is by splitting the network into, quite a few, smaller models, e.g. have a network that looks at different stacks, split for each stage of the game, split into how many players are active, split into... etc...

You've may have already thought of/tried doing so, or maybe i'm completely off track?

Even for going the opposite way around, i.e. building NNs based on models, I feel like this could be the only way of really getting accurate results.

You're correct in your assumption of what I'm doing. It's for HUNL SNGs.

I want to believe that I could do this in just a few models, like maybe one for each round. Right now I'm just using preflop/postflop NNs. Maybe I need to switch to deep belief networks, or something like random forests. The problem is finding a package that offers iterative training and interfaces with .NET.

The guy that did poker snowie, what kind of NNs was he using? I found a website awhile ago by this guy from MIT (I think) that created a new kind of random forest modeling method and was selling it on his own website. Supposedly it was extremely accurate and robust. I can't find it now.

There's Multivariate adaptive regression splines (MARS):
http://cran.r-project.org/web/packages/earth/index.html

That's in C++, though, and I'm not sure if it's iterative.

- A slice of the data is a part where one or more values are constant or within a small range. So a single stack size is a slice. There are lots of ways of forming slices
- Say you had models for each stack size. Use those to generate data for the model "above" at the moment the NN needs the data
- Yes there might be loss of accuracy.

Do your inputs include your own betting history? That affects the current action probability.

The rmse for this problem should be tiny. If it isn't tiny there's a problem. If the NN hasn't converged that would cause cause a high rmse. A bad choice of inputs should give high rmse also I think.

It's not iterative. Why does it have to be iterative?

Given the size and difficulty of the problem you are facing, I'd find whatever works and sort out the language interfacing problems later.

I guess it doesn't have to be if I were to do it in slices.

I'm not even sure how to use MARS with my data, or if it will work at all. Anybody want to write a .NET wrapper for it?