Poker-AI.org

I wondered which environment/tools you guys use for developping your bot.

Do you use a testbed with other "demobots" (selfmade or not) or do you play online against other humans/bots?
If you use a testbed, did you develop it yourself or did you get it form somewhere/someone else?
Are there any handy development tools I should know about?

I develop my own environment/tools. As testbed, I develop a Meerkat interface that allows me to play against the bots included in Poker Academy or any other bots compatible with the Meerkat API.

I recommend you the Meerkat API as handy development tools and this http://code.google.com/p/opentestbed/ as testbed.

I wrote a simple evaluator which uses two bots and lets the play against each other for xM games (HU only) and prints the resulting WR after the runs. Its pretty simple, but good to test the effect e.g. for different sizes of abstractions or different abstraction strategies. I might take a look at the Meerkat API though, as it would be helpful to play against the bot on my own to get an idea how it plays against different kind of explotable players and afaik the testbed allows a human player too.

The Meerkat API is great but essentially just a set of interfaces that you still need to implement yourself. It works very well as a standard of communication for bots out there though. Except that e.g. ACPC uses a different communication protocol. I think there was a ACPC-Meerkat API bridge at some point and I extended it to work for me but that was a while ago (3 years) and is likely very outdated/not pretty.

Making opentestbed compatible with the ACPC protocol would be a great new feature though.

Something we can work on once we (try to) get our own competitions up and running.

_________________
Cheers.

I used PokerAcademy and Meerkat to test out the sanity of my poker engine and my own interface in the middle. Later I replaced all that with my own tests. Right now pokereval is the only 3rd party poker/botting related library that I use. Meerkat was also a great reference of how a game state interpreter for a (rule based) bot could look like.

I test mine solely in play money against real villains. I find this to be the best way to see how it reacts to all different kinds of player types and situations. The slow part of it is that I have to watch it play and take notes. I then fix the issues and start over. It is very time consuming but it has worked for me very well.

I'm using a test framework where the bot will play hands from hand histories and compute the difference in wins/losses compared to how they were actually played by me - I have several million hands of my own play. I'm working on getting it to highlight specific nodes in the decision tree with the biggest differences. Eventually, my bot will beat me, I guess.

@Heuristics: How do you cope with the many hands where your bot does something slightly different (e.g. you raise to 4bb, he suggested a 5bb raise; or you would raise and he just calls)?

Whenever there is a difference in the amounts, I just assume that the action remains the same - of course, this will introduce inaccuracies, especially for stuff like overshove versus small value bet on the river. I think it still is a pretty good way to evaluate the bot's performance.

Whenever there is a difference in the actual action that unfolded in the hand, there are two cases. If the bot folds where I would have continued, I can simply take into account the difference in the end result of the hand (the bot won / lost less). If the bot wants to keep playing on a street where I folded, I can obviously not take into account that hand, which will kind of skew the big picture a little bit. The latter case doesn't happen much in general, though.