Image Image Image




Post new topic Reply to topic  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
Author Message
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 9:21 am 
Offline
Senior member
User avatar

Posts: 172
Favourite Bot: don't know
Coffee4tw wrote:
We do have a an option that exports hand histories in FTP format. So you can also just import them in PT3 or HoldemManager etc.

So could you at least release this to the public as well, so one didn't have to code it twice ?
In its current state I find the testbed too limiting and discouraging to start developing a (meerkat) bot with it.
Yeah its fast, but when I can't easily check the results of a tourney its worthless.

So the only other option is to start (with meerkat) is to get Poker Academy again and be annoyed of the slowness of it too.

I don't know if you get submissions or you would still embrace them ... but if you do, lowering the barriers of creating and testing bots would do really good.


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 9:27 am 
Offline
Senior member
User avatar

Posts: 172
Favourite Bot: don't know
absolut666 wrote:
would you be interested to be in a "focus group"? i mean besides GUI, i'm building another data analysis module(something like PT/HH, but much more configurable/user friendly)

I'm good at complaining about what I don't like *g*. With trackers I always thought that I don't like the big amount of data they just present and no clues how to reasonable start with them.
Poker Academy Prospector I always loved most because of its well choosen diagrams. Holdem Manager at least has some good tutorials included. PokerTracker I never like as a beginner :)
So feel free to ask me for my opinions :) ... maybe you should start a new thread for helper-projects around bot development, so we don't spoil this one.
I'll be watching !


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 11:39 am 
Offline
PokerAI fellow
User avatar

Posts: 1673
Favourite Bot: none
bluegaspode wrote:
Coffee4tw wrote:
We do have a an option that exports hand histories in FTP format. So you can also just import them in PT3 or HoldemManager etc.

So could you at least release this to the public as well, so one didn't have to code it twice ?
In its current state I find the testbed too limiting and discouraging to start developing a (meerkat) bot with it.
Yeah its fast, but when I can't easily check the results of a tourney its worthless.

So the only other option is to start (with meerkat) is to get Poker Academy again and be annoyed of the slowness of it too.

I don't know if you get submissions or you would still embrace them ... but if you do, lowering the barriers of creating and testing bots would do really good.


Yes we will release a new version when the time is right. Including a cash game option aswell.
I don't get why you say you can't view the results. You can set log levels to see when a player busts so you can have that detail of info if you want to have the finifh places for each tourney or at the very end you definitely get a summary of who won how much over the course of all tourneys. Thats what the testbed is for anyway. Running a lot of tourneys to see if a bot is profitable or not. If you want to go into more detail you can higher the log level and analyze the hands seperatly, not in a nice way at the moment, I agree but with an HH exporter option that completes basically all you need to analyze your bots play IMO.

_________________
Cheers.


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 12:12 pm 
Offline
PokerAI fellow
User avatar

Posts: 1562
Location: Finland
Favourite Bot: Self-made
I'd say that Poker Academy is good for checking the functionality of your bot. You can make fabricated setups (cards, button seat etc) and run individual hands. That's what I'd use it for. Then when your pot satisfies the obvious functional correctness requirements, you want to see if there are long term issues, like e.g. how is your result on UTG with 10 players or how is your net for different levelts. Then you don't want to watch to bot play individual hands, but rather have it run 10k hands and analyze that stuff in PT/HEM/PAProspector with "statistical" viewpoint. That's what I'd use the Testbed for.

Use right tools for different situations, you wouldn't use a load testing tool to run unit tests either, now would you...

_________________
Opinions expressed are my own, your mileage may vary... ;)
Warning: If I spot an opportunity to give sarcastic replies, I will take it. Nothing personal. I don't even know you.


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 12:55 pm 
Offline
Senior member
User avatar

Posts: 172
Favourite Bot: don't know
TooMuchCoffee wrote:
I'd say that Poker Academy is good for checking the functionality of your bot. You can make fabricated setups (cards, button seat etc) and run individual hands. That's what I'd use it for.

But I cannot debug in PA, if there is anything I don't like.
The testbed is really cool when it comes do debugging ! That's why I think the proposed GUI would get the tool to the next level. Away from 'just' running multiple and lots of tourneys to a good bot development and simulation tool, equally suited for individual development and big simulations.
And that's why I don't understand why the idea of a better GUI is not too much welcomed/appreciated (or lets say 'rejected' ?)

TooMuchCoffee wrote:
Then when your pot satisfies the obvious functional correctness requirements, you want to see if there are long term issues, like e.g. how is your result on UTG with 10 players or how is your net for different levelts. Then you don't want to watch to bot play individual hands, but rather have it run 10k hands and analyze that stuff in PT/HEM/PAProspector with "statistical" viewpoint.

But I cannot analyze the result in PT/HEM/PAProspector right now (at least not with the public one thats available).
Yes - I get a lot of log-output - for each hand and for each tourney a summary. Are you really expecting someone to read this after >100 round have been played ?

So essentialy for the usecase you wrote the tool for
1a) running big simulations and analyzing the results
1b) thus preparing a bot to compete against yours
in its free form (in my opinion) the testbed is not usable/too much restricted.

Anyway - I don't wont to complain too much about it - just want to (hopefully) open the eyes of the people that work all the day with their better tool and can't see the problems with the free one anymore. Thanks a lot for providing this tool, even if I have to write my HH-exporter now alone.

One other question: does your full featured tool already has a DIVAT-analysis or would that be a valuable contribution ?


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 1:23 pm 
Offline
PokerAI fellow
User avatar

Posts: 1562
Location: Finland
Favourite Bot: Self-made
bluegaspode wrote:
But I cannot debug in PA, if there is anything I don't like.
Well, depends on how you exactly define debug and what level of debugging is satisfying for you. From a tool like PA, I'm not expecting source level breakpoints and live variable observing etc. I'm quite happy with being able to write tracing stuff into a log. You might not be.
bluegaspode wrote:
But I cannot analyze the result in PT/HEM/PAProspector right now (at least not with the public one thats available).
Well, this might change soonish, we'll see... ;)
bluegaspode wrote:
Yes - I get a lot of log-output - for each hand and for each tourney a summary. Are you really expecting someone to read this after >100 round have been played ?
Definitely not.
bluegaspode wrote:
One other question: does your full featured tool already has a DIVAT-analysis or would that be a valuable contribution ?
It doesn't and I think it well could be.

_________________
Opinions expressed are my own, your mileage may vary... ;)
Warning: If I spot an opportunity to give sarcastic replies, I will take it. Nothing personal. I don't even know you.


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 4:04 pm 
Offline
PokerAI fellow
User avatar

Posts: 1673
Favourite Bot: none
I didnt reject a GUI. I just said it's not enough to make it into our subforum. If anyone wants to develop anything on top of the public version they are more than welcome to. And like I said its going to be compatible with future versions aswell so why not just go ahaed and code such a GUI.

How ever a DIVAT analysis might be a different thing tha might get you entrance into our group.

But these things aside I think you got something majorly wrong. We didn't release or code the testbed to have a great tool for everyone to develop bots in. We wanted to run a lot of simulations (that PA can't do) in a short amount of time with bots that are compatible to the meerkat API, as bots have to be if you want to run them in PA aswell. Releasing ANY version of this was more a good will act so that anyone can benefit from that. Plus it was intended to attract other botters that might have valuable information or code for our group. So far we had limited success with that meaning noone applied to the group with a reasonable strategy to gain access. Now with our goal sorta failed, what would be our motivation to drive this public version any further? To be nice? To make botting and development of bots mainstream? I don't think so.
Feel free to point our some motivation for us.

_________________
Cheers.


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 4:35 pm 
Offline
PokerAI fellow
User avatar

Posts: 1562
Location: Finland
Favourite Bot: Self-made
In other words, it's worth every euro you pay for it... ;)

_________________
Opinions expressed are my own, your mileage may vary... ;)
Warning: If I spot an opportunity to give sarcastic replies, I will take it. Nothing personal. I don't even know you.


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Fri Mar 12, 2010 5:34 pm 
Offline
Senior member
User avatar

Posts: 172
Favourite Bot: don't know
Coffee4tw wrote:
So far we had limited success with that meaning noone applied to the group with a reasonable strategy to gain access.
Now with our goal sorta failed, what would be our motivation to drive this public version any further? To be nice? To make botting and development of bots mainstream? I don't think so. Feel free to point our some motivation for us.


It's difficult to answer that, but its a chicken/egg problem I think.
I think with open-sourcing stuff you always need to believe in goodwill of people (while ignoring those that just take advantage of yours).
Why do people write open source software?

So first lets write a 'good' story:
Given there is a nice testbed, people will see the value in it and start programming against the meercat-api. More bots compatible means more competition and even more applicants to you. Maybe meerkat API gets brought to live again (I consider it quite dead currently), and we will see nice public bot competitions like done by UoA but hosted on pokerai.org.
Also for people writing publications a good testbed will have major positive impact. They safe a lot of time and give us better papers to read. You'll be proud to have your name in those papers on the 'thank you'-page.

Now an average story:
Some people will just find interest in a public testbet and though maybe not giving you good input on bots make contributions to an ever greater testbed.
It won't help you as you are already satisfied with what you have - still you get cool piece of software that evolves and gets inspiration for additions.
And hey - everyone writing a bot for the testbed understands java and if he doesn't like pieces of it he will change it (and most of the time even contribute it, because there is really not so much about hiding some testbed features). Much better situation to start than with other open source software with lots of users, but not too many devs beeing able to push it forward. Every eser could be your next contributor.

Now the 'bad' story:
Everone will use the testbed, but uses the bot only for his own fortune, makes good money but you had the work.

And finally my story:
I started my bot in java. Stopped, when I realized that screenscaping was to slow (on first sight). Switched to C#. Now realize that I don't like it I'm not feeling home. Thinking of switching back to java. Never thought of doing big simulations yet I always just wanted to write a bot, which makes good proposals as a hobby for a year or two. Didn't plan to use meerkat - no my mind starts to change, as I found this testbet. The challenge to beat other bots start changing my priorities. Man - I'm even thinking of implementing and contributing DIVAT analysis even though that wouldn't help my initials ideas.
So conclusion: I don't know yet where it will lead me, but the existence of the testbed definitely changed my priorities and thinking. There is a 50:50 chance that this could be to your good :mrgreen:

Anyway - if you don't believe on a good outcome it won't come.


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Sat Mar 13, 2010 1:14 pm 
Offline
PokerAI fellow
User avatar

Posts: 2342
Favourite Bot: My next one
Again. New version will be released when it's done.


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Sat Mar 13, 2010 7:19 pm 
Offline
Junior member
User avatar

Posts: 36
Favourite Bot: selfmade
I also looked at this testbed because I was interested in creating an SNG bot. I quickly realized that there are a lot of bugs in the implementation of some of the meerkat interfaces (either bugs or intentionally not implemented correctly). For example, in gameinfo, getNumSidePots always returns 0. So I've begun to work on my own testbed.


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Sun Mar 14, 2010 11:35 am 
Offline
Regular member
User avatar

Posts: 77
Favourite Bot: homebrew
doormat wrote:
I also looked at this testbed because I was interested in creating an SNG bot. I quickly realized that there are a lot of bugs in the implementation of some of the meerkat interfaces (either bugs or intentionally not implemented correctly). For example, in gameinfo, getNumSidePots always returns 0. So I've begun to work on my own testbed.


The source code is provided - why not just fix it?


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Sun Mar 14, 2010 12:06 pm 
Offline
PokerAI fellow
User avatar

Posts: 1673
Favourite Bot: none
That part is actually not released as source code. Those are not bugs but intendedly not implemented correctly, as we didn't need them so far. The Meerkat API is not completely implemented, neither in this public nor in our latest version but the most important things work.
If you find bugs or something unsatisfying feel free to post or PM about it instead of just going into the tank angry and start developing your own stuff. It could be just a small fix.

_________________
Cheers.


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Tue Apr 27, 2010 10:29 pm 
Offline
Junior member
User avatar

Posts: 10
Favourite Bot: HAL 9000
Hello,
I've programmed a bot that consistently beats SimpleBot in 5vs5 SNG Tournaments, as you requested in this thread:
http://pokerai.org/pf3/viewtopic.php?f=3&t=1402
In that thread you also mention the bot KillPhil, but I cannot find it in the Testbed, where can I download it? Will you release any more test bots I can play against?

My bot uses a fairly simple formula-based push-fold strategy and shouldn't be able to beat your very best bots, so I'm not sure if there's any point in sending it to you other than to get a general idea of how good it defends against more advanced bots. If I understand correctly, even if my bot was able to also beat KillPhil I would still not get access to any private forum, am I wrong?


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Tue Apr 27, 2010 10:43 pm 
Offline
Senior member
User avatar

Posts: 458
Location: Now in the mighty PolarBoar variant!
Favourite Bot: ...
matlab5765 wrote:
I've programmed a bot that consistently beats SimpleBot in 5vs5 SNG Tournaments


What is a 5vs5 SNG Tournament?


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Tue Apr 27, 2010 10:53 pm 
Offline
Junior member
User avatar

Posts: 10
Favourite Bot: HAL 9000
PolarBear wrote:
matlab5765 wrote:
I've programmed a bot that consistently beats SimpleBot in 5vs5 SNG Tournaments


What is a 5vs5 SNG Tournament?

I got that from the other thread. It means a 10 player SNG with:

Player 1: MyBot
Player 2: SimpleBot
Player 3: MyBot
Player 4: SimpleBot
Player 5: MyBot
Player 6: SimpleBot
Player 7: MyBot
Player 8: SimpleBot
Player 9: MyBot
Player 10: SimpleBot


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Tue Apr 27, 2010 11:23 pm 
Offline
Senior member
User avatar

Posts: 172
Favourite Bot: don't know
wrong post


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Wed Apr 28, 2010 3:10 pm 
Offline
PokerAI fellow
User avatar

Posts: 1673
Favourite Bot: none
Please post you results in numbers please @matlab.
We are working on the testbed continuosly and once we decide that we have a new version that should be released, we will. There might even be more test bots in future versions that you can test your bot against. For now feel free to PM me with more questions about the SNG-Testbed group. Let me know about your background, knowledge, plans and desires and upon that we make decisions if we let new people in or not. An already good strategy that beats our bots will be another very strong indicator to let you in.

_________________
Cheers.


Top
 Profile  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Wed Apr 28, 2010 9:50 pm 
Offline
Junior member
User avatar

Posts: 10
Favourite Bot: HAL 9000
Here are my results after 100000 tourneys:

Results, ROI: 18.27%
Balance of player Pusher0: 20102.0
Balance of player SimpleBot1: -41236.0
Balance of player Pusher2: 20112.0
Balance of player SimpleBot3: -41052.0
Balance of player Pusher4: 20082.0
Balance of player SimpleBot5: -41093.0
Balance of player Pusher6: 21908.0
Balance of player SimpleBot7: -39546.0
Balance of player Pusher8: 20911.0
Balance of player SimpleBot9: -40188.0
{Simpl=-203115.0, Pushe=103115.0}
Finished for 16856 sec.
Played 892 hands/sec.

I will give you more details by PM

Edit: I have made a small change to the algorithm that significantly improves the results (ROI about 28%) but I'm not going to run 100000 tourneys again. I will send you the PM tomorrow as now I'm going to sleep.


Top
 Profile E-mail  
 
 Post subject: Re: Public SNG Testbed
PostPosted: Thu May 27, 2010 11:25 am 
Offline
Junior member
User avatar

Posts: 10
Favourite Bot: HAL 9000
I'm having some trouble trying to integrate my bot into Poker Academy. I understand that the SNG Testbed uses the same API as Poker Academy, so if it works in the Testbed it should work in the Academy too, but when I try to import the .jar file into Poker Academy it gives me an error.
I think the problem may be in the way I generate the .jar file rather than the code itself, as I'm not familiar with the Eclipse IDE (or java for that matter, I'm more of a C programmer).

Do you have an idea which are the more likely causes (wrong version, wrong library, wrong compilation options, etc)?
Is there somewhere a fool-proof guide for integrating Meerkat bots into Poker Academy?
I'd really appreciate some help, and I think it's probably a common problem so it may be helpful to other forum members as well.

Sorry if there's already another thread that explains how to do this, I tried the search function but didn't find it.

PD: I've made some improvements to my bot's algorithm, but I guess it's still way too simple to be of any use to you. I can send you the code if you're interested.


Top
 Profile E-mail  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next


Who is online

Users browsing this forum: No registered users and 14 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: