Image Image Image




Post new topic Reply to topic  [ 18 posts ] 
Author Message
 Post subject: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 3:17 am 
Offline
New member
User avatar

Posts: 5
Favourite Bot: hawking
... so far its not winning (very small sample size, so maybe I'm just freaking out). I am pleased to have gotten this far (its taken alot of part time coding, sometimes very part time, to get to this point), but now I'd like to take it to the next level. I'd like input from you guys as to how I could take it to the next level...

The Format:
Hyper turbo (2 min blinds) heads up SNG (25 blinds)

How the bot works:
<15 Blinds: I went in with the very naive idea that nash equilibrium was the holy grail (without every really researching what it meant until recently \o/). So when the minimum stack gets below 15 blinds it follows a shove/fold strategy from http://www.holdemresources.net/h/poker-theory/hune.html. I
>15 Blinds: Its plays a horrible naive heuristics based strategy. Read as "help".

I guess I should have done more research as to how to play above 15 blinds, but I haven't to this point. So a few questions:

Questions
1. Firstly, has anyone code a Hyper HU SNG bot? If so, what strategy did you use?
2. Would nash equilibrium even make sense/make a profit here?
3. What would be the simplest non-hueristics based approach to playing above 15 blinds?
4. Finally, a long time ago, I purchased 1 million hand histories off a pro (playing 9 max as I initially aimed to make a 9 max bot). I had the view of basically mimicking the pro's move in every spot e.g. query the action when hero has small pair, on KXX board, given the action etc. Has anyone tried this? Or is this strategy madness?
5. tl;dr: I am making a Hyper HU SNG bot. Its running as I speak, but making a loss, any suggestions for a profitable model?
6. Ive checked the poker papers thread but can't see much on heads up, any papers on HU available?

Ive arrived at the stage where the framework has been written i.e. it joins games automatically, it reads the cards, it knows when its its turn to go, it knows the hand so far etc, what I really need is help creating a profitable Hyper HU SNG model, any suggestions?

Thanks


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 4:00 am 
Offline
Regular member
User avatar

Posts: 76
Favourite Bot: ...
A quick thing to try that I can think of: On X BB stacks apply 0,5X BB nash rules maybe (for 10 BB effective stack -> use 5 BB chart/formula). It's hyper turbo after all... you get the idea, take the increasing blinds into account + It's a good place to start doing some simple opponent modeling

(I've never done any Nash pushbotting myself.)


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 10:43 am 
Offline
PokerAI fellow
User avatar

Posts: 1562
Location: Finland
Favourite Bot: Self-made
hawking wrote:
4. Finally, a long time ago, I purchased 1 million hand histories off a pro (playing 9 max as I initially aimed to make a 9 max bot). I had the view of basically mimicking the pro's move in every spot e.g. query the action when hero has small pair, on KXX board, given the action etc. Has anyone tried this? Or is this strategy madness?
IMO this is impossible because the pro most likely isn't basing his actions purely on his cards and board cards. To be a pro, you definitely need to adapt your game to other players and you can't tell from the hand histories what specific tendency/stat/gutfeel the pro has used in a specific hand. Against player WEAK-FISH he might have raised with a small pair on KXX board when the FISH donks as he will almost never have K in the pocket there but against NUTSROCK he might just fold the small pair on KXX board because NUTSROCK will have AK/KK/KQ in the pocket 100% of the time there and he will not fold it. You can't tell that stuff from the hand histories really so IMO the bot would be very mechanical and badly exploitable.

_________________
Opinions expressed are my own, your mileage may vary... ;)
Warning: If I spot an opportunity to give sarcastic replies, I will take it. Nothing personal. I don't even know you.


Top
 Profile  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 1:30 pm 
Offline
New member
User avatar

Posts: 5
Favourite Bot: hawking
TooMuchCoffee wrote:
hawking wrote:
4. Finally, a long time ago, I purchased 1 million hand histories off a pro (playing 9 max as I initially aimed to make a 9 max bot). I had the view of basically mimicking the pro's move in every spot e.g. query the action when hero has small pair, on KXX board, given the action etc. Has anyone tried this? Or is this strategy madness?
IMO this is impossible because the pro most likely isn't basing his actions purely on his cards and board cards. To be a pro, you definitely need to adapt your game to other players and you can't tell from the hand histories what specific tendency/stat/gutfeel the pro has used in a specific hand. Against player WEAK-FISH he might have raised with a small pair on KXX board when the FISH donks as he will almost never have K in the pocket there but against NUTSROCK he might just fold the small pair on KXX board because NUTSROCK will have AK/KK/KQ in the pocket 100% of the time there and he will not fold it. You can't tell that stuff from the hand histories really so IMO the bot would be very mechanical and badly exploitable.


Fair enough, I planned to model more than just the card however. i.e. villain VPIP etc, previous actions etc. I thought I could implement a machine learning approach (SVM for example) to classify between call/fold/raise etc based on vectors e.g. [holecards, board, villain VPIP, previous actions, etc]...

Kratt wrote:
A quick thing to try that I can think of: On X BB stacks apply 0,5X BB nash rules maybe (for 10 BB effective stack -> use 5 BB chart/formula). It's hyper turbo after all... you get the idea, take the increasing blinds into account + It's a good place to start doing some simple opponent modeling

(I've never done any Nash pushbotting myself.)


So, long story short, as its a hyper, shove wider than nash equilibrium?


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 1:47 pm 
Offline
Regular member
User avatar

Posts: 76
Favourite Bot: ...
No, ignore my previous advice. That was pretty stupid actually. Thinking again, I wouldn't dare to apply it to a > 8bb game myself.

Guess you are losing mainly by ignoring any stealing/3betting and such. More than you gained by having a marginal edge on taking flips.

Try out MCTS(tree searh using opponent modelling, try out just push-fold first) or CFRM(using self play).
And about the papers, most of the research begins with heads-up, so you have plenty of material available. Just a bit more effort


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 3:38 pm 
Offline
Senior member
User avatar

Posts: 136
Location: Siberia
Favourite Bot: the profitable one
My two cents.

Look at the picture attached. These are finish positions for the tournaments with the same structure but different speed. The left part is for regular, and right one is for turbo.

Bot plays the same way and plays NE when effective stacks <14bb. In turbos it is ok because stack/blind ratio is usually low in HU. But it is not ok for regular tourneys.

So, my point is - playing pure NE is not enough to make winning HU bot, even for hyperturbos. For breaking even, may be.
Due to post flop in early stage you will have less chip then opponent entering into push-fold zone - and it is the one leak.

Also opponent can limp or raise - what are you going to do with it?

Lot of work ahead :)


Attachments:
aaa.png
aaa.png [ 10.8 KB | Viewed 1829 times ]

_________________
Бетономешалка!!! Мешает бетон!!!
Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 4:22 pm 
Offline
New member
User avatar

Posts: 5
Favourite Bot: hawking
concretemixer wrote:
My two cents.

Look at the picture attached. These are finish positions for the tournaments with the same structure but different speed. The left part is for regular, and right one is for turbo.

Bot plays the same way and plays NE when effective stacks <14bb. In turbos it is ok because stack/blind ratio is usually low in HU. But it is not ok for regular tourneys.

So, my point is - playing pure NE is not enough to make winning HU bot, even for hyperturbos. For breaking even, may be.
Due to post flop in early stage you will have less chip then opponent entering into push-fold zone - and it is the one leak.

Also opponent can limp or raise - what are you going to do with it?

Lot of work ahead :)


Very interesting! At the moment my approach for >15bb is a simple (and horribly complex) rule base system, but i want to move away from this, into something cleaner. So how does your bot play >14bb, what strategy? Also, how do you play <14bb when the villain limps? Do you shove the NE Calling range? (thats what I do atm)

Thanks


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Sun Dec 02, 2012 9:16 pm 
Offline
Regular member
User avatar

Posts: 87
Favourite Bot: Mine
Quote:
4. Finally, a long time ago, I purchased 1 million hand histories off a pro (playing 9 max as I initially aimed to make a 9 max bot). I had the view of basically mimicking the pro's move in every spot e.g. query the action when hero has small pair, on KXX board, given the action etc. Has anyone tried this? Or is this strategy madness?


That is not the way to do it. You have to understand the reasoning behind how the pro plays. Just having the histories will not help much. You have to send your bots histories to the pro and have him correct it. Bad fold here, should have bet here etc. This process could take months and months believe me. I had to do this for my FL bot and it was very time consuming and slow but it worked and paid off.


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Mon Dec 03, 2012 6:37 am 
Offline
PokerAI fellow
User avatar

Posts: 1562
Location: Finland
Favourite Bot: Self-made
hawking wrote:
Fair enough, I planned to model more than just the card however. i.e. villain VPIP etc, previous actions etc. I thought I could implement a machine learning approach (SVM for example) to classify between call/fold/raise etc based on vectors e.g. [holecards, board, villain VPIP, previous actions, etc]...
You could, but how would you know WHAT information the pro used in what specific situation? He could have used VPIP on some hand, Fold to 3Bet on another, Fold to T Cbet in 3Bet Pot on another. And obv he could have had some specific hand in mind vs certain villain two days ago which has relevance to his decision on a hand. Or he sees that some player is tilting (for some reason or another, maybe some chat) and exploits that on the next hand.

_________________
Opinions expressed are my own, your mileage may vary... ;)
Warning: If I spot an opportunity to give sarcastic replies, I will take it. Nothing personal. I don't even know you.


Top
 Profile  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Mon Dec 03, 2012 9:36 am 
Offline
Senior member
User avatar

Posts: 136
Location: Siberia
Favourite Bot: the profitable one
hawking wrote:
So how does your bot play >14bb, what strategy?

Same strategy as for full table. I have no specific HU code - and human has the edge here. I don't care yet because my bot is for MTSNG.

hawking wrote:
Also, how do you play <14bb when the villain limps? Do you shove the NE Calling range? (thats what I do atm)

It pushes NE push range. Lots of fold equity here.

_________________
Бетономешалка!!! Мешает бетон!!!


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Mon Dec 03, 2012 4:25 pm 
Offline
New member
User avatar

Posts: 1
Favourite Bot: OpenHoldem
I'd advise you not to use Nash for 15 bb deep game.
Having the possibility to raise/call and raise/fold is much better again most opponents.

Also you say it does not play well above 15bb deep, so basically you enter Nash phase, in average, behind vilain (smaller stack).
And the small edge you have playing Nash does not compensate enough the equity you lost in the early stage.

Then, below 8-10 bb, you can use Nash, but if vilain is not following Nash then you could be winning much more by deviating from Nash yourself (shove lighter if he's too tight, or the opposite).


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Mon Dec 03, 2012 7:22 pm 
Offline
PokerAI fellow
User avatar

Posts: 1115
Favourite Bot: Johnny #5
Push/Fold < 12BB, otherwise multiple NNs trained on descent players combined with importance sampling...

Still loses at $2 SNGs. ;)

I'm not sure what I'm doing wrong, but it's not a simple problem. I'd like to try an EQ, but my EQs don't seem to be converging right. O_o

Frankly, I'm starting to think I've angered the NL gods. I'm about to try sacrificing a lamb or something.


Top
 Profile  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Mon Dec 03, 2012 8:21 pm 
Offline
PokerAI fellow
User avatar

Posts: 1115
Favourite Bot: Johnny #5
Does it matter what he used to make his decision as long as it was a sound judgement and can be regressed somehow?

TooMuchCoffee wrote:
You could, but how would you know WHAT information the pro used in what specific situation? He could have used VPIP on some hand, Fold to 3Bet on another, Fold to T Cbet in 3Bet Pot on another. And obv he could have had some specific hand in mind vs certain villain two days ago which has relevance to his decision on a hand. Or he sees that some player is tilting (for some reason or another, maybe some chat) and exploits that on the next hand.


Top
 Profile  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Tue Dec 04, 2012 12:25 am 
Offline
New member
User avatar

Posts: 2
Favourite Bot: My Own
c2008 wrote:
Does it matter what he used to make his decision as long as it was a sound judgement and can be regressed somehow?


Then you would also need to have the player database of the pro and all his notes and you would still have to manually decide which of the information played how large a role in every decision.


Top
 Profile  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Tue Dec 04, 2012 2:04 am 
Offline
New member
User avatar

Posts: 5
Favourite Bot: hawking
TooMuchCoffee wrote:
hawking wrote:
Fair enough, I planned to model more than just the card however. i.e. villain VPIP etc, previous actions etc. I thought I could implement a machine learning approach (SVM for example) to classify between call/fold/raise etc based on vectors e.g. [holecards, board, villain VPIP, previous actions, etc]...
You could, but how would you know WHAT information the pro used in what specific situation? He could have used VPIP on some hand, Fold to 3Bet on another, Fold to T Cbet in 3Bet Pot on another. And obv he could have had some specific hand in mind vs certain villain two days ago which has relevance to his decision on a hand. Or he sees that some player is tilting (for some reason or another, maybe some chat) and exploits that on the next hand.


Fair point, I think this is very true for a deep stack full ring game. I think in a hyper HU SNG environment, the decisions are *kind of* robotic though, and depend mostly on the cards and the villain style. So maybe classifying vectors of features (holecards, villian VPIP) maaay work, I may be wrong though.


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Thu Dec 06, 2012 1:16 am 
Offline
Regular member
User avatar

Posts: 87
Favourite Bot: Mine
I am completely new to the HU SNG game but have decided to tackle it after finishing my 9 player deep stack NL cash bot that is doing very well (10 months of work). A few days ago I was completely in the dark as to what NE was or anything related to SNG but now I understand what everybody is talking about in this thread. After just a few days I have a working bot that I am testing on the 33BB Turbo games. I read as many posts about strategy on these games as I could find and came up with the following roughed in strategy that I could use advice on.

First of all I noticed that many good players recommend raising just about button. This makes sense as you will be able to run over weak players, gain information, and use standard techniques for position poker post flop. So I adopted a 70+% raise strategy right out of gate. It runs until the player starts playing back at me or calls too many raises..it then adapts and starts a combination of raising 40% and limping. If the villian attacks the limp it folds more and throws in a few strong limps to mix it up a bit to keep the guy off balance. OOP is similar but tighter ranges obviously. It will start raising more OOP if the villian is a constant limper and does not raise often however. Eventually they get sick of it and play back at you but when they do they usually have a hand.

It just keeps playing like this until NE kicks in at around 11M (for now). Hopefully by then I have busted them. Any insight would be appreciated.


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Fri Dec 07, 2012 2:24 am 
Offline
New member
User avatar

Posts: 5
Favourite Bot: hawking
H
shalako wrote:
I am completely new to the HU SNG game but have decided to tackle it after finishing my 9 player deep stack NL cash bot that is doing very well (10 months of work). A few days ago I was completely in the dark as to what NE was or anything related to SNG but now I understand what everybody is talking about in this thread. After just a few days I have a working bot that I am testing on the 33BB Turbo games. I read as many posts about strategy on these games as I could find and came up with the following roughed in strategy that I could use advice on.

First of all I noticed that many good players recommend raising just about button. This makes sense as you will be able to run over weak players, gain information, and use standard techniques for position poker post flop. So I adopted a 70+% raise strategy right out of gate. It runs until the player starts playing back at me or calls too many raises..it then adapts and starts a combination of raising 40% and limping. If the villian attacks the limp it folds more and throws in a few strong limps to mix it up a bit to keep the guy off balance. OOP is similar but tighter ranges obviously. It will start raising more OOP if the villian is a constant limper and does not raise often however. Eventually they get sick of it and play back at you but when they do they usually have a hand.

It just keeps playing like this until NE kicks in at around 11M (for now). Hopefully by then I have busted them. Any insight would be appreciated.


Sounds good. How do you plan on playing post flop? As i believe early on, this is where alot of the chips are accumulated. It sounds like you are in a similar position to, would be good to maybe get an IM chat or something..


Top
 Profile E-mail  
 
 Post subject: Re: I've spent 6 months coding a Hyper Turbo Heads up bot but...
PostPosted: Fri Dec 07, 2012 4:23 am 
Offline
Regular member
User avatar

Posts: 87
Favourite Bot: Mine
Quote:
Sounds good. How do you plan on playing post flop? As i believe early on, this is where alot of the chips are accumulated. It sounds like you are in a similar position to, would be good to maybe get an IM chat or something..


Right now I have adopted my 9 player HU post flop strategy (which is quite extensive) with a few modifications for starters. It will need to be loosened up quite a bit for a pure HU game but it will work for now. I need to study a bit more HU game theory as I am not real comfortable with HU yet. I like the way preflop is working now but I did make a bunch of OOP changes and how it reacts to 3 betting. It was not adapting quick enough to what the villian was doing so I had to shorten that up too.


Top
 Profile E-mail  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 18 posts ] 


Who is online

Users browsing this forum: Google and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: