Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 5:47 pm

All times are UTC




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: PureCFRM
PostPosted: Mon Sep 02, 2013 5:33 am 
Offline
Veteran Member

Joined: Thu Feb 28, 2013 2:39 am
Posts: 437
I found the below code for PureCFRM floating about on the interwebs and implemented it. However, it doesn't seem to work. I believe it's either the UpdateCFR function or the sample function that I'm misunderstanding.

For UpdateCFR, I'm just doing:
regret(hand) += u[a] - u[sampledAction];

For sample, I'm selecting an action probabilistically from regret(hand).

It's really such a simple algorithm I don't know where else I could have gone wrong. Regrets are integers, and the cumulative strategies are Uint16s.

Code:
// Oskari Tammelin 2010
//
// + no multiplications so cfr values can be integers
// + low iteration cost

function pureExternalSampling(player, node, hands)
{
    if (node.isTerminal())
        return node.getPayoff(player, hands);

    var sampledAction = sample(node.getCurrentStrategy(hands[node.player]));

    if (node.player == player)
    {
        var u = new Array(node.children.length);

        for (var a = 0; a < node.children.length; a++)
            u[a] = pureExternalSampling(player, node.children[a], hands);

        for (var a = 0; a < node.children.length; a++)
            node.updateCFR(hands[player], a, u[a] - u[sampledAction]);

        return u[sampledAction];
    }
    else
    {
        node.strategy[hands[node.player]][sampledAction]++;
        return pureExternalSampling(player, node.children[sampledAction], hands);
    }

}

function iteration()
{
    var hands = dealHands();
    pureExternalSampling(0, game, hands);
    pureExternalSampling(1, game, hands);
}


Top
 Profile  
 
 Post subject: Re: PureCFRM
PostPosted: Mon Sep 02, 2013 7:35 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
Nice find.

Haven't a clue what it all means but maybe this will help.

Code:
    GNode.prototype.updateCFR = function (i, a, delta) {
        if (qMode == 0) {
            this.cfr[i][a] += delta;
        } else if (qMode == 1) {
            this.cfr[i][a] = Math.round(this.cfr[i][a] + delta * qScale);
        } else {
            delta *= qScale;
            var idelta;

            if (delta >= 0) {
                idelta = Math.floor(delta);
                var fraction = delta - idelta;
                if (Math.random() < fraction)
                    idelta++;
            } else {
                idelta = -Math.floor(-delta);
                var fraction = idelta - delta;
                if (Math.random() < fraction)
                    idelta--;
            }

            this.cfr[i][a] += idelta;
        }
    };


Code:
function sample(s) {
    var r = Math.random();

    var acc = 0;

    for (var i = 0; i < s.length; i++) {
        acc += s[i];
        if (r < acc)
            return i;
    }

    return 0;
}


Top
 Profile  
 
 Post subject: Re: PureCFRM
PostPosted: Mon Sep 02, 2013 9:49 pm 
Offline
Veteran Member

Joined: Thu Feb 28, 2013 2:39 am
Posts: 437
Thanks Spears. I found it now.

As far as I can tell, the way I was updating the regrets is correct. I was also sampling correctly as well.

Hmmm.... qMode and qScale: maybe has to do with conversion of decimals to integers? Yeah, thinking about this further, for fractional utilities you would need some kind of quantitative conversion.

Spears, did you see qScale anywhere?


Top
 Profile  
 
 Post subject: Re: PureCFRM
PostPosted: Tue Sep 03, 2013 6:20 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
The ui resets them at app.js/startWorker() Presumably there is a way to debug javascript


Top
 Profile  
 
 Post subject: Re: PureCFRM
PostPosted: Wed Sep 04, 2013 3:51 am 
Offline
Veteran Member

Joined: Thu Feb 28, 2013 2:39 am
Posts: 437
qMode 1, qScale 1000 didn't work either. Hmm....


Top
 Profile  
 
 Post subject: Re: PureCFRM
PostPosted: Wed Sep 04, 2013 4:14 am 
Offline
Veteran Member

Joined: Thu Feb 28, 2013 2:39 am
Posts: 437
qScale is from 10^-2 to 10^2, which wouldn't have much of an affect, all other things being equal. The plain external sampling code is as it's implemented elsewhere, again, leaving me wondering what I'm doing wrong.

Code:
    $("#scaling").slider({
        orientation: "horizontal",
        min: 0,
        max: 1000,
        value: 500,
        slide: function (event, ui) {
            if (ui.value >= 490 && ui.value <= 510)
                scale = 1.0; else
                scale = Math.pow(10, ui.value / 250 - 2);

            delayedRestartWorker();
        }
    });


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group