Poker-AI.org
http://poker-ai.org/phpbb/

PureCFRM
http://poker-ai.org/phpbb/viewtopic.php?f=24&t=2571
Page 1 of 1

Author:  cantina [ Mon Sep 02, 2013 5:33 am ]
Post subject:  PureCFRM

I found the below code for PureCFRM floating about on the interwebs and implemented it. However, it doesn't seem to work. I believe it's either the UpdateCFR function or the sample function that I'm misunderstanding.

For UpdateCFR, I'm just doing:
regret(hand) += u[a] - u[sampledAction];

For sample, I'm selecting an action probabilistically from regret(hand).

It's really such a simple algorithm I don't know where else I could have gone wrong. Regrets are integers, and the cumulative strategies are Uint16s.

Code:
// Oskari Tammelin 2010
//
// + no multiplications so cfr values can be integers
// + low iteration cost

function pureExternalSampling(player, node, hands)
{
    if (node.isTerminal())
        return node.getPayoff(player, hands);

    var sampledAction = sample(node.getCurrentStrategy(hands[node.player]));

    if (node.player == player)
    {
        var u = new Array(node.children.length);

        for (var a = 0; a < node.children.length; a++)
            u[a] = pureExternalSampling(player, node.children[a], hands);

        for (var a = 0; a < node.children.length; a++)
            node.updateCFR(hands[player], a, u[a] - u[sampledAction]);

        return u[sampledAction];
    }
    else
    {
        node.strategy[hands[node.player]][sampledAction]++;
        return pureExternalSampling(player, node.children[sampledAction], hands);
    }

}

function iteration()
{
    var hands = dealHands();
    pureExternalSampling(0, game, hands);
    pureExternalSampling(1, game, hands);
}

Author:  spears [ Mon Sep 02, 2013 7:35 am ]
Post subject:  Re: PureCFRM

Nice find.

Haven't a clue what it all means but maybe this will help.

Code:
    GNode.prototype.updateCFR = function (i, a, delta) {
        if (qMode == 0) {
            this.cfr[i][a] += delta;
        } else if (qMode == 1) {
            this.cfr[i][a] = Math.round(this.cfr[i][a] + delta * qScale);
        } else {
            delta *= qScale;
            var idelta;

            if (delta >= 0) {
                idelta = Math.floor(delta);
                var fraction = delta - idelta;
                if (Math.random() < fraction)
                    idelta++;
            } else {
                idelta = -Math.floor(-delta);
                var fraction = idelta - delta;
                if (Math.random() < fraction)
                    idelta--;
            }

            this.cfr[i][a] += idelta;
        }
    };


Code:
function sample(s) {
    var r = Math.random();

    var acc = 0;

    for (var i = 0; i < s.length; i++) {
        acc += s[i];
        if (r < acc)
            return i;
    }

    return 0;
}

Author:  cantina [ Mon Sep 02, 2013 9:49 pm ]
Post subject:  Re: PureCFRM

Thanks Spears. I found it now.

As far as I can tell, the way I was updating the regrets is correct. I was also sampling correctly as well.

Hmmm.... qMode and qScale: maybe has to do with conversion of decimals to integers? Yeah, thinking about this further, for fractional utilities you would need some kind of quantitative conversion.

Spears, did you see qScale anywhere?

Author:  spears [ Tue Sep 03, 2013 6:20 am ]
Post subject:  Re: PureCFRM

The ui resets them at app.js/startWorker() Presumably there is a way to debug javascript

Author:  cantina [ Wed Sep 04, 2013 3:51 am ]
Post subject:  Re: PureCFRM

qMode 1, qScale 1000 didn't work either. Hmm....

Author:  cantina [ Wed Sep 04, 2013 4:14 am ]
Post subject:  Re: PureCFRM

qScale is from 10^-2 to 10^2, which wouldn't have much of an affect, all other things being equal. The plain external sampling code is as it's implemented elsewhere, again, leaving me wondering what I'm doing wrong.

Code:
    $("#scaling").slider({
        orientation: "horizontal",
        min: 0,
        max: 1000,
        value: 500,
        slide: function (event, ui) {
            if (ui.value >= 490 && ui.value <= 510)
                scale = 1.0; else
                scale = Math.pow(10, ui.value / 250 - 2);

            delayedRestartWorker();
        }
    });

Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/