Poker-AI.org

Re: PureCFRM

2013-09-04T04:14:15+00:00

qScale is from 10^-2 to 10^2, which wouldn't have much of an affect, all other things being equal. The plain external sampling code is as it's implemented elsewhere, again, leaving me wondering what I'm doing wrong.

Code:

    $("#scaling").slider({
        orientation: "horizontal",
        min: 0,
        max: 1000,
        value: 500,
        slide: function (event, ui) {
            if (ui.value >= 490 && ui.value <= 510)
                scale = 1.0; else
                scale = Math.pow(10, ui.value / 250 - 2);

            delayedRestartWorker();
        }
    });

Statistics: Posted by cantina — Wed Sep 04, 2013 4:14 am

Re: PureCFRM

2013-09-04T03:51:12+00:00

qMode 1, qScale 1000 didn't work either. Hmm....

Statistics: Posted by cantina — Wed Sep 04, 2013 3:51 am

Re: PureCFRM

2013-09-03T06:20:35+00:00

The ui resets them at app.js/startWorker() Presumably there is a way to debug javascript

Statistics: Posted by spears — Tue Sep 03, 2013 6:20 am

Re: PureCFRM

2013-09-02T21:49:06+00:00

Thanks Spears. I found it now.

As far as I can tell, the way I was updating the regrets is correct. I was also sampling correctly as well.

Hmmm.... qMode and qScale: maybe has to do with conversion of decimals to integers? Yeah, thinking about this further, for fractional utilities you would need some kind of quantitative conversion.

Spears, did you see qScale anywhere?

Statistics: Posted by cantina — Mon Sep 02, 2013 9:49 pm

Re: PureCFRM

2013-09-02T07:35:47+00:00

Nice find.

Haven't a clue what it all means but maybe this will help.

Code:

    GNode.prototype.updateCFR = function (i, a, delta) {
        if (qMode == 0) {
            this.cfr[i][a] += delta;
        } else if (qMode == 1) {
            this.cfr[i][a] = Math.round(this.cfr[i][a] + delta * qScale);
        } else {
            delta *= qScale;
            var idelta;

            if (delta >= 0) {
                idelta = Math.floor(delta);
                var fraction = delta - idelta;
                if (Math.random() < fraction)
                    idelta++;
            } else {
                idelta = -Math.floor(-delta);
                var fraction = idelta - delta;
                if (Math.random() < fraction)
                    idelta--;
            }

            this.cfr[i][a] += idelta;
        }
    };

Code:

function sample(s) {
    var r = Math.random();

    var acc = 0;

    for (var i = 0; i < s.length; i++) {
        acc += s[i];
        if (r < acc)
            return i;
    }

    return 0;
}

Statistics: Posted by spears — Mon Sep 02, 2013 7:35 am

PureCFRM

2013-09-02T05:33:18+00:00

I found the below code for PureCFRM floating about on the interwebs and implemented it. However, it doesn't seem to work. I believe it's either the UpdateCFR function or the sample function that I'm misunderstanding.

For UpdateCFR, I'm just doing:
regret(hand) += u[a] - u[sampledAction];

For sample, I'm selecting an action probabilistically from regret(hand).

It's really such a simple algorithm I don't know where else I could have gone wrong. Regrets are integers, and the cumulative strategies are Uint16s.

Code:

// Oskari Tammelin 2010
//
// + no multiplications so cfr values can be integers
// + low iteration cost

function pureExternalSampling(player, node, hands)
{
    if (node.isTerminal())
        return node.getPayoff(player, hands);

    var sampledAction = sample(node.getCurrentStrategy(hands[node.player]));

    if (node.player == player)
    {
        var u = new Array(node.children.length);

        for (var a = 0; a < node.children.length; a++)
            u[a] = pureExternalSampling(player, node.children[a], hands);

        for (var a = 0; a < node.children.length; a++)
            node.updateCFR(hands[player], a, u[a] - u[sampledAction]);

        return u[sampledAction];
    }
    else
    {
        node.strategy[hands[node.player]][sampledAction]++;
        return pureExternalSampling(player, node.children[sampledAction], hands);
    }

}

function iteration()
{
    var hands = dealHands();
    pureExternalSampling(0, game, hands);
    pureExternalSampling(1, game, hands);
}

Statistics: Posted by cantina — Mon Sep 02, 2013 5:33 am