Sunday, February 12, 2012

GNUbg 0-ply benchmarks reference

As a reference, here are the average error rates for GNUbg 0-ply on the three benchmark databases:

Contact: 10.5
Crashed: 11.0
Race: 1.01

The values are milli-error rates to be comparable to the values in the table in my benchmark results post (so 10.5 means 10.5e-3 = 0.0105 ppg/move average error rate).

My best player so far (Player 2.4) scores 16.4, 13.8, and 1.70 respectively. A multivariate linear regression  of Benchmark 2 score against the three benchmarks predicts GNUbg 0-ply would beat my best player by +0.11ppg. Most of the difference is in the Contact benchmark. Hopefully this will see some substantial relative improvement when I train using supervised learning instead of TD learning.

Crashed and Race came from Joseph Heled's page describing the GNUbg training program, and Contact came from an email to the bug-gnubg mailing list from Ã˜ystein Schønning-Johansen.



No comments:

Post a Comment