Also-Rans list updated

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Daniel Shawul
Posts: 4186
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Also-Rans list updated

Post by Daniel Shawul »

That is weird I have been struggling with it for 2 days using the online integrator, may be that is the problem then. Can you please try the general cases where 2 is replaced by b and also the probability p is not 0.5. It probably doesn't work for this one which is the primary reason i wanted to see if it works with some values.

First this

limit (Integral[ b^x, {x,0,d/2} ] +Integral[ exp(-2 * ((d*(0.5-1) + x)^2)/d) b^x, {x,d/2,d}])^(1/d) as d->infinity

Then this

limit (Integral[ b^x, {x,0,d/2} ] +Integral[ exp(-2 * ((d*(p-1) + x)^2)/d) b^x, {x,d/2,d}])^(1/d) as d->infinity

If it works please post the result.
Thanks.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Also-Rans list updated

Post by Laskos »

Daniel Shawul wrote:That is weird I have been struggling with it for 2 days using the online integrator, may be that is the problem then. Can you please try the general cases where 2 is replaced by b and also the probability p is not 0.5. It probably doesn't work for this one which is the primary reason i wanted to see if it works with some values.

First this

limit (Integral[ b^x, {x,0,d/2} ] +Integral[ exp(-2 * ((d*(0.5-1) + x)^2)/d) b^x, {x,d/2,d}])^(1/d) as d->infinity

Then this

limit (Integral[ b^x, {x,0,d/2} ] +Integral[ exp(-2 * ((d*(p-1) + x)^2)/d) b^x, {x,d/2,d}])^(1/d) as d->infinity

If it works please post the result.
Thanks.
First limit, after some complications with assumptions was
Sqrt(b)*exp(1/8*(Log b)^2), for b>1

Second limit it doesn't solve with assumptions I make (b>1, 1>p>0). Maybe I have to be more restrictive on b.
Daniel Shawul
Posts: 4186
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Also-Rans list updated

Post by Daniel Shawul »

Thank you very much Kai. Atleast now i know it can't have a simple form like b^(kd) that is common for uniform trees and the like. I will have to visit some ..ahm..warez and get this beast :)
Thanks again.
Daniel
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Also-Rans list updated

Post by Michel »

Here is an approximate formula I think for the difference between unscaled BayesElo and Logistic Elo.

Code: Select all

xb-xl=400 log10 [ (E+p)/(pE+1) ]

where E=10^(-x/400)         (the formula is exact if we take x=xb but the error is small if we take x=xl instead)

and  p=(1/2)(10^(d/400)+10^(-d/400))

with d=draw_elo
As Kai pointed out there is of course an exact formula but it does not look very nice.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Also-Rans list updated

Post by Laskos »

Michel wrote:Here is an approximate formula I think for the difference between unscaled BayesElo and Logistic Elo.

Code: Select all

xb-xl=400 log10 [ (E+p)/(pE+1) ]

where E=10^(-x/400)         (the formula is exact if we take x=xb but the error is small if we take x=xl instead)

and  p=(1/2)(10^(d/400)+10^(-d/400))

with d=draw_elo
As Kai pointed out there is of course an exact formula but it does not look very nice.
Neat formula. I plotted it against the exact one (d=240):

Image

Th exact one is a bit taller. The offset at x->infinity is perfectly matched, and is given by 400*Log10(p). There seems to be an issue with the derivative in 0, which is (p-1)/(p+1) in your formula. Can it be fixed or it will get too ugly?
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Also-Rans list updated

Post by Michel »

There seems to be an issue with the derivative in 0, which is (p-1)/(p+1) in your formula. Can it be fixed or it will get too ugly?
It seems much better matching can be achieved by making x equal to
xl/scale(d)
where scale(d) is the usual scale factor (this is a good approximation for xb for small elo)
scale(d)=(4 . 10^(-d/400))/(1+10^(-d/400))^2
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Also-Rans list updated

Post by Michel »

Nicer formating


Image

Of course this is only true for a match between two engines. Not for a tournament.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Also-Rans list updated

Post by Laskos »

Michel wrote:Nicer formating


Image

Of course this is only true for a match between two engines. Not for a tournament.
Yes, the scaling works. The derivative in 0 is now (p-1)/((p+1)*sd) and matches perfectly. Very nice, Michel. For a rating list, it it will be a similar formula, just needing to adjust empirically parametesrs to some values adapted to list connectedness among closer in strength engines. But for a single match this is perfect, and gives actually large effect, some 90 points for 200 points difference (for d=240).
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Also-Rans list updated

Post by Michel »

@Adam

Would it be possible for you to run BayesElo once again, but this time
with prior 0?

It seems to me that assuming that a random mover scored 2 draws against Houdini must lead to some serious rating compression, independently of the scaling issue.
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Also-Rans list updated

Post by Adam Hair »

I will share the results this evening.

By the way, I have been unsure about how the prior is applied. Is each engine assumed to have x draws against every engine in the database or is it assumed to have x draws against each engine it has played against?