can we estimate strength of an engine given time handicap?

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
tiger
Posts: 819
Joined: Sat Mar 11, 2006 3:15 am
Location: Guadeloupe (french caribbean island)

Re: can we estimate strength of an engine given time handica

Post by tiger »

hgm wrote:Hmm, I was using this table:

http://gobase.org/studying/articles/elo/

Should I multiply by a factor sqrt(2), because 200 is the standard deviation per player, and what you measure is the difference in strength of two players?

I do not know why their table differs from what I know. They may be using different scaling factors.

Anyway for chess there exists a very simple estimate for an otherwise complicated function which I do not even remember (full of logs and stuff you do no want to meet in person).

If the winning percentage for a program A against program B is between 20% and 80%, take this winning percentage expressed in percents, remove 50 from it, and mutiply the result by seven. You get a fairly good approximate of the elo difference elo(A)-elo(B).

The formula get too inaccurate outside the 20%-80% winning percentage range, so in this case do not use it (or if you do, please don't mention my name).

Examples:

1- program A wins by 60% against program B. Program A is ~70 elo points above program B.

2- program A does only 30% against program B. Program A is ~140 elo points below program B.

3- program A wins by 90% against program B. Play it on the safe side and just say that program A is more than 210 elo points better then program B (who cares anyway :) ).

Please note that this formula doesn't say a word about error margins, which depend mainly on the number of games played, not on the winning percentage. For error margins I use precomputed tables.


// Christophe
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: can we estimate strength of an engine given time handica

Post by hgm »

OK, thanks. I was using that rule, but with a factor 5 in stead of 7, as I apparantly based it on this erroneous table. (Which suggested the standard deviation was 200.) :oops:

7 is indeed 5*sqrt(2), as sqrt(2)=1.4142...

On the positive side is that I made far mor progress in ELO with my engines that I so far thought I had, based on the win percentages! :lol: 8-)