Best tool for estimating engine rating?

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

User avatar
gleperlier
Posts: 1033
Joined: Sat Feb 04, 2012 10:03 pm

Re: Best tool for estimating engine rating?

Post by gleperlier »

bob wrote:
gleperlier wrote:Hello,

Is there a way to have an elo rating just with the results without PGN ? (I you lost some PGN for exemple) :?:
If you have a file with the results, it is simple enough to write a program that will spit out some bogus PGN (no moves are required) that BayesElo will cheerfully read and process.

If you mean a tournament crosstable, might be a bit trickier to write, but still, if you know the outcome, producing PGN that matches that outcome is not hard at all...
So basically the way it at the end to have some PGN, even if generated afterwards if you lost part of them.

Thanks !
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Best tool for estimating engine rating?

Post by Don »

gleperlier wrote:
bob wrote:
gleperlier wrote:Hello,

Is there a way to have an elo rating just with the results without PGN ? (I you lost some PGN for exemple) :?:
If you have a file with the results, it is simple enough to write a program that will spit out some bogus PGN (no moves are required) that BayesElo will cheerfully read and process.

If you mean a tournament crosstable, might be a bit trickier to write, but still, if you know the outcome, producing PGN that matches that outcome is not hard at all...
So basically the way it at the end to have some PGN, even if generated afterwards if you lost part of them.

Thanks !
Bayeselo and as far as I know also elostat rates games from PGN files. Whether you use PGN or some other format, the program must UNDERSTAND it. You don't specify what the input to the program should be and these utilities cannot guess at it.

Are you looking for a utility that prompts you for every value?

If you just have 2 programs and want to know based on a head to head results you can convert the score to ELO directly.

You can get the formula from the wikipedia on the ELO rating system.

This link gives you a formula for converting a winning percentage to an ELO difference:

http://www.stmintz.com/ccc/index.php?id=166443
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
gleperlier
Posts: 1033
Joined: Sat Feb 04, 2012 10:03 pm

Re: Best tool for estimating engine rating?

Post by gleperlier »

Don wrote:
gleperlier wrote:
bob wrote:
gleperlier wrote:Hello,

Is there a way to have an elo rating just with the results without PGN ? (I you lost some PGN for exemple) :?:
If you have a file with the results, it is simple enough to write a program that will spit out some bogus PGN (no moves are required) that BayesElo will cheerfully read and process.

If you mean a tournament crosstable, might be a bit trickier to write, but still, if you know the outcome, producing PGN that matches that outcome is not hard at all...
So basically the way it at the end to have some PGN, even if generated afterwards if you lost part of them.

Thanks !
Bayeselo and as far as I know also elostat rates games from PGN files. Whether you use PGN or some other format, the program must UNDERSTAND it. You don't specify what the input to the program should be and these utilities cannot guess at it.

Are you looking for a utility that prompts you for every value?

If you just have 2 programs and want to know based on a head to head results you can convert the score to ELO directly.

You can get the formula from the wikipedia on the ELO rating system.

This link gives you a formula for converting a winning percentage to an ELO difference:

http://www.stmintz.com/ccc/index.php?id=166443
Thanks Don !

Gab
User avatar
Ajedrecista
Posts: 1952
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: Best tool for estimating engine rating?

Post by Ajedrecista »

Hello Gabriel:

Quoting Don:
Don wrote:Are you looking for a utility that prompts you for every value?

If you just have 2 programs and want to know based on a head to head results you can convert the score to ELO directly.
I made a programme for those cases: nor PGN neither other files are required. It also gives you error bars for 1, 2 and 3-sigma confidence. Here is an example of its modest output. The use is very simple: double click the executable and follow the instructions. The links for download are in this post.

The mathematical model I used may be similar to EloSTAT one (I do not know exactly what EloSTAT uses). But my programme has the limitation of supporting only direct matches between two engines... and of course you can be sure that it is not the best tool for estimating engine ratings, although the formula posted by Don is logically the same I use; the main differences could be in error bars.

Do not expect miracles of my programme! :wink:

Regards from Spain.

Ajedrecista.
User avatar
lucasart
Posts: 3232
Joined: Mon May 31, 2010 1:29 pm
Full name: lucasart

Re: Best tool for estimating engine rating?

Post by lucasart »

kinderchocolate wrote:Nothing else for just running a match and reports the results? Rather than feeding the pgn games to the command tool?
there are two distinct features there. playing a tournament and statistics calculations. everyone has their preference, but my favorite tools for these jobs are respectively cutechess-cli for playing tournaments and bayeselo for calculating ratings and stats.