Best tool for estimating engine rating?

[kinderchocolate]

I want to run tournaments for estimating engine ratings. I have some reference engines with known ratings, I would like to estimate the performance for all other engines participated in the tournament (based on the reference engine). What would be the best tool for doing it?

For example, if my reference engine is 2000. And if my engine scores 50%, I expect the tool will give 2000 performance for my engine.

[hgm]

BayesElo?

[kinderchocolate]

Nothing else for just running a match and reports the results? Rather than feeding the pgn games to the command tool?

[rbarreira]

Why don't you make a simple script that runs the tournament and then runs bayeselo with the resulting pgn?

[kinderchocolate]

Well, I have been using Arena. I don't know how to batch it and send the pgn games to bayeselo.

[Adam Hair]

I want to run tournaments for estimating engine ratings. I have some reference engines with known ratings, I would like to estimate the performance for all other engines participated in the tournament (based on the reference engine). What would be the best tool for doing it?

For example, if my reference engine is 2000. And if my engine scores 50%, I expect the tool will give 2000 performance for my engine.

EloStat 1.3
Bayeselo
Ordo

All three should rate your engine at 2000 given the stated condition. Overall, I suspect Bayeselo is the most accurate if the additional options are used. But, Miguel Ballicora's program Ordo is also very good. I use Bayeselo simply out of habit and familiarity.

[Adam Hair]

Well, I have been using Arena. I don't know how to batch it and send the pgn games to bayeselo.

EloStat is built into Arena. Open the pgn with Arena and click on the EloStat button at the bottom of the database window (the window that appears when you open a pgn with Arena).

[tpetzke]

If you intend to run a large number of games cutechess is probably better than Arena and it gives you an ELO indication right away.

Score of iCE 0.3 v2716 vs iCE 0.3 v2708: 166 - 173 - 175 [0.49] 514
Started game 515 of 600 (iCE 0.3 v2716 vs iCE 0.3 v2708)
Finished game 515 (iCE 0.3 v2716 vs iCE 0.3 v2708): * {No result}
Score of iCE 0.3 v2716 vs iCE 0.3 v2708: 166 - 173 - 175 [0.49] 514
ELO difference: -5
Finished match

For a better estimate I use bayeselo as it also shows the error bar and stacks the new results with the previous results I had so far.

Thomas...

[gleperlier]

Hello,

Is there a way to have an elo rating just with the results without PGN ? (I you lost some PGN for exemple)

[bob]

Hello,

Is there a way to have an elo rating just with the results without PGN ? (I you lost some PGN for exemple)

If you have a file with the results, it is simple enough to write a program that will spit out some bogus PGN (no moves are required) that BayesElo will cheerfully read and process.

If you mean a tournament crosstable, might be a bit trickier to write, but still, if you know the outcome, producing PGN that matches that outcome is not hard at all...