Is there a tool to generate my own rating list?

[Uri Blass]

I can play more than 10 games in a minute with small fixed depth or small fixed nodes per move in the banksia interface but I know no tool to take the games and do from them a rating list.

I can make one tournament with rating list with banksia gui (round robin) but I know no way to add participants to the tournament after the tournament is finished and I know no tool to make rating list based on pgn of some tournaments or based on cross table of some tournaments.

I also do not like banksia rating because it can give different rating for engines that scored the same number of points in the same round robin tournament.

[Wilhelm]

I use Ordo, very simple. Put your games in a pgn in the Ordo folder, configure the batch once and you are done.
Ordo:
https://mega.nz/file/PfgkXCqC#nKu7ugHdc ... zvst_MTk_k

[Modern Times]

Yes that is right, with the right parameters Ordo generates some fairly good output.

[Uri Blass]

I use Ordo, very simple. Put your games in a pgn in the Ordo folder, configure the batch once and you are done.
Ordo:
https://mega.nz/file/PfgkXCqC#nKu7ugHdc ... zvst_MTk_k

Thanks.
Downloaded it.
Not sure what I need to do to configure the batch.

[Wilhelm]

I use Ordo, very simple. Put your games in a pgn in the Ordo folder, configure the batch once and you are done.
Ordo:
https://mega.nz/file/PfgkXCqC#nKu7ugHdc ... zvst_MTk_k

Thanks.
Downloaded it.
Not sure what I need to do to configure the batch.

Uri there is a good manuel inside
I try to make it simple
Generel: a minimum of 500 games against 20 opponents is a good begin to get realistic elo ratings. 30 matches between two engines is also a good minimum because there could be 7 games in a row with looses/wins so 10 games could give higher false ratings.



Wilhelm

[Dann Corbit]

I use Ordo, very simple. Put your games in a pgn in the Ordo folder, configure the batch once and you are done.
Ordo:
https://mega.nz/file/PfgkXCqC#nKu7ugHdc ... zvst_MTk_k

Thanks.
Downloaded it.
Not sure what I need to do to configure the batch.

Just edit the sample batch file.
You can also use other rating programs like Elostat.
I think you entered a Kaggle contest to make an Elo calculator. You could even use that.
This search might help too:
https://www.google.com/search?q=github+ ... e&ie=UTF-8

[jpqy]

Maybe this can help you also ,a little tutorial from Ipmanchess how he use Ordo.

https://ipmanchess.yolasite.com/tutorial.php

Scroll to the end off that page.

[Uri Blass]

I use Ordo, very simple. Put your games in a pgn in the Ordo folder, configure the batch once and you are done.
Ordo:
https://mega.nz/file/PfgkXCqC#nKu7ugHdc ... zvst_MTk_k

Thanks.
Downloaded it.
Not sure what I need to do to configure the batch.

Just edit the sample batch file.
You can also use other rating programs like Elostat.
I think you entered a Kaggle contest to make an Elo calculator. You could even use that.
This search might help too:
https://www.google.com/search?q=github+ ... e&ie=UTF-8

Thanks
I edited the sample file.
I made a round robin of 100 players and here is my rating list
I am happy with the fact that engines with the same number of points got the same rating.

Note that after running the erdo example file and running it I cannot find it and I can find the original example file only by unzipping the file again in a different folder.

I basically replaced
ordo-win32.exe -a 2834 -A "Naum 4.2" -W -p games.pgn -o rating.txt
by
ordo-win32.exe -a 0 -A Caissa 1.17 depth 1"" -W -p uritour.pgn -o rating1.txt

and got the following rating.

Suppose I run it with another pgn in the same folder.

Does it go to use also the result of the previous pgn?

Note that I will probably not going to do it because it was only an example and I find part of the engines as not reliable for rating because they behave in a way no human behave(for example Seer Depth 1 can play well only to make stalemate in the end of the game.

I prefer that even the weak engines in the list can practically win with a big advantage like weak humans.
I think fixed nodes that are not too low number of nodes are ok for that purpose.

I will probably avoid fixed depth for a future rating list because with fixed depth engines can play good in the opening and bad in the endgame.



# PLAYER : RATING POINTS PLAYED (%)
1 Stockfish 16.1 2048nodes : 1218.1 186.0 198 94
2 Caissa 1.17 depth 7 : 1066.2 174.0 198 88
3 Alexandria-6.0 depth7 : 1051.3 172.5 198 87
4 Wasp 6.50 depth 7 : 1046.5 172.0 198 87
5 Alexandria-6.0 2048 nodes : 1032.3 170.5 198 86
6 Stockfish 16.1 1024nodes : 1027.7 170.0 198 86
7 Wasp 6.50 depth 4 : 1027.7 170.0 198 86
8 Caissa 1.17 AVX2 2048nodes : 1023.2 169.5 198 86
9 Seer 2.8.0 depth 7 : 1005.5 167.5 198 85
10 Wasp 6.50 depth 6 : 992.7 166.0 198 84
11 Wasp 6.50 depth 5 : 968.1 163.0 198 82
12 Alexandria-6.0 depth6 : 964.1 162.5 198 82
13 Obsidian 11.0 depth 7 : 944.7 160.0 198 81
14 Wasp 6.50 depth 3 : 933.3 158.5 198 80
15 Stockfish 16.1 depth 7 : 925.9 157.5 198 80
16 Caissa 1.17 depth 6 : 904.3 154.5 198 78
17 Igel 3.5.0 depth 7 : 900.8 154.0 198 78
18 Seer 2.8.0 2048 nodes : 883.4 151.5 198 77
19 Caissa 1.17 AVX2 1024nodes : 880.0 151.0 198 76
20 Alexandria-6.0 depth5 : 869.9 149.5 198 76
21 Wasp 6.50 2048 nodes : 866.5 149.0 198 75
22 Berserk 12 2048 nodes : 856.6 147.5 198 74
23 Wasp 6.50 depth 2 : 837.2 144.5 198 73
24 Obsidian 11.0 2048nodes : 837.2 144.5 198 73
25 Alexandria-6.0 1024 nodes : 793.7 137.5 198 69
26 RubiChess 20240112 2048nodes : 790.7 137.0 198 69
27 Caissa 1.17 depth 5 : 784.7 136.0 198 69
28 Igel 3.5.0 2048 nodes : 761.1 132.0 198 67
29 Clover 6.1 depth 7 : 761.1 132.0 198 67
30 Stockfish 16.1 512 nodes : 752.4 130.5 198 66
31 Berserk 12 depth7 : 740.9 128.5 198 65
32 Wasp 6.50 1024 nodes : 732.4 127.0 198 64
33 Alexandria-6.0 depth4 : 729.5 126.5 198 64
34 Stockfish 16.1 depth 6 : 721.1 125.0 198 63
35 Wasp 6.50 512 nodes : 712.7 123.5 198 62
36 Alexandria-6.0 512 nodes : 704.4 122.0 198 62
37 Rubi20240112 depth7 : 685.2 118.5 198 60
38 Seer 2.8.0 1024 nodes : 685.2 118.5 198 60
39 Clover 6.1 2048 nodes : 679.8 117.5 198 59
40 Igel 3.5.0 depth 6 : 645.0 111.0 198 56
41 Alexandria-6.0 depth 3 : 618.7 106.0 198 54
42 Obsidian 11.0 depth 6 : 613.5 105.0 198 53
43 Berserk 12 1024 nodes : 603.1 103.0 198 52
44 Caissa 1.17 512 nodes : 600.5 102.5 198 52
45 Rubi20240112 depth6 : 600.5 102.5 198 52
46 RubiChess 20240112 1024nodes : 595.3 101.5 198 51
47 Wasp 6.50 depth 1 : 592.7 101.0 198 51
48 Caissa 1.17 depth 4 : 590.1 100.5 198 51
49 Obsidian 11.0 1024nodes : 585.0 99.5 198 50
50 Seer 2.8.0 depth 6 : 579.8 98.5 198 50
51 Stockfish 16.1 depth 5 : 543.9 91.5 198 46
52 Igel 3.5.0 1024 nodes : 543.9 91.5 198 46
53 Berserk 12 depth6 : 508.2 84.5 198 43
54 Seer 2.8.0 depth 5 : 500.5 83.0 198 42
55 Rubi20240112 512nodes : 498.0 82.5 198 42
56 Alexandria-6.0 depth 2 : 495.4 82.0 198 41
57 Seer 2.8.0 512 nodes : 487.7 80.5 198 41
58 Obsidian 11.0 depth 5 : 472.3 77.5 198 39
59 Obsidian 11.0 depth 1 : 469.7 77.0 198 39
60 Rubi20240112 depth5 : 456.8 74.5 198 38
61 Clover 6.1 1024 nodes : 441.2 71.5 198 36
62 Clover 6.1 depth 6 : 438.5 71.0 198 36
63 Rubi20240112 depth2 : 438.5 71.0 198 36
64 Igel 3.5.0 depth 5 : 435.9 70.5 198 36
65 Obsidian 11.0 512 nodes : 433.3 70.0 198 35
66 Igel 3.5.0 depth 1 : 425.4 68.5 198 35
67 Rubi20240112 depth4 : 412.1 66.0 198 33
68 Obsidian 11.0 depth 4 : 412.1 66.0 198 33
69 Stockfish 16.1 depth 1 : 412.1 66.0 198 33
70 Rubi20240112 depth3 : 406.8 65.0 198 33
71 Stockfish 16.1 depth 4 : 398.7 63.5 198 32
72 Igel 3.5.0 512 nodes : 385.1 61.0 198 31
73 Berserk 12 512 nodes : 385.1 61.0 198 31
74 Igel 3.5.0 depth 3 : 385.1 61.0 198 31
75 Seer 2.8.0 depth 1 : 382.4 60.5 198 31
76 Igel 3.5.0 depth 4 : 382.4 60.5 198 31
77 Seer 2.8.0 depth 4 : 374.1 59.0 198 30
78 Clover 6.1 depth 1 : 371.3 58.5 198 30
79 Berserk 12 depth5 : 368.6 58.0 198 29
80 Igel 3.5.0 depth 2 : 365.8 57.5 198 29
81 Rubi20240112 depth1 : 363.0 57.0 198 29
82 Alexandria-6.0 depth1 : 363.0 57.0 198 29
83 Clover 6.1 512 nodes : 357.3 56.0 198 28
84 Stockfish 16.1 depth 3 : 351.7 55.0 198 28
85 Obsidian 11.0 depth 2 : 351.7 55.0 198 28
86 Clover 6.1 depth 2 : 351.7 55.0 198 28
87 Clover 6.1 depth 5 : 351.7 55.0 198 28
88 Caissa 1.17 depth 3 : 337.3 52.5 198 27
89 Clover 6.1 depth 3 : 325.6 50.5 198 26
90 Berserk 12 depth2 : 313.6 48.5 198 24
91 Clover 6.1 depth 4 : 298.4 46.0 198 23
92 Berserk 12 depth4 : 273.0 42.0 198 21
93 Berserk 12 depth1 : 266.4 41.0 198 21
94 Obsidian 11.0 depth 3 : 249.7 38.5 198 19
95 Seer 2.8.0 depth 2 : 239.3 37.0 198 19
96 Berserk 12 depth3 : 217.7 34.0 198 17
97 Seer 2.8.0 depth 3 : 210.3 33.0 198 17
98 Stockfish 16.1 depth 2 : 195.0 31.0 198 16
99 Caissa 1.17 depth 2 : 139.9 24.5 198 12
100 Caissa 1.17 depth 1 : 0.0 12.5 198 6

White advantage = 22.29
Draw rate (equal opponents) = 50.00 %

[Uri Blass]

I use Ordo, very simple. Put your games in a pgn in the Ordo folder, configure the batch once and you are done.
Ordo:
https://mega.nz/file/PfgkXCqC#nKu7ugHdc ... zvst_MTk_k

Thanks.
Downloaded it.
Not sure what I need to do to configure the batch.

Uri there is a good manuel inside
I try to make it simple
Generel: a minimum of 500 games against 20 opponents is a good begin to get realistic elo ratings. 30 matches between two engines is also a good minimum because there could be 7 games in a row with looses/wins so 10 games could give higher false ratings.



Wilhelm

I would like to have a different rating list that I call deterministic rating list.
The idea is that all engines are deterministic and every pair of engines play exactly 2 games from the same position with reversed colors.

The idea is to get accurate rating for a specific position by having hundrends of opponents for every players but only 2 games between every 2 engines and that people can reproduce the same rating list if they use the same deterministic engines(it means 1 core and no time control(can be fixed nodes or fixed depth but I decided that I prefer fixed nodes).

The idea is also that you can compare different chess positions based on the difference in rating between chess engines with the idea that more complex positions tend to have a bigger difference in rating between engines.

[Archimedes]

Some time ago, I also created a tool or batch file like this. Simply unpack the archive into a directory of your choice. Then copy all PGN files into the input directory and start the batch processing (all PGN files are processed one after the other). The result is a BayesElo, ELOstat and Ordo rating list for each PGN file. If desired, the batch file can also create a move average list. For further information, simply read the readme file.

PGNStat 2023.12.12:
https://app.box.com/s/dn54238h3knuef7z9v914xvp3yof3ud5