Full 1000 games:
lc0 59x 30s+1s - lc0 59x 15s+0.5s : 571.0/1000 158-16-826
Elo: +49 -49
Ordo:
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 59x 30s+1 : 2325.1 7.5 571.0 1000 57
2 lc0 59x 15s+0.5s : 2274.9 7.5 429.0 1000 43
Difference: 50.2
Moderators: hgm, Rebel, chrisw
Full 1000 games:
...and still no games available for checking for weird things
The last match is already for the trash bin, it contains 700 entries with 'forfeits on time'.
Code: Select all
Ra2xf2+) 0.00/37 0} 97. ... {0-1 White forfeits on time} 97. Kxf5 {(Kf4xf5
Ra2xf2+) 0.00/38 0} Rxf2+ {(Ra2xf2+) 0.00/64 0} 98. ... {0-1 White forfeits
on time} 99. Kg6 {(Kf5-g6 Kh3xg3 Kg6xh5 Rf2-f8 Kh5-g5 Rf8-g8+ Kg5-f6 Rg8-h8
Kf6-e7 Rh8xh4 Rb4xh4 Kg3xh4 Ke7-e6 Kh4-g5 Ke6-e5 Kg5-g4) 0.00/13 0} Ra2
{(Rf2-a2) 0.00/11 0} 100. ... {0-1 White forfeits on time} 100. Rb5
{(Rb4-b5 Kh3xg3 Kg6-g5 Ra2-a4 Rb5-b3+ Kg3-g2 Kg5xh5 Ra4-a5+ Kh5-g4 Ra5-a4+)
0.00/14 0} Kxg3 {(Kh3xg3 Kg6-g5 Ra2-a3 Rb5-c5 Kg3-f3 Rc5-d5 Kf3-g3 Rd5-b5
Kg3-f3 Kg5xh5 Kf3-f4 Rb5-b4+ Kf4-g3 Rb4-g4+ Kg3-h3 Rg4-g1 Ra3-a5+ Kh5-g6
Kh3xh4) 0.00/13 0} 101. ... {0-1 White forfeits on time} 102. Kg5 {(Kg6-g5
Ra2-a4 Rb5-b3+ Kg3-g2 Kg5xh5 Ra4-a5+ Kh5-g6 Ra5-a6+ Kg6-g5 Ra6-a5+) 0.00/15
0} Ra3 {(Ra2-a3 Kg5xh5 Ra3-a4 Rb5-g5+ Kg3-h3 Rg5-d5 Ra4xh4+ Kh5-g5 Rh4-g4+
Kg5-f6 Rg4-a4 Kf6-e5 Kh3-g3 Ke5-f6) 0.00/12 0} 103. ... {0-1 White forfeits
on time} 103. Rc5 {(Rb5-c5 Kg3-f3 Kg5xh5 Kf3-f4 Rc5-g5 Ra3-a7 Kh5-h6 Ra7-a8
Rg5-g1 Ra8-h8+ Kh6-g7 Rh8xh4 Rg1-e1 Rh4-h2 Re1-e8 Rh2-a2 Kg7-f7 Ra2-g2)
0.00/16 0} Kf3 {(Kg3-f3 Kg5xh5 Kf3-f4 Rc5-c6 Kf4-g3 Kh5-g5 Ra3-a5+ Kg5-f6
Kg3xh4 Rc6-c3 Ra5-a7 Rc3-c8 Ra7-a5) 0.00/13 0} 104. ... {0-1 White forfeits
on time} 105. Kxh5 {(Kg5xh5) 0.00/19 0 Arena Adjudication (Tablebases)}
1/2-1/2
A large book has its minuses. Since both sides are played, it shouldn't be too big of a deal for getting the right Elo (Ordo) difference.
No, this is not the case with Arena. I can set up a 1s/0 tournament with a 20 GB cache and it works fine, whether the engines are loaded beforehand or not.Guenther wrote: ↑Mon Feb 17, 2020 9:45 amLast not least, ofc Alyan was correct that using 20GB hash per program added extreme noise and errors too, because loading 20GB hash in a 10s or 5s game might already use most of the basetime for loading and slowing things down. (even 64 or 128MB would be enough here).
Yeah, I will try Cute Chess CLI. Any large book recommendations?
Thanks. This one match should be discarded anyway because it's the one with mistaken time controls. I see a total of 3 other forfeits on time in all other matches.
This is wrong and will defeat the whole purpose of the test. Bad openings cannot be cured by playing them for both sides.mmt wrote: ↑Mon Feb 17, 2020 5:08 pm Thanks for looking! Definitely have to clean look at these time forfeit games. I can create cleaned-up versions of the pgn files without PV.
A large book has its minuses. Since both sides are played, it shouldn't be too big of a deal for getting the right Elo (Ordo) difference.
Well, in one file there were seven unterminated games, which Arena replayed automatically for you, in some of them the second program may bemmt wrote: ↑Mon Feb 17, 2020 5:08 pmNo, this is not the case with Arena. I can set up a 1s/0 tournament with a 20 GB cache and it works fine, whether the engines are loaded beforehand or not.Guenther wrote: ↑Mon Feb 17, 2020 9:45 amLast not least, ofc Alyan was correct that using 20GB hash per program added extreme noise and errors too, because loading 20GB hash in a 10s or 5s game might already use most of the basetime for loading and slowing things down. (even 64 or 128MB would be enough here).
No, see above, I would use a 6-12 (at max!) plies opening file.