Vice CCRL rating

[amanjpro]

That is actually a very big issue. Let's say Engine A performs terribly against Engine B, and Engine B performs terribly against Engine C, and no engine has issues with Engine D.

If the CCRL tester to choose Engine B for testing Engine A, but Engine D was chosen to test Engine B. Engine A would appear rather weaker than Engine B, eventhough in reality they might be exactly the same.

It doesn't matter if the gauntlet contains 10 engines or not, a score of 70-0-10 against Engine B is going to affect the final rating of Engine A badly anyways

We are aware of the problem. That is the reason we select about 30 opponents and play 32 games against each opponent when testing a new engine. Anomalies even out.

I'm sure you guys are much better suited than me for this task. What I really meant is that the diff I talked about cannot be attributed to the engine matching alone.

Both Guenther's and Lithanders explanation makes more sense that I simply have gotten the source code from a modified copy of vice, which rates lower than what CCRL has used to test.

I got the link from Weiss repo, which is not necessarily an official source for getting Vice source code

[mvanthoor]

That is actually a very big issue. Let's say Engine A performs terribly against Engine B, and Engine B performs terribly against Engine C, and no engine has issues with Engine D.

If the CCRL tester to choose Engine B for testing Engine A, but Engine D was chosen to test Engine B. Engine A would appear rather weaker than Engine B, eventhough in reality they might be exactly the same.

It doesn't matter if the gauntlet contains 10 engines or not, a score of 70-0-10 against Engine B is going to affect the final rating of Engine A badly anyways

That's just the way it is. Sometimes one engine has a style that just destroys some other engines in its strength range, but gets destroyed itself by other engines. That's the way how rating lists work, and the reason why engines should play a large number of games against many different engines.

There's an easy way to make an engine play bad. Just give Rustic an opening book that _completely contradicts_ its PST's, and it'll first start shuffling pieces to get them where they are "supposed" to be. That costs time, and if a different engine doesn't shuffle, it'll get a lead in development, and with that, the better winning chances.

When Rustic Alpha 2 was first tested in the CCRL-list, it was rated 1780. It has had another run, and was then used in the gauntlets of some other engines, and it crept up to 1815, which is about 25 points short of my own test. (Which used different engines, different time controls, and different opening books, so that's no surprise.) So RA2 gained 35 ratings points by playing more/different engines. (And RA1 lost 20 rating points as soon as it got destroyed by TSCP; as expected.)

[amanjpro]

That is actually a very big issue. Let's say Engine A performs terribly against Engine B, and Engine B performs terribly against Engine C, and no engine has issues with Engine D.

If the CCRL tester to choose Engine B for testing Engine A, but Engine D was chosen to test Engine B. Engine A would appear rather weaker than Engine B, eventhough in reality they might be exactly the same.

It doesn't matter if the gauntlet contains 10 engines or not, a score of 70-0-10 against Engine B is going to affect the final rating of Engine A badly anyways

That's just the way it is. Sometimes one engine has a style that just destroys some other engines in its strength range, but gets destroyed itself by other engines. That's the way how rating lists work, and the reason why engines should play a large number of games against many different engines.

There's an easy way to make an engine play bad. Just give Rustic an opening book that _completely contradicts_ its PST's, and it'll first start shuffling pieces to get them where they are "supposed" to be. That costs time, and if a different engine doesn't shuffle, it'll get a lead in development, and with that, the better winning chances.

When Rustic Alpha 2 was first tested in the CCRL-list, it was rated 1780. It has had another run, and was then used in the gauntlets of some other engines, and it crept up to 1815, which is about 25 points short of my own test. (Which used different engines, different time controls, and different opening books, so that's no surprise.) So RA2 gained 35 ratings points by playing more/different engines. (And RA1 lost 20 rating points as soon as it got destroyed by TSCP; as expected.)

I'm not arguing with that. I'm simply arguing that incompatibility alone between two engines wont result in massive scores like 70-10-20

That is why I believe what Guenther and Lithander said makes more sense

[lithander]

That is why I believe what Guenther and Lithander said makes more sense

I let the two versions (self-built Zahak and Vice1.1 from the google drive linked in this thread) battle it out for a little bit on 5s + 0.5s and found Zahak to be ~30 ELO stronger where the CCRL rating would predict it to be ~30 ELO weaker than Vice. But that kind of variance is neither uncommon nor concerning, imo.

Code: Select all

Score of zahak100 vs Vice11: 194 - 154 - 69  [0.548] 417
...      zahak100 playing White: 103 - 75 - 31  [0.567] 209
...      zahak100 playing Black: 91 - 79 - 38  [0.529] 208
...      White vs Black: 182 - 166 - 69  [0.519] 417
Elo difference: 33.4 +/- 30.6, LOS: 98.4 %, DrawRatio: 16.5 %

Case closed I would say!

By the way... I was surprised to see Vice play an illegal move in one game. ("Black makes an illegal move: c3d5")
[pgn][Event "?"]
[Site "?"]
[Date "2021.04.29"]
[Round "94"]
[White "zahak100"]
[Black "Vice11"]
[Result "1-0"]
[ECO "A04"]
[GameDuration "00:04:01"]
[GameEndTime "2021-04-29T22:42:39.757 Mitteleuropäische Sommerzeit"]
[GameStartTime "2021-04-29T22:38:38.186 Mitteleuropäische Sommerzeit"]
[Opening "Reti Opening"]
[PlyCount "477"]
[Termination "illegal move"]
[TimeControl "5+0.5"]

1. Nf3 {book} c6 {book} 2. c4 {book} d5 {book} 3. b3 {book} Nf6 {book}
4. g3 {book} Bg4 {book} 5. Bg2 {book} Nbd7 {book} 6. Bb2 {book} e6 {book}
7. d3 {book} Bb4+ {+0.60/6 0.64s} 8. Nbd2 {-1.24/8 0.63s} O-O {+0.55/7 0.63s}
9. O-O {-1.27/8 0.63s} c5 {+0.48/6 0.62s} 10. e4 {-0.92/9 0.62s}
Bxd2 {+1.03/8 0.62s} 11. Qxd2 {-0.58/10 0.62s} dxe4 {+0.92/8 0.61s}
12. dxe4 {-0.19/11 0.62s} Nxe4 {+0.87/8 0.61s} 13. Qe3 {+0.01/10 0.61s}
Nef6 {+0.82/8 0.61s} 14. h3 {0.00/10 0.61s} Bxf3 {+0.90/8 0.61s}
15. Bxf3 {-0.22/11 0.60s} Qb6 {+0.90/8 0.59s} 16. g4 {-0.28/9 0.60s}
h6 {+0.92/8 0.60s} 17. h4 {+0.36/10 0.60s} Rad8 {+0.92/7 0.60s}
18. g5 {+0.35/9 0.59s} hxg5 {+0.85/9 0.59s} 19. hxg5 {+0.42/10 0.59s}
Nh7 {+0.80/8 0.58s} 20. Rad1 {+0.42/9 0.59s} f6 {+0.82/7 0.58s}
21. gxf6 {+0.50/9 0.59s} Nhxf6 {+0.72/8 0.58s} 22. Rfe1 {+0.51/9 0.58s}
Rde8 {+0.75/7 0.59s} 23. Qg5 {+0.53/9 0.58s} Qa5 {+0.72/6 0.57s}
24. Bxb7 {+0.98/9 0.58s} e5 {-0.28/6 0.58s} 25. Bc6 {+1.53/9 0.57s}
Re6 {-0.38/7 0.56s} 26. Qg2 {+1.60/9 0.57s} Rf7 {-0.25/7 0.57s}
27. f4 {+2.05/9 0.57s} Qb6 {-0.25/7 0.57s} 28. fxe5 {+1.75/9 0.57s}
Qxc6 {-0.35/8 0.57s} 29. Qxc6 {+2.61/12 0.56s} Rxc6 {-1.25/12 0.57s}
30. e6 {+2.37/12 0.56s} Re7 {-1.30/12 0.56s} 31. exd7 {+2.26/12 0.56s}
Rxe1+ {-1.35/11 0.56s} 32. Rxe1 {+2.33/11 0.56s} Nxd7 {-1.30/12 0.55s}
33. Re7 {+2.47/11 0.56s} Rd6 {-1.30/10 0.55s} 34. Rxg7+ {+2.11/10 0.56s}
Kf8 {-1.20/10 0.55s} 35. Kf1 {+2.44/11 0.55s} Ke8 {-1.20/10 0.55s}
36. Bc3 {+2.59/10 0.55s} Rd3 {-1.25/9 0.56s} 37. Be1 {+2.46/10 0.55s}
Rd1 {-1.30/9 0.56s} 38. Rg6 {+2.80/10 0.55s} Nb6 {-1.35/9 0.54s}
39. Rh6 {+2.76/10 0.55s} Kd8 {-1.40/9 0.55s} 40. Ke2 {+2.79/10 0.55s}
Ra1 {-1.40/9 0.55s} 41. a4 {+2.76/9 0.54s} Rb1 {-1.15/9 0.53s}
42. Rh3 {+2.84/10 0.54s} Rb2+ {-1.05/8 0.53s} 43. Bd2 {+2.79/9 0.54s}
Nd7 {-1.10/10 0.53s} 44. a5 {+2.63/9 0.54s} Ke7 {-1.10/9 0.54s}
45. Kd1 {+2.68/10 0.54s} Nf6 {-1.15/8 0.54s} 46. Re3+ {+3.57/10 0.54s}
Kf7 {-2.35/11 0.54s} 47. Rf3 {+3.91/11 0.54s} Rb1+ {-2.40/12 0.54s}
48. Kc2 {+4.01/11 0.54s} Rg1 {-2.45/13 0.54s} 49. Be3 {+3.92/11 0.53s}
Rg2+ {-2.45/11 0.54s} 50. Kc1 {+4.22/11 0.53s} a6 {-2.15/11 0.54s}
51. Bxc5 {+4.00/11 0.53s} Ke6 {-2.20/11 0.53s} 52. Ba7 {+3.95/9 0.53s}
Ne4 {-2.15/8 0.53s} 53. Bd4 {+3.88/9 0.53s} Kd6 {-2.20/8 0.53s}
54. Rh3 {+4.02/10 0.53s} Nc5 {-2.20/10 0.53s} 55. b4 {+4.04/10 0.53s}
Ne6 {-2.30/10 0.52s} 56. Bc3 {+3.96/9 0.53s} Kc6 {-2.30/9 0.52s}
57. Rh8 {+4.26/9 0.53s} Rg4 {-2.30/9 0.52s} 58. Rh6 {+4.26/10 0.53s}
Kd6 {-2.25/10 0.52s} 59. c5+ {+4.06/10 0.53s} Kd5 {-2.30/9 0.52s}
60. Kc2 {+4.23/10 0.52s} Rg3 {-2.40/9 0.52s} 61. Rh5+ {+4.11/9 0.52s}
Rg5 {-2.35/9 0.52s} 62. Rxg5+ {+4.60/10 0.52s} Nxg5 {-2.30/13 0.52s}
63. Kd3 {+4.32/11 0.52s} Ne4 {-2.30/14 0.52s} 64. Be1 {+4.27/12 0.52s}
Ng5 {-2.30/15 0.52s} 65. Bc3 {+4.30/12 0.52s} Ne4 {0.00/25 0.52s}
66. Bd4 {+4.30/13 0.52s} Ng5 {-2.30/13 0.52s} 67. Bg7 {+4.30/12 0.52s}
Ne6 {-2.25/16 0.52s} 68. Bf6 {+4.28/13 0.52s} Nf4+ {-2.25/15 0.53s}
69. Kc3 {+4.31/12 0.52s} Ne2+ {-2.25/15 0.53s} 70. Kd2 {+4.26/12 0.52s}
Nf4 {-2.25/14 0.51s} 71. Kc3 {+4.27/12 0.52s} Ne2+ {0.00/17 0.51s}
72. Kc2 {+4.28/13 0.52s} Nf4 {-2.25/15 0.51s} 73. Kb3 {+4.27/12 0.52s}
Nd3 {-2.25/14 0.51s} 74. Kc3 {+4.29/12 0.51s} Nf4 {-2.25/14 0.51s}
75. Bg7 {+4.29/13 0.51s} Ne6 {-2.25/16 0.51s} 76. Bf6 {+4.32/13 0.51s}
Nf4 {0.00/22 0.51s} 77. Bg7 {+4.27/13 0.51s} Ne6 {0.00/19 0.51s}
78. Bh6 {+4.27/14 0.51s} Nd4 {-2.25/14 0.52s} 79. Be3 {+4.27/13 0.51s}
Nb5+ {-2.25/15 0.52s} 80. Kd3 {+4.30/12 0.51s} Nc7 {-2.25/16 0.50s}
81. Bg5 {+4.32/12 0.51s} Ne6 {-2.25/17 0.50s} 82. Bd2 {+4.27/13 0.51s}
Nd4 {-2.25/15 0.52s} 83. Bf4 {+4.26/13 0.51s} Nc6 {-2.25/16 0.50s}
84. Kc3 {+4.29/13 0.51s} Nd4 {-2.25/16 0.50s} 85. Bd2 {+4.28/13 0.51s}
Nb5+ {-2.00/15 0.50s} 86. Kd3 {+4.27/13 0.51s} Nd4 {0.00/32 0.50s}
87. Bf4 {+4.26/13 0.51s} Nc6 {0.00/19 0.50s} 88. Kc3 {+4.28/14 0.51s}
Nd4 {0.00/19 0.50s} 89. Be3 {+4.25/13 0.51s} Nb5+ {0.00/19 0.51s}
90. Kd3 {+4.27/13 0.51s} Nc7 {0.00/27 0.50s} 91. Bd4 {+4.26/12 0.51s}
Ne6 {-2.25/16 0.50s} 92. Bf2 {+4.27/12 0.51s} Nf4+ {-2.25/15 0.50s}
93. Kc3 {+4.28/12 0.51s} Ne2+ {-2.25/14 0.51s} 94. Kd2 {+4.22/12 0.51s}
Nf4 {-2.25/15 0.50s} 95. Bg3 {+4.25/13 0.51s} Ne6 {-2.25/14 0.51s}
96. Kc3 {+4.25/13 0.51s} Nd4 {-2.25/16 0.50s} 97. Bh4 {+4.24/12 0.51s}
Ne2+ {-2.25/13 0.51s} 98. Kd2 {+4.21/12 0.51s} Nf4 {-2.15/14 0.51s}
99. Bg3 {+4.25/13 0.51s} Ne6 {0.00/19 0.51s} 100. Kd3 {+4.21/13 0.51s}
Nd4 {-2.25/15 0.51s} 101. Bh4 {+4.21/13 0.51s} Ne6 {-2.25/16 0.51s}
102. Bg3 {+4.22/13 0.51s} Nd4 {0.00/22 0.51s} 103. Bh4 {+4.25/14 0.51s}
Ne6 {0.00/17 0.51s} 104. Bf2 {+4.24/13 0.51s} Nf4+ {0.00/23 0.51s}
105. Kc3 {+4.27/12 0.51s} Ne2+ {0.00/20 0.51s} 106. Kc2 {+4.29/12 0.51s}
Nf4 {-2.40/14 0.51s} 107. Be3 {+4.22/13 0.51s} Ne6 {-2.00/14 0.51s}
108. Bd2 {+3.32/11 0.50s} Nd4+ {-1.00/13 0.51s} 109. Kd3 {+2.90/11 0.50s}
Nc6 {-0.90/13 0.51s} 110. Kc3 {+2.59/12 0.50s} Na7 {-0.85/14 0.51s}
111. c6 {+2.60/13 0.50s} Nxc6 {-0.90/15 0.51s} 112. Bh6 {+2.56/13 0.51s}
Ke4 {-0.90/14 0.51s} 113. Bf8 {+2.56/12 0.50s} Kd5 {-0.90/16 0.51s}
114. Bg7 {+2.64/13 0.50s} Na7 {-0.90/17 0.51s} 115. Bh6 {+2.60/13 0.50s}
Nb5+ {-0.90/15 0.51s} 116. Kd3 {+2.60/13 0.50s} Nd4 {-0.90/15 0.51s}
117. Bd2 {+2.60/13 0.50s} Nc6 {-0.90/16 0.51s} 118. Bc3 {+2.59/14 0.50s}
Nd8 {-0.85/15 0.51s} 119. Bd2 {+2.53/13 0.50s} Nc6 {0.00/23 0.51s}
120. Bc3 {+2.59/14 0.50s} Nd8 {0.00/22 0.51s} 121. Bf6 {+2.59/14 0.50s}
Ne6 {-0.90/15 0.51s} 122. Kc3 {+2.59/13 0.50s} Nc7 {-0.90/15 0.51s}
123. Kd3 {+2.59/13 0.50s} Ne6 {0.00/17 0.51s} 124. Kc3 {+2.58/13 0.50s}
Nc7 {0.00/22 0.51s} 125. Be7 {+2.59/13 0.50s} Ne6 {-0.90/14 0.51s}
126. Bh4 {+2.56/13 0.50s} Nd4 {-0.90/14 0.51s} 127. Bg5 {+2.59/12 0.50s}
Ne2+ {-0.90/14 0.51s} 128. Kd3 {+2.58/13 0.50s} Nd4 {-0.90/17 0.51s}
129. Bh6 {+2.57/13 0.50s} Nc6 {0.00/16 0.51s} 130. Kc3 {+2.58/13 0.50s}
Ke4 {0.00/20 0.51s} 131. Bg5 {+2.56/13 0.50s} Kd5 {-0.90/17 0.51s}
132. Bf6 {+2.59/14 0.50s} Na7 {-0.90/17 0.51s} 133. Bh4 {+2.58/13 0.50s}
Nb5+ {-0.90/16 0.51s} 134. Kd3 {+2.59/13 0.50s} Nd4 {-0.90/17 0.49s}
135. Bf2 {+2.58/14 0.50s} Nc6 {-0.90/17 0.51s} 136. Kc3 {+2.59/14 0.50s}
Nb8 {-0.90/16 0.49s} 137. Bd4 {+2.59/14 0.50s} Nc6 {-0.90/15 0.51s}
138. Bf2 {+2.59/14 0.50s} Nb8 {0.00/20 0.49s} 139. Bd4 {+2.58/14 0.50s}
Nc6 {0.00/26 0.49s} 140. Be3 {+2.59/14 0.50s} Ne5 {-0.90/14 0.51s}
141. Bg1 {+2.59/13 0.50s} Nc4 {-0.90/16 0.49s} 142. Bd4 {+2.59/13 0.50s}
Nd6 {-0.90/15 0.51s} 143. Kd3 {+2.66/12 0.50s} Nb5 {-0.90/16 0.49s}
144. Bc5 {+2.60/13 0.50s} Na3 {-0.90/16 0.51s} 145. Bd4 {+2.62/13 0.50s}
Nb5 {0.00/23 0.49s} 146. Bf6 {+2.60/13 0.50s} Na7 {-0.85/14 0.49s}
147. Bg5 {+2.58/13 0.50s} Nc6 {0.00/15 0.51s} 148. Kc3 {+2.62/13 0.50s}
Nd4 {-0.90/15 0.49s} 149. Be3 {+2.60/13 0.50s} Nb5+ {-0.90/14 0.51s}
150. Kd3 {+2.59/13 0.50s} Nd6 {-0.90/15 0.49s} 151. Bb6 {+2.60/12 0.50s}
Nc4 {-0.90/17 0.49s} 152. Bc7 {+2.61/13 0.50s} Nb2+ {-0.90/14 0.51s}
153. Kc3 {+2.59/12 0.50s} Nc4 {-0.90/14 0.49s} 154. Kd3 {+2.62/13 0.50s}
Nb2+ {0.00/21 0.51s} 155. Kc3 {+2.44/12 0.50s} Nc4 {0.00/19 0.49s}
156. Bd8 {+2.43/13 0.50s} Na3 {-0.90/14 0.49s} 157. Bb6 {+2.17/11 0.50s}
Ke4 {0.00/17 0.51s} 158. b5 {+1.30/12 0.50s} Nxb5+ {0.00/16 0.49s}
159. Kc4 {+1.32/12 0.50s} Ke5 {0.00/16 0.51s} 160. Kc5 {+1.42/12 0.50s}
Nc3 {0.00/17 0.49s} 161. Ba7 {+1.44/11 0.50s} Nd5 {0.00/17 0.49s}
162. Kc6 {+1.44/13 0.50s} Nb4+ {0.00/17 0.51s} 163. Kb7 {+1.44/12 0.50s}
Kd5 {0.00/18 0.49s} 164. Bf2 {+0.98/13 0.50s} Kc4 {0.00/17 0.51s}
165. Kb6 {+1.58/13 0.50s} Kd5 {0.00/18 0.49s} 166. Be1 {+1.71/13 0.50s}
Nd3 {0.00/15 0.49s} 167. Bc3 {+1.71/13 0.50s} Nc5 {0.00/17 0.51s}
168. Be1 {+1.71/13 0.50s} Kc4 {0.00/16 0.49s} 169. Bg3 {+1.77/12 0.50s}
Ne4 {0.00/16 0.49s} 170. Bf4 {+1.77/13 0.50s} Nc5 {0.00/17 0.51s}
171. Bd6 {+1.53/13 0.50s} Ne4 {0.00/17 0.49s} 172. Bf4 {+1.53/14 0.50s}
Nc5 {0.00/19 0.51s} 173. Be3 {+1.45/14 0.50s} Nb3 {0.00/17 0.49s}
174. Bg5 {+1.46/13 0.50s} Nc5 {0.00/19 0.49s} 175. Be7 {+1.61/13 0.50s}
Nb3 {0.00/17 0.51s} 176. Bh4 {+1.39/13 0.50s} Nc5 {0.00/19 0.49s}
177. Be7 {+1.42/13 0.50s} Nb3 {0.00/18 0.51s} 178. Bd6 {+1.19/13 0.50s}
Kd5 {0.00/19 0.49s} 179. Bf8 {+1.35/13 0.50s} Ke6 {0.00/18 0.49s}
180. Bg7 {+1.39/13 0.50s} Kf5 {0.00/17 0.51s} 181. Bc3 {+1.42/12 0.50s}
Kf4 {0.00/18 0.49s} 182. Bg7 {+1.29/12 0.50s} Kf5 {0.00/25 0.49s}
183. Bh6 {+1.42/13 0.50s} Ke6 {0.00/18 0.51s} 184. Bf4 {+1.38/13 0.50s}
Kf5 {0.00/21 0.49s} 185. Be3 {+1.44/13 0.50s} Ke4 {0.00/19 0.51s}
186. Bh6 {+1.35/13 0.50s} Kd5 {0.00/18 0.49s} 187. Bg5 {+1.42/13 0.50s}
Nc5 {0.00/18 0.49s} 188. Bh4 {+1.27/13 0.50s} Kc4 {0.00/17 0.51s}
189. Bg3 {+1.36/13 0.50s} Ne4 {0.00/18 0.49s} 190. Be1 {+1.35/14 0.50s}
Nc5 {0.00/25 0.49s} 191. Bd2 {+1.35/14 0.50s} Kd5 {0.00/17 0.51s}
192. Be3 {+1.35/14 0.50s} Nb3 {0.00/18 0.49s} 193. Bf4 {+1.20/13 0.50s}
Ke4 {0.00/22 0.49s} 194. Bg3 {+1.37/13 0.50s} Kf3 {0.00/18 0.51s}
195. Be1 {+1.42/13 0.50s} Ke2 {0.00/19 0.49s} 196. Bc3 {+1.29/13 0.50s}
Kd3 {0.00/18 0.49s} 197. Be1 {+1.14/13 0.50s} Ke2 {0.00/24 0.51s}
198. Bg3 {+1.22/13 0.50s} Kd3 {0.00/18 0.49s} 199. Bf2 {+1.22/13 0.50s}
Kc3 {0.00/18 0.51s} 200. Bh4 {+1.25/13 0.50s} Kd3 {0.00/18 0.49s}
201. Bg5 {+1.20/13 0.50s} Kd4 {0.00/18 0.49s} 202. Bf4 {+1.33/13 0.50s}
Ke4 {0.00/24 0.51s} 203. Bg5 {+1.08/13 0.50s} Kf5 {0.00/21 0.49s}
204. Be3 {+1.01/12 0.50s} Ke5 {0.00/18 0.51s} 205. Bc5 {+0.83/12 0.50s}
Ke4 {0.00/18 0.49s} 206. Bg1 {+0.77/12 0.50s} Kf3 {0.00/18 0.51s}
207. Kxa6 {+0.72/12 0.50s} Nxa5 {0.00/19 0.49s} 208. Bd4 {+0.75/13 0.50s}
Nc4 {0.00/18 0.49s} 209. Bc3 {+0.75/11 0.50s} Ke4 {0.00/17 0.51s}
210. Bf6 {+0.71/13 0.50s} Kf3 {0.00/18 0.49s} 211. Bc3 {+0.71/13 0.50s}
Ke4 {0.00/33 0.49s} 212. Kb7 {+0.71/13 0.50s} Kd3 {0.00/19 0.51s}
213. Bf6 {+0.72/13 0.50s} Ne3 {0.00/16 0.49s} 214. Kc6 {+0.72/12 0.50s}
Ke4 {0.00/18 0.51s} 215. Kc5 {+0.72/13 0.50s} Nf1 {0.00/17 0.49s}
216. Kd6 {+0.71/13 0.50s} Kf4 {0.00/17 0.49s} 217. Bd4 {+0.73/11 0.50s}
Kf3 {0.00/18 0.51s} 218. Ke7 {+0.74/12 0.50s} Ke4 {0.00/20 0.49s}
219. Bc5 {+0.73/13 0.50s} Kd3 {0.00/18 0.51s} 220. Kf7 {+0.73/13 0.50s}
Ng3 {0.00/18 0.49s} 221. Bd6 {+0.73/12 0.50s} Ke4 {0.00/19 0.49s}
222. Bc7 {+0.72/13 0.50s} Kd4 {0.00/19 0.51s} 223. Ke6 {+0.73/12 0.50s}
Nh1 {0.00/18 0.49s} 224. Be5+ {+0.72/12 0.50s} Kc4 {0.00/17 0.51s}
225. Kd7 {+0.74/12 0.50s} Ng3 {0.00/18 0.49s} 226. Kc7 {+0.72/12 0.50s}
Nh1 {0.00/19 0.51s} 227. Bf6 {+0.72/13 0.50s} Ng3 {0.00/18 0.49s}
228. Kc6 {+0.72/13 0.50s} Nf5 {0.00/17 0.49s} 229. Be5 {+0.72/13 0.50s}
Kd3 {0.00/17 0.51s} 230. Kd5 {+0.72/12 0.50s} Ne3+ {0.00/17 0.49s}
231. Ke6 {+0.72/11 0.50s} Nd5 {0.00/18 0.51s} 232. Kf7 {+0.72/12 0.50s}
Ke3 {0.00/17 0.49s} 233. Bd6 {+0.72/12 0.50s} Kd2 {0.00/17 0.49s}
234. Kg7 {+0.71/12 0.50s} Kd3 {0.00/18 0.51s} 235. Kf7 {+0.71/12 0.50s}
Ke4 {0.00/18 0.49s} 236. Kg6 {+0.72/13 0.50s} Nc3 {0.00/18 0.51s}
237. Kg7 {+0.72/13 0.50s} Kf3 {0.00/18 0.49s} 238. Be5 {+0.71/13 0.50s}
Nd5 {0.00/19 0.51s} 239. Kg6 {+0.72/13 0.50s, Black makes an illegal move: c3d5}
1-0[/pgn]

[amanjpro]

That is why I believe what Guenther and Lithander said makes more sense

I let the two versions (self-built Zahak and Vice1.1 from the google drive linked in this thread) battle it out for a little bit on 5s + 0.5s and found Zahak to be ~30 ELO stronger where the CCRL rating would predict is to be ~30 ELO weaker than Vice. But that kind of variance is neither uncommon nor concerning, imo.

Code: Select all

Score of zahak100 vs Vice11: 194 - 154 - 69  [0.548] 417
...      zahak100 playing White: 103 - 75 - 31  [0.567] 209
...      zahak100 playing Black: 91 - 79 - 38  [0.529] 208
...      White vs Black: 182 - 166 - 69  [0.519] 417
Elo difference: 33.4 +/- 30.6, LOS: 98.4 %, DrawRatio: 16.5 %

Case closed I would say!

By the way... I was surprised to see Vice play an illegal move in one game. ("Black makes an illegal move: c3d5")
[pgn][Event "?"]
[Site "?"]
[Date "2021.04.29"]
[Round "94"]
[White "zahak100"]
[Black "Vice11"]
[Result "1-0"]
[ECO "A04"]
[GameDuration "00:04:01"]
[GameEndTime "2021-04-29T22:42:39.757 Mitteleuropäische Sommerzeit"]
[GameStartTime "2021-04-29T22:38:38.186 Mitteleuropäische Sommerzeit"]
[Opening "Reti Opening"]
[PlyCount "477"]
[Termination "illegal move"]
[TimeControl "5+0.5"]

1. Nf3 {book} c6 {book} 2. c4 {book} d5 {book} 3. b3 {book} Nf6 {book}
4. g3 {book} Bg4 {book} 5. Bg2 {book} Nbd7 {book} 6. Bb2 {book} e6 {book}
7. d3 {book} Bb4+ {+0.60/6 0.64s} 8. Nbd2 {-1.24/8 0.63s} O-O {+0.55/7 0.63s}
9. O-O {-1.27/8 0.63s} c5 {+0.48/6 0.62s} 10. e4 {-0.92/9 0.62s}
Bxd2 {+1.03/8 0.62s} 11. Qxd2 {-0.58/10 0.62s} dxe4 {+0.92/8 0.61s}
12. dxe4 {-0.19/11 0.62s} Nxe4 {+0.87/8 0.61s} 13. Qe3 {+0.01/10 0.61s}
Nef6 {+0.82/8 0.61s} 14. h3 {0.00/10 0.61s} Bxf3 {+0.90/8 0.61s}
15. Bxf3 {-0.22/11 0.60s} Qb6 {+0.90/8 0.59s} 16. g4 {-0.28/9 0.60s}
h6 {+0.92/8 0.60s} 17. h4 {+0.36/10 0.60s} Rad8 {+0.92/7 0.60s}
18. g5 {+0.35/9 0.59s} hxg5 {+0.85/9 0.59s} 19. hxg5 {+0.42/10 0.59s}
Nh7 {+0.80/8 0.58s} 20. Rad1 {+0.42/9 0.59s} f6 {+0.82/7 0.58s}
21. gxf6 {+0.50/9 0.59s} Nhxf6 {+0.72/8 0.58s} 22. Rfe1 {+0.51/9 0.58s}
Rde8 {+0.75/7 0.59s} 23. Qg5 {+0.53/9 0.58s} Qa5 {+0.72/6 0.57s}
24. Bxb7 {+0.98/9 0.58s} e5 {-0.28/6 0.58s} 25. Bc6 {+1.53/9 0.57s}
Re6 {-0.38/7 0.56s} 26. Qg2 {+1.60/9 0.57s} Rf7 {-0.25/7 0.57s}
27. f4 {+2.05/9 0.57s} Qb6 {-0.25/7 0.57s} 28. fxe5 {+1.75/9 0.57s}
Qxc6 {-0.35/8 0.57s} 29. Qxc6 {+2.61/12 0.56s} Rxc6 {-1.25/12 0.57s}
30. e6 {+2.37/12 0.56s} Re7 {-1.30/12 0.56s} 31. exd7 {+2.26/12 0.56s}
Rxe1+ {-1.35/11 0.56s} 32. Rxe1 {+2.33/11 0.56s} Nxd7 {-1.30/12 0.55s}
33. Re7 {+2.47/11 0.56s} Rd6 {-1.30/10 0.55s} 34. Rxg7+ {+2.11/10 0.56s}
Kf8 {-1.20/10 0.55s} 35. Kf1 {+2.44/11 0.55s} Ke8 {-1.20/10 0.55s}
36. Bc3 {+2.59/10 0.55s} Rd3 {-1.25/9 0.56s} 37. Be1 {+2.46/10 0.55s}
Rd1 {-1.30/9 0.56s} 38. Rg6 {+2.80/10 0.55s} Nb6 {-1.35/9 0.54s}
39. Rh6 {+2.76/10 0.55s} Kd8 {-1.40/9 0.55s} 40. Ke2 {+2.79/10 0.55s}
Ra1 {-1.40/9 0.55s} 41. a4 {+2.76/9 0.54s} Rb1 {-1.15/9 0.53s}
42. Rh3 {+2.84/10 0.54s} Rb2+ {-1.05/8 0.53s} 43. Bd2 {+2.79/9 0.54s}
Nd7 {-1.10/10 0.53s} 44. a5 {+2.63/9 0.54s} Ke7 {-1.10/9 0.54s}
45. Kd1 {+2.68/10 0.54s} Nf6 {-1.15/8 0.54s} 46. Re3+ {+3.57/10 0.54s}
Kf7 {-2.35/11 0.54s} 47. Rf3 {+3.91/11 0.54s} Rb1+ {-2.40/12 0.54s}
48. Kc2 {+4.01/11 0.54s} Rg1 {-2.45/13 0.54s} 49. Be3 {+3.92/11 0.53s}
Rg2+ {-2.45/11 0.54s} 50. Kc1 {+4.22/11 0.53s} a6 {-2.15/11 0.54s}
51. Bxc5 {+4.00/11 0.53s} Ke6 {-2.20/11 0.53s} 52. Ba7 {+3.95/9 0.53s}
Ne4 {-2.15/8 0.53s} 53. Bd4 {+3.88/9 0.53s} Kd6 {-2.20/8 0.53s}
54. Rh3 {+4.02/10 0.53s} Nc5 {-2.20/10 0.53s} 55. b4 {+4.04/10 0.53s}
Ne6 {-2.30/10 0.52s} 56. Bc3 {+3.96/9 0.53s} Kc6 {-2.30/9 0.52s}
57. Rh8 {+4.26/9 0.53s} Rg4 {-2.30/9 0.52s} 58. Rh6 {+4.26/10 0.53s}
Kd6 {-2.25/10 0.52s} 59. c5+ {+4.06/10 0.53s} Kd5 {-2.30/9 0.52s}
60. Kc2 {+4.23/10 0.52s} Rg3 {-2.40/9 0.52s} 61. Rh5+ {+4.11/9 0.52s}
Rg5 {-2.35/9 0.52s} 62. Rxg5+ {+4.60/10 0.52s} Nxg5 {-2.30/13 0.52s}
63. Kd3 {+4.32/11 0.52s} Ne4 {-2.30/14 0.52s} 64. Be1 {+4.27/12 0.52s}
Ng5 {-2.30/15 0.52s} 65. Bc3 {+4.30/12 0.52s} Ne4 {0.00/25 0.52s}
66. Bd4 {+4.30/13 0.52s} Ng5 {-2.30/13 0.52s} 67. Bg7 {+4.30/12 0.52s}
Ne6 {-2.25/16 0.52s} 68. Bf6 {+4.28/13 0.52s} Nf4+ {-2.25/15 0.53s}
69. Kc3 {+4.31/12 0.52s} Ne2+ {-2.25/15 0.53s} 70. Kd2 {+4.26/12 0.52s}
Nf4 {-2.25/14 0.51s} 71. Kc3 {+4.27/12 0.52s} Ne2+ {0.00/17 0.51s}
72. Kc2 {+4.28/13 0.52s} Nf4 {-2.25/15 0.51s} 73. Kb3 {+4.27/12 0.52s}
Nd3 {-2.25/14 0.51s} 74. Kc3 {+4.29/12 0.51s} Nf4 {-2.25/14 0.51s}
75. Bg7 {+4.29/13 0.51s} Ne6 {-2.25/16 0.51s} 76. Bf6 {+4.32/13 0.51s}
Nf4 {0.00/22 0.51s} 77. Bg7 {+4.27/13 0.51s} Ne6 {0.00/19 0.51s}
78. Bh6 {+4.27/14 0.51s} Nd4 {-2.25/14 0.52s} 79. Be3 {+4.27/13 0.51s}
Nb5+ {-2.25/15 0.52s} 80. Kd3 {+4.30/12 0.51s} Nc7 {-2.25/16 0.50s}
81. Bg5 {+4.32/12 0.51s} Ne6 {-2.25/17 0.50s} 82. Bd2 {+4.27/13 0.51s}
Nd4 {-2.25/15 0.52s} 83. Bf4 {+4.26/13 0.51s} Nc6 {-2.25/16 0.50s}
84. Kc3 {+4.29/13 0.51s} Nd4 {-2.25/16 0.50s} 85. Bd2 {+4.28/13 0.51s}
Nb5+ {-2.00/15 0.50s} 86. Kd3 {+4.27/13 0.51s} Nd4 {0.00/32 0.50s}
87. Bf4 {+4.26/13 0.51s} Nc6 {0.00/19 0.50s} 88. Kc3 {+4.28/14 0.51s}
Nd4 {0.00/19 0.50s} 89. Be3 {+4.25/13 0.51s} Nb5+ {0.00/19 0.51s}
90. Kd3 {+4.27/13 0.51s} Nc7 {0.00/27 0.50s} 91. Bd4 {+4.26/12 0.51s}
Ne6 {-2.25/16 0.50s} 92. Bf2 {+4.27/12 0.51s} Nf4+ {-2.25/15 0.50s}
93. Kc3 {+4.28/12 0.51s} Ne2+ {-2.25/14 0.51s} 94. Kd2 {+4.22/12 0.51s}
Nf4 {-2.25/15 0.50s} 95. Bg3 {+4.25/13 0.51s} Ne6 {-2.25/14 0.51s}
96. Kc3 {+4.25/13 0.51s} Nd4 {-2.25/16 0.50s} 97. Bh4 {+4.24/12 0.51s}
Ne2+ {-2.25/13 0.51s} 98. Kd2 {+4.21/12 0.51s} Nf4 {-2.15/14 0.51s}
99. Bg3 {+4.25/13 0.51s} Ne6 {0.00/19 0.51s} 100. Kd3 {+4.21/13 0.51s}
Nd4 {-2.25/15 0.51s} 101. Bh4 {+4.21/13 0.51s} Ne6 {-2.25/16 0.51s}
102. Bg3 {+4.22/13 0.51s} Nd4 {0.00/22 0.51s} 103. Bh4 {+4.25/14 0.51s}
Ne6 {0.00/17 0.51s} 104. Bf2 {+4.24/13 0.51s} Nf4+ {0.00/23 0.51s}
105. Kc3 {+4.27/12 0.51s} Ne2+ {0.00/20 0.51s} 106. Kc2 {+4.29/12 0.51s}
Nf4 {-2.40/14 0.51s} 107. Be3 {+4.22/13 0.51s} Ne6 {-2.00/14 0.51s}
108. Bd2 {+3.32/11 0.50s} Nd4+ {-1.00/13 0.51s} 109. Kd3 {+2.90/11 0.50s}
Nc6 {-0.90/13 0.51s} 110. Kc3 {+2.59/12 0.50s} Na7 {-0.85/14 0.51s}
111. c6 {+2.60/13 0.50s} Nxc6 {-0.90/15 0.51s} 112. Bh6 {+2.56/13 0.51s}
Ke4 {-0.90/14 0.51s} 113. Bf8 {+2.56/12 0.50s} Kd5 {-0.90/16 0.51s}
114. Bg7 {+2.64/13 0.50s} Na7 {-0.90/17 0.51s} 115. Bh6 {+2.60/13 0.50s}
Nb5+ {-0.90/15 0.51s} 116. Kd3 {+2.60/13 0.50s} Nd4 {-0.90/15 0.51s}
117. Bd2 {+2.60/13 0.50s} Nc6 {-0.90/16 0.51s} 118. Bc3 {+2.59/14 0.50s}
Nd8 {-0.85/15 0.51s} 119. Bd2 {+2.53/13 0.50s} Nc6 {0.00/23 0.51s}
120. Bc3 {+2.59/14 0.50s} Nd8 {0.00/22 0.51s} 121. Bf6 {+2.59/14 0.50s}
Ne6 {-0.90/15 0.51s} 122. Kc3 {+2.59/13 0.50s} Nc7 {-0.90/15 0.51s}
123. Kd3 {+2.59/13 0.50s} Ne6 {0.00/17 0.51s} 124. Kc3 {+2.58/13 0.50s}
Nc7 {0.00/22 0.51s} 125. Be7 {+2.59/13 0.50s} Ne6 {-0.90/14 0.51s}
126. Bh4 {+2.56/13 0.50s} Nd4 {-0.90/14 0.51s} 127. Bg5 {+2.59/12 0.50s}
Ne2+ {-0.90/14 0.51s} 128. Kd3 {+2.58/13 0.50s} Nd4 {-0.90/17 0.51s}
129. Bh6 {+2.57/13 0.50s} Nc6 {0.00/16 0.51s} 130. Kc3 {+2.58/13 0.50s}
Ke4 {0.00/20 0.51s} 131. Bg5 {+2.56/13 0.50s} Kd5 {-0.90/17 0.51s}
132. Bf6 {+2.59/14 0.50s} Na7 {-0.90/17 0.51s} 133. Bh4 {+2.58/13 0.50s}
Nb5+ {-0.90/16 0.51s} 134. Kd3 {+2.59/13 0.50s} Nd4 {-0.90/17 0.49s}
135. Bf2 {+2.58/14 0.50s} Nc6 {-0.90/17 0.51s} 136. Kc3 {+2.59/14 0.50s}
Nb8 {-0.90/16 0.49s} 137. Bd4 {+2.59/14 0.50s} Nc6 {-0.90/15 0.51s}
138. Bf2 {+2.59/14 0.50s} Nb8 {0.00/20 0.49s} 139. Bd4 {+2.58/14 0.50s}
Nc6 {0.00/26 0.49s} 140. Be3 {+2.59/14 0.50s} Ne5 {-0.90/14 0.51s}
141. Bg1 {+2.59/13 0.50s} Nc4 {-0.90/16 0.49s} 142. Bd4 {+2.59/13 0.50s}
Nd6 {-0.90/15 0.51s} 143. Kd3 {+2.66/12 0.50s} Nb5 {-0.90/16 0.49s}
144. Bc5 {+2.60/13 0.50s} Na3 {-0.90/16 0.51s} 145. Bd4 {+2.62/13 0.50s}
Nb5 {0.00/23 0.49s} 146. Bf6 {+2.60/13 0.50s} Na7 {-0.85/14 0.49s}
147. Bg5 {+2.58/13 0.50s} Nc6 {0.00/15 0.51s} 148. Kc3 {+2.62/13 0.50s}
Nd4 {-0.90/15 0.49s} 149. Be3 {+2.60/13 0.50s} Nb5+ {-0.90/14 0.51s}
150. Kd3 {+2.59/13 0.50s} Nd6 {-0.90/15 0.49s} 151. Bb6 {+2.60/12 0.50s}
Nc4 {-0.90/17 0.49s} 152. Bc7 {+2.61/13 0.50s} Nb2+ {-0.90/14 0.51s}
153. Kc3 {+2.59/12 0.50s} Nc4 {-0.90/14 0.49s} 154. Kd3 {+2.62/13 0.50s}
Nb2+ {0.00/21 0.51s} 155. Kc3 {+2.44/12 0.50s} Nc4 {0.00/19 0.49s}
156. Bd8 {+2.43/13 0.50s} Na3 {-0.90/14 0.49s} 157. Bb6 {+2.17/11 0.50s}
Ke4 {0.00/17 0.51s} 158. b5 {+1.30/12 0.50s} Nxb5+ {0.00/16 0.49s}
159. Kc4 {+1.32/12 0.50s} Ke5 {0.00/16 0.51s} 160. Kc5 {+1.42/12 0.50s}
Nc3 {0.00/17 0.49s} 161. Ba7 {+1.44/11 0.50s} Nd5 {0.00/17 0.49s}
162. Kc6 {+1.44/13 0.50s} Nb4+ {0.00/17 0.51s} 163. Kb7 {+1.44/12 0.50s}
Kd5 {0.00/18 0.49s} 164. Bf2 {+0.98/13 0.50s} Kc4 {0.00/17 0.51s}
165. Kb6 {+1.58/13 0.50s} Kd5 {0.00/18 0.49s} 166. Be1 {+1.71/13 0.50s}
Nd3 {0.00/15 0.49s} 167. Bc3 {+1.71/13 0.50s} Nc5 {0.00/17 0.51s}
168. Be1 {+1.71/13 0.50s} Kc4 {0.00/16 0.49s} 169. Bg3 {+1.77/12 0.50s}
Ne4 {0.00/16 0.49s} 170. Bf4 {+1.77/13 0.50s} Nc5 {0.00/17 0.51s}
171. Bd6 {+1.53/13 0.50s} Ne4 {0.00/17 0.49s} 172. Bf4 {+1.53/14 0.50s}
Nc5 {0.00/19 0.51s} 173. Be3 {+1.45/14 0.50s} Nb3 {0.00/17 0.49s}
174. Bg5 {+1.46/13 0.50s} Nc5 {0.00/19 0.49s} 175. Be7 {+1.61/13 0.50s}
Nb3 {0.00/17 0.51s} 176. Bh4 {+1.39/13 0.50s} Nc5 {0.00/19 0.49s}
177. Be7 {+1.42/13 0.50s} Nb3 {0.00/18 0.51s} 178. Bd6 {+1.19/13 0.50s}
Kd5 {0.00/19 0.49s} 179. Bf8 {+1.35/13 0.50s} Ke6 {0.00/18 0.49s}
180. Bg7 {+1.39/13 0.50s} Kf5 {0.00/17 0.51s} 181. Bc3 {+1.42/12 0.50s}
Kf4 {0.00/18 0.49s} 182. Bg7 {+1.29/12 0.50s} Kf5 {0.00/25 0.49s}
183. Bh6 {+1.42/13 0.50s} Ke6 {0.00/18 0.51s} 184. Bf4 {+1.38/13 0.50s}
Kf5 {0.00/21 0.49s} 185. Be3 {+1.44/13 0.50s} Ke4 {0.00/19 0.51s}
186. Bh6 {+1.35/13 0.50s} Kd5 {0.00/18 0.49s} 187. Bg5 {+1.42/13 0.50s}
Nc5 {0.00/18 0.49s} 188. Bh4 {+1.27/13 0.50s} Kc4 {0.00/17 0.51s}
189. Bg3 {+1.36/13 0.50s} Ne4 {0.00/18 0.49s} 190. Be1 {+1.35/14 0.50s}
Nc5 {0.00/25 0.49s} 191. Bd2 {+1.35/14 0.50s} Kd5 {0.00/17 0.51s}
192. Be3 {+1.35/14 0.50s} Nb3 {0.00/18 0.49s} 193. Bf4 {+1.20/13 0.50s}
Ke4 {0.00/22 0.49s} 194. Bg3 {+1.37/13 0.50s} Kf3 {0.00/18 0.51s}
195. Be1 {+1.42/13 0.50s} Ke2 {0.00/19 0.49s} 196. Bc3 {+1.29/13 0.50s}
Kd3 {0.00/18 0.49s} 197. Be1 {+1.14/13 0.50s} Ke2 {0.00/24 0.51s}
198. Bg3 {+1.22/13 0.50s} Kd3 {0.00/18 0.49s} 199. Bf2 {+1.22/13 0.50s}
Kc3 {0.00/18 0.51s} 200. Bh4 {+1.25/13 0.50s} Kd3 {0.00/18 0.49s}
201. Bg5 {+1.20/13 0.50s} Kd4 {0.00/18 0.49s} 202. Bf4 {+1.33/13 0.50s}
Ke4 {0.00/24 0.51s} 203. Bg5 {+1.08/13 0.50s} Kf5 {0.00/21 0.49s}
204. Be3 {+1.01/12 0.50s} Ke5 {0.00/18 0.51s} 205. Bc5 {+0.83/12 0.50s}
Ke4 {0.00/18 0.49s} 206. Bg1 {+0.77/12 0.50s} Kf3 {0.00/18 0.51s}
207. Kxa6 {+0.72/12 0.50s} Nxa5 {0.00/19 0.49s} 208. Bd4 {+0.75/13 0.50s}
Nc4 {0.00/18 0.49s} 209. Bc3 {+0.75/11 0.50s} Ke4 {0.00/17 0.51s}
210. Bf6 {+0.71/13 0.50s} Kf3 {0.00/18 0.49s} 211. Bc3 {+0.71/13 0.50s}
Ke4 {0.00/33 0.49s} 212. Kb7 {+0.71/13 0.50s} Kd3 {0.00/19 0.51s}
213. Bf6 {+0.72/13 0.50s} Ne3 {0.00/16 0.49s} 214. Kc6 {+0.72/12 0.50s}
Ke4 {0.00/18 0.51s} 215. Kc5 {+0.72/13 0.50s} Nf1 {0.00/17 0.49s}
216. Kd6 {+0.71/13 0.50s} Kf4 {0.00/17 0.49s} 217. Bd4 {+0.73/11 0.50s}
Kf3 {0.00/18 0.51s} 218. Ke7 {+0.74/12 0.50s} Ke4 {0.00/20 0.49s}
219. Bc5 {+0.73/13 0.50s} Kd3 {0.00/18 0.51s} 220. Kf7 {+0.73/13 0.50s}
Ng3 {0.00/18 0.49s} 221. Bd6 {+0.73/12 0.50s} Ke4 {0.00/19 0.49s}
222. Bc7 {+0.72/13 0.50s} Kd4 {0.00/19 0.51s} 223. Ke6 {+0.73/12 0.50s}
Nh1 {0.00/18 0.49s} 224. Be5+ {+0.72/12 0.50s} Kc4 {0.00/17 0.51s}
225. Kd7 {+0.74/12 0.50s} Ng3 {0.00/18 0.49s} 226. Kc7 {+0.72/12 0.50s}
Nh1 {0.00/19 0.51s} 227. Bf6 {+0.72/13 0.50s} Ng3 {0.00/18 0.49s}
228. Kc6 {+0.72/13 0.50s} Nf5 {0.00/17 0.49s} 229. Be5 {+0.72/13 0.50s}
Kd3 {0.00/17 0.51s} 230. Kd5 {+0.72/12 0.50s} Ne3+ {0.00/17 0.49s}
231. Ke6 {+0.72/11 0.50s} Nd5 {0.00/18 0.51s} 232. Kf7 {+0.72/12 0.50s}
Ke3 {0.00/17 0.49s} 233. Bd6 {+0.72/12 0.50s} Kd2 {0.00/17 0.49s}
234. Kg7 {+0.71/12 0.50s} Kd3 {0.00/18 0.51s} 235. Kf7 {+0.71/12 0.50s}
Ke4 {0.00/18 0.49s} 236. Kg6 {+0.72/13 0.50s} Nc3 {0.00/18 0.51s}
237. Kg7 {+0.72/13 0.50s} Kf3 {0.00/18 0.49s} 238. Be5 {+0.71/13 0.50s}
Nd5 {0.00/19 0.51s} 239. Kg6 {+0.72/13 0.50s, Black makes an illegal move: c3d5}
1-0[/pgn]

Thanks for testing it for me! Really appreciated. And I am happy that Zahak didn't do too bad with this short TC, I usually do all my tests in 1m+1s

[Gabor Szots]

Let me add one more thing.

An engine's calculated performance may depend upon the opponent's range. E.g. if you make your engine play against stronger opposition, the calculated Elo may differ considerably from wat you would have obtained had you let your engine play against lower ranked opponents. I have seen this several times, look at Marcel's example above with Rustic.
From the blitz list you can see that Zahak played against a pool of opponents 68 Elos stronger than itself. The selection was based upon a wrong estimate of its strength. Had it played against a weaker pool its rating might easily be 50 Elos higher. Of course I was planning to correct that by extending its tests to weaker opponents but now that Zahak 2 is out I'm not going to do that.

[lithander]

An engine's calculated performance may depend upon the opponent's range. E.g. if you make your engine play against stronger opposition, the calculated Elo may differ considerably from wat you would have obtained had you let your engine play against lower ranked opponents. [...]
From the blitz list you can see that Zahak played against a pool of opponents 68 Elos stronger than itself. The selection was based upon a wrong estimate of its strength. Had it played against a weaker pool its rating might easily be 50 Elos higher.

That's interesting. I remember being a little disappointed by the CCRL rating of MinimalChess 0.2 being lower than expected. So I just checked it again and indeed the Average Opponent is +151.3 ELO stronger. (with a draw rate of only 11.2%)

[mvanthoor]

From the blitz list you can see that Zahak played against a pool of opponents 68 Elos stronger than itself. The selection was based upon a wrong estimate of its strength. Had it played against a weaker pool its rating might easily be 50 Elos higher. Of course I was planning to correct that by extending its tests to weaker opponents but now that Zahak 2 is out I'm not going to do that.

This is also the reason why I'm not hurrying with the development of Rustic; if the engine is tested and scores considerably less (or more) than expected, I can raise an issue about that and the testers have the option and time to run another gauntlet to compensate. I also have a bit of OCD-like tendencies... if at all possible, I'd like to see all the Rustic versions tested in CCRL (except for the ones that add functionality or fix issues that don't change strength, but those will be 0.0.x releases. Adding XBoard, or the build script to the engine will be one of those, for example.)

[amanjpro]

Let me add one more thing.

An engine's calculated performance may depend upon the opponent's range. E.g. if you make your engine play against stronger opposition, the calculated Elo may differ considerably from wat you would have obtained had you let your engine play against lower ranked opponents. I have seen this several times, look at Marcel's example above with Rustic.
From the blitz list you can see that Zahak played against a pool of opponents 68 Elos stronger than itself. The selection was based upon a wrong estimate of its strength. Had it played against a weaker pool its rating might easily be 50 Elos higher. Of course I was planning to correct that by extending its tests to weaker opponents but now that Zahak 2 is out I'm not going to do that.

That is also a good point. And the only one to be blamed for this is me, I gave the wrong estimation

[Gabor Szots]

An engine's calculated performance may depend upon the opponent's range. E.g. if you make your engine play against stronger opposition, the calculated Elo may differ considerably from wat you would have obtained had you let your engine play against lower ranked opponents. [...]
From the blitz list you can see that Zahak played against a pool of opponents 68 Elos stronger than itself. The selection was based upon a wrong estimate of its strength. Had it played against a weaker pool its rating might easily be 50 Elos higher.

That's interesting. I remember being a little disappointed by the CCRL rating of MinimalChess 0.2 being lower than expected. So I just checked it again and indeed the Average Opponent is +151.3 ELO stronger. (with a draw rate of only 11.2%)

With MinimalChess 0.2 (and other engines of similar strength) I had the problem of not having enough suitable opponents. I have only a couple of engines below 1000, so I had the choice of a rating distortion or refusing to test it.