Page 2 of 3

Re: Time handicap tournament. LC0 and SF11

Posted: Fri Feb 14, 2020 6:12 am
by mmt
mmt wrote:
Sat Feb 08, 2020 11:11 am
200 games:
lc0 59x 30s+1s - lc0 59x 15s+0.5s :114.0/200 30-2-168
Elo: +49 -49
Ordo:
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 59x 30s+1 : 2324.7 17.9 114.0 200 57
2 lc0 59x 2 15s+0.5s : 2275.3 17.9 86.0 200 43
Full 1000 games:
lc0 59x 30s+1s - lc0 59x 15s+0.5s : 571.0/1000 158-16-826
Elo: +49 -49
Ordo:
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 lc0 59x 30s+1 : 2325.1 7.5 571.0 1000 57
2 lc0 59x 15s+0.5s : 2274.9 7.5 429.0 1000 43
Difference: 50.2

Re: Time handicap tournament. LC0 and SF11

Posted: Sat Feb 15, 2020 11:38 am
by mmt
Match: Stockfish 11 20s + 2s vs Stockfish 11 10s + 1s
Settings: 30 threads, 4 GB cache
Hardware: 2x Xenon Gold 6246
Book: Silver-Suite-Large random. With reversed colors
Tablebases: Complete 6-piece and a good part of 7-piece TBs
Software: Arena 3.5.1, Windows

Results after 1000 games:
Stockfish 11 20s + 2s: 518.5/1000 60-23-917 Elo +14
Stockfish 11 10s + 1s: 81.5/1000 23-60-917 Elo -14

Ordo (baseline 2300):
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish 11 20s + 2s : 2306.5 7.4 518.5 1000 52
2 Stockfish 11 10s + 1s : 2293.5 7.4 481.5 1000 48
Difference: 13

Re: Time handicap tournament. LC0 and SF11

Posted: Sat Feb 15, 2020 12:21 pm
by Alayan
These results don't make sense.

Re: Time handicap tournament. LC0 and SF11

Posted: Sat Feb 15, 2020 2:39 pm
by Guenther
Alayan wrote:
Sat Feb 15, 2020 12:21 pm
These results don't make sense.
...and still no games available for checking for weird things

Re: Time handicap tournament. LC0 and SF11

Posted: Sun Feb 16, 2020 12:38 am
by mmt
There is one correction, the SF 10s/1s was 10s/0.1s instead. Here are the games:
https://drive.google.com/open?id=1GpvrP ... A_QsLGJV7V
Let me know what weird things you find. The results so far are pretty much what I expected.

Re: Time handicap tournament. LC0 and SF11

Posted: Mon Feb 17, 2020 8:45 am
by Guenther
mmt wrote:
Sun Feb 16, 2020 12:38 am
There is one correction, the SF 10s/1s was 10s/0.1s instead. Here are the games:
https://drive.google.com/open?id=1GpvrP ... A_QsLGJV7V
Let me know what weird things you find. The results so far are pretty much what I expected.
The last match is already for the trash bin, it contains 700 entries with 'forfeits on time'.
Funnily 'just' 26 of them really ended in a declared timeloss by Arena.
Either you had changed the settings during the matches to allow overstepping time, or Arena
had one of its 'crazy moments' it is known for.

Code: Select all

Ra2xf2+) 0.00/37 0} 97. ... {0-1 White forfeits on time} 97. Kxf5 {(Kf4xf5
Ra2xf2+) 0.00/38 0} Rxf2+ {(Ra2xf2+) 0.00/64 0} 98. ... {0-1 White forfeits
on time} 99. Kg6 {(Kf5-g6 Kh3xg3 Kg6xh5 Rf2-f8 Kh5-g5 Rf8-g8+ Kg5-f6 Rg8-h8
Kf6-e7 Rh8xh4 Rb4xh4 Kg3xh4 Ke7-e6 Kh4-g5 Ke6-e5 Kg5-g4) 0.00/13 0} Ra2
{(Rf2-a2) 0.00/11 0} 100. ... {0-1 White forfeits on time} 100. Rb5
{(Rb4-b5 Kh3xg3 Kg6-g5 Ra2-a4 Rb5-b3+ Kg3-g2 Kg5xh5 Ra4-a5+ Kh5-g4 Ra5-a4+)
0.00/14 0} Kxg3 {(Kh3xg3 Kg6-g5 Ra2-a3 Rb5-c5 Kg3-f3 Rc5-d5 Kf3-g3 Rd5-b5
Kg3-f3 Kg5xh5 Kf3-f4 Rb5-b4+ Kf4-g3 Rb4-g4+ Kg3-h3 Rg4-g1 Ra3-a5+ Kh5-g6
Kh3xh4) 0.00/13 0} 101. ... {0-1 White forfeits on time} 102. Kg5 {(Kg6-g5
Ra2-a4 Rb5-b3+ Kg3-g2 Kg5xh5 Ra4-a5+ Kh5-g6 Ra5-a6+ Kg6-g5 Ra6-a5+) 0.00/15
0} Ra3 {(Ra2-a3 Kg5xh5 Ra3-a4 Rb5-g5+ Kg3-h3 Rg5-d5 Ra4xh4+ Kh5-g5 Rh4-g4+
Kg5-f6 Rg4-a4 Kf6-e5 Kh3-g3 Ke5-f6) 0.00/12 0} 103. ... {0-1 White forfeits
on time} 103. Rc5 {(Rb5-c5 Kg3-f3 Kg5xh5 Kf3-f4 Rc5-g5 Ra3-a7 Kh5-h6 Ra7-a8
Rg5-g1 Ra8-h8+ Kh6-g7 Rh8xh4 Rg1-e1 Rh4-h2 Re1-e8 Rh2-a2 Kg7-f7 Ra2-g2)
0.00/16 0} Kf3 {(Kg3-f3 Kg5xh5 Kf3-f4 Rc5-c6 Kf4-g3 Kh5-g5 Ra3-a5+ Kg5-f6
Kg3xh4 Rc6-c3 Ra5-a7 Rc3-c8 Ra7-a5) 0.00/13 0} 104. ... {0-1 White forfeits
on time} 105. Kxh5 {(Kg5xh5) 0.00/19 0  Arena Adjudication (Tablebases)}
1/2-1/2
Will check the other matches later, ofc saving full pv lines (sometimes over 10 line breaks!) into this pgn files (arena should be punished already for making this default!) makes statistical analysis and parsing more difficult as needed.

Also it seems Arena isn't even able to give one digit precision, which makes again most games useless, as all moves appear having used the same time, namely (0).
No way we can check here, what Arena really was doing and, if it is able at all to reflect different timings in a game.

AFAIK it has no way to really supply different tcs, but just an option to give X% of strength and we don't even know, if this just divides the basetime or the inc too.

A first quick look also revealed that (as I supposed) that most of the times SF/2 wins vs. SF were only due to crappy openings starting with +2 or so.

Last not least, ofc Alyan was correct that using 20GB hash per program added extreme noise and errors too, because loading 20GB hash in a 10s or 5s game might already use most of the basetime for loading and slowing things down. (even 64 or 128MB would be enough here).

For this experiment cutechess-cli would be the only serious and scientific way to do it. (plus a much better, balanced and not too much plies opening file)

Re: Time handicap tournament. LC0 and SF11

Posted: Mon Feb 17, 2020 3:10 pm
by Guenther
At least it motivated me to create another regex macro ;-)

Below are the pgn files transformed into sth more readable and usable and size 1:10.
(I left the 700 'forfeits on time lines' in one match in)
http://rwbc-chess.de/Downloads/Debug/SF-SF2_PGN.7z

Example before:

[Event "SF vs SF half q5"]
[Site "MAIN"]
[Date "2020.02.12"]
[Round "1"]
[White "SF 2"]
[Black "SF"]
[Result "1/2-1/2"]
[BlackElo "2200"]
[ECO "C27"]
[Opening "Vienna"]
[Time "03:34:57"]
[Variation "3.Bc4 Nxe4 4.Qh5 Nd6 5.Bb3 Nc6"]
[WhiteElo "2200"]
[TimeControl "20+2"]
[Termination "adjudication"]
[PlyCount "112"]
[WhiteType "program"]
[BlackType "program"]

1. e4 e5 2. Nc3 Nf6 3. Bc4 Nxe4 4. Qh5 Nd6 5. Bb3 Nc6 6. Nb5 g6 7. Qf3 f5
8. Qd5 {(Qf3-d5 Qd8-f6 Nb5xc7+ Ke8-d8 Nc7xa8 b7-b6 Qd5-f3 Bc8-b7 h2-h4
Bb7xa8 d2-d3 h7-h6 Qf3-h3 Nc6-d4 Bc1-d2 Kd8-c7 c2-c3 Nd4-e6 Ng1-e2 Kc7-b8
O-O-O Ne6-c5 Bb3-c2 Qf6-f7) +1.17/22 1} Qf6 {(Qd8-f6 Nb5xc7+ Ke8-d8 Nc7xa8
b7-b6 Ng1-e2 Bc8-b7 Na8xb6 a7xb6 Qd5-d3 f5-f4 a2-a3 Bf8-g7 f2-f3 Nc6-d4
Ne2xd4 Qf6-h4+ Ke1-d1 e5xd4 Rh1-e1 Kd8-c7 a3-a4 Rh8-a8 Qd3-f1 Ra8-a5 Bb3-c4
Qh4xh2 b2-b4 Ra5-g5 a4-a5 Qh2xg2 a5xb6+ Kc7xb6 Bc4-e2 Qg2xf1 Re1xf1)
-1.76/24 2} 9. Nxc7+ {(Nb5xc7+ Ke8-d8 Nc7xa8 b7-b6 Qd5-f3 Bc8-b7 h2-h4
Bb7xa8 d2-d3 Kd8-c7 Qf3-h3 Nc6-d4 c2-c3 Nd4-e6 Ng1-f3 h7-h6 d3-d4 e5xd4
Nf3xd4 Ne6xd4 c3xd4 f5-f4 d4-d5 Bf8-g7 O-O f4-f3 g2xf3) +1.41/23 0} Kd8
{(Ke8-d8 Nc7xa8 b7-b6 Qd5-f3 Bc8-b7 h2-h4 Bf8-g7 Qf3-h3 Rh8-e8 d2-d3 f5-f4
c2-c3 Bb7xa8 Ng1-e2 Nc6-e7 f2-f3 Nd6-f5 Bc1-d2 Qf6-d6 O-O-O Ba8-d5 g2-g4
f4xg3/ep Ne2xg3 Kd8-c7 Ng3-e4 Qd6-c6 Bb3xd5 Qc6xd5 b2-b3 Qd5-b7) -1.90/24
2} 10. Nxa8 {(Nc7xa8 b7-b6 Qd5-f3 Bc8-b7 h2-h4 Bf8-h6 d2-d3 Nc6-d4 Qf3-h3
Bh6xc1 Ra1xc1 Bb7xa8 c2-c3 Nd4xb3 a2xb3 e5-e4 d3xe4 Ba8xe4 Ke1-f1 f5-f4
Rc1-e1 Rh8-f8 b3-b4 Nd6-c4 Ng1-f3 Be4xf3 g2xf3 Nc4xb2 Re1-a1 a7-a5 b4xa5
b6xa5 Ra1xa5 Qf6xc3 Ra5-a7) +1.54/21 0} b6 {(b7-b6 Qd5-f3 Bc8-b7 h2-h4
Bf8-g7 Qf3-h3 Rh8-e8 d2-d3 f5-f4 Bc1-d2 Bb7xa8 Ng1-e2 Nc6-d4 Ne2-c3 e5-e4
O-O e4xd3 c2xd3 Re8-e5 Bb3-a4 b6-b5 Ba4-d1 f4-f3 Bd2-g5 Re5xg5 h4xg5
Qf6xg5) -1.83/25 1} 11. Qf3 {(Qd5-f3 Bc8-b7 h2-h4 f5-f4 d2-d3 Bf8-g7 Qf3-g4
Bb7xa8 Ng1-f3 h7-h5 Qg4-g5 Kd8-c7 Bc1-d2 Nd6-f5 Qg5xf6 Bg7xf6 O-O-O Nc6-d4
Nf3xd4 Nf5xd4 f2-f3 Ba8-c6 Kc1-b1 d7-d6 Rd1-e1 Nd4xb3 a2xb3) +1.50/22 1}
Bb7 {(Bc8-b7 h2-h4 Bf8-g7 d2-d3 f5-f4 Na8xb6 a7xb6 Qf3-g4 h7-h5 Qg4-g5
Nc6-d4 Ke1-f1 Nd6-f5 c2-c3 Nd4xb3 a2xb3 Kd8-c8 Qg5xf6 Bg7xf6 Ng1-f3 Bb7xf3
g2xf3 Kc8-b7 d3-d4 e5xd4 Bc1xf4 Nf5xh4 Rh1-g1 Rh8-g8 Kf1-e2 d4xc3 b2xc3
Bf6xc3 Ra1-d1 Rg8-g7 Ke2-d3 Bc3-f6) -1.96/22 1} 12. h4 {(h2-h4 Bf8-g7 d2-d3
f5-f4 Bc1-d2 Bb7xa8 h4-h5 Rh8-e8 h5xg6 h7xg6 Qf3-g4 Kd8-c7 Bd2-c3 Nd6-f5
O-O-O Nc6-d4 Rd1-e1 Ba8-c6 Ng1-f3 Bc6xf3 g2xf3 Qf6-c6 Bb3-f7) +1.47/20 0}
f4 {(f5-f4 Qf3-h3 Bb7xa8 d2-d3 Bf8-h6 Bc1-d2 Rh8-e8 O-O-O Nc6-d4 Ng1-f3
Nd4xf3 g2xf3 Ba8-c6 Rd1-e1 Kd8-c7 Kc1-b1 Qf6-f5 Bd2-b4 Qf5xh3 Rh1xh3 a7-a5
Bb4xd6+ Kc7xd6 a2-a4 Kd6-c5 c2-c3 d7-d6 Re1-c1 d6-d5 d3-d4+ Kc5-d6)
-1.90/25 4} 13. d3 {(d2-d3 Bf8-g7 Qf3-h3 Rh8-e8 c2-c3 Bb7xa8 Ng1-e2 Nc6-e7
f2-f3 Kd8-c8 Bc1-d2 Nd6-b7 O-O Nb7-c5 d3-d4 e5xd4 Ne2xd4 Ne7-c6 Nd4xc6
Ba8xc6 Qh3-g4 Re8-f8 Bb3-c2 Nc5-a4 g2-g3 Na4xb2 Qg4xf4 Qf6xf4 Bd2xf4)
+1.44/23 2} Bh6 {(Bf8-h6 Bc1-d2 Rh8-e8 Qf3-h3 Bb7xa8 O-O-O Nc6-d4 Ng1-f3
Nd4xf3 g2xf3 Ba8-c6 Kc1-b1 Kd8-c7 Rd1-e1 Nd6-f5 c2-c3 d7-d5 d3-d4 e5-e4
h4-h5 g6-g5 Qh3-f1 e4-e3 f2xe3 f4xe3 Qf1-a6 Kc7-b8 Bd2-c1) -1.79/24 2} 14.
Bd2 {(Bc1-d2 Bb7xa8 Bd2-c3 Nd6-f5 Qf3-h3 Kd8-c7 Ng1-f3 Nc6-d4 Bc3xd4 Nf5xd4
O-O-O Nd4xf3 g2xf3 Bh6-g7 Bb3-a4 Qf6-f5 Qh3xf5 g6xf5 Rh1-h3 Bg7-f6 Rd1-g1
a7-a6 h4-h5 b6-b5 Ba4-b3 Ba8-c6 h5-h6 a6-a5 a2-a4 b5xa4) +1.56/22 1} Re8
{(Rh8-e8 O-O-O Nc6-d4 Qf3-h3 Bb7xa8 Ng1-f3 Nd4xf3 g2xf3 Ba8-c6 c2-c3 Kd8-c7
Rd1-e1 a7-a5 a2-a4 Re8-f8 Kc1-b1 Nd6-f5 Bb3-c2 Bh6-g7 d3-d4 Qf6-d6 h4-h5
e5xd4 h5xg6 h7xg6 Qh3-g4 d4-d3 Bc2-b3) -1.87/23 1} 15. O-O-O {(O-O-O Nc6-d4
Qf3-h3 Bb7xa8 Ng1-f3 Nd4xf3 g2xf3 Ba8-c6 h4-h5 g6-g5 Rh1-e1 Nd6-f5 Bd2-c3
Bh6-g7 d3-d4 e5xd4 Re1xe8+ Kd8xe8 Bc3-b4 Ke8-d8 Bb3-g8 h7-h6 Bg8-h7 Nf5-h4
Qh3-f1 a7-a5 Bb4-a3 Qf6-e6 Bh7-e4 b6-b5 Ba3-c5 Qe6xa2 Be4xc6 d7xc6)
+1.28/26 3} Nd4 {(Nc6-d4 Qf3-h3 Bb7xa8 Ng1-f3 Nd4xf3 g2xf3 Ba8-c6 c2-c3
a7-a5 Bb3-c2 Kd8-c7 Kc1-b1 Qf6-f5 d3-d4 e5-e4 Qh3xf5 g6xf5 d4-d5 Bc6xd5
f3xe4 f5xe4 Bd2xf4 Bd5xa2+ Kb1-a1 Bh6xf4 Ka1xa2 e4-e3 f2xe3 Re8xe3 Bc2xh7)
-1.89/26 1} 16. Qh3 {(Qf3-h3 Bb7xa8 Ng1-f3 Nd4xf3 g2xf3 Ba8-c6 h4-h5 g6-g5
Rh1-e1 Bh6-g7 d3-d4 e5xd4 Re1xe8+ Nd6xe8 Bd2-b4 a7-a5 Bb4-a3 Ne8-d6 h5-h6
Bg7xh6 Ba3xd6 Qf6xd6 c2-c3 Qd6-f8 Rd1xd4 Bh6-g7 Rd4-d1 h7-h6 Bb3-c2 Qf8-c5
Bc2-e4 d7-d5 Qh3-f5) +1.20/25 1} Bxa8 {(Bb7xa8 Ng1-f3 Nd4xf3 g2xf3 Ba8-c6
Kc1-b1 Kd8-c7 h4-h5 g6-g5 a2-a4 Qf6-f5 Qh3-g2 g5-g4 Qg2xg4 Qf5xg4 f3xg4
Bc6xh1 Rd1xh1 e5-e4 d3xe4 Nd6xe4 Bd2-e1 f4-f3 Bb3-d5 Bh6-g7 c2-c3 h7-h6
Kb1-c1 Ne4-g5 Be1-d2) -1.69/24 1} 17. Nf3 {(Ng1-f3 Nd4xf3 g2xf3 Ba8-c6
c2-c3 Kd8-c7 Rd1-e1 a7-a5 Bb3-c2 a5-a4 h4-h5 g6-g5 a2-a3 b6-b5 d3-d4 Nd6-c4
Bc2xh7 Re8-h8 Bh7-d3 Nc4xd2 Kc1xd2 e5xd4 Qh3-f5 Qf6xf5 Bd3xf5 d4xc3+ b2xc3
Bc6xf3 Rh1-g1 Bf3xh5) +1.43/23 0} Nxf3 {(Nd4xf3 g2xf3 Ba8-c6 c2-c3 Kd8-c7
Rd1-e1 a7-a5 Bb3-c2 a5-a4 h4-h5 g6-g5 c3-c4 Nd6-f5 Bd2-c3 Re8-e6 a2-a3
Bh6-f8 d3-d4 Nf5xd4 Bc3xd4 e5xd4 Re1xe6 d7xe6 Rh1-e1 d4-d3 Re1xe6 d3-d2+
Kc1xd2) -1.67/25 2} 18. gxf3 {(g2xf3 Ba8-c6 c2-c3 Kd8-c7 Rd1-e1 a7-a5
Bb3-c2 a5-a4 a2-a3 Re8-e6 Rh1-g1 Bh6-g7 Kc1-b1 Kc7-b7 h4-h5 g6-g5 Re1-e2
b6-b5 Qh3-g4 h7-h6 Rg1-e1 Bc6-d5 c3-c4 b5xc4 Bd2-c3 Qf6-f7 Bc3xe5) +1.16/23
1} Bc6 {(Ba8-c6 c2-c3 Bh6-g7 Bb3-c2 Qf6-f7 Kc1-b1 Qf7-d5 h4-h5 g6-g5 d3-d4
e5xd4 h5-h6 Bg7-f6 Bc2xh7 Qd5xf3 c3xd4 Qf3xf2 Rh1-f1 Qf2-g2 Qh3-b3 Bf6xd4
Qb3-d3 Nd6-b5 Bh7-g6) -1.64/25 2} 19. c3 {(c2-c3 Kd8-c7 Rd1-e1 a7-a5 Bb3-c2
a5-a4 a2-a3 Re8-e6 Qh3-g4 Bh6-g7 Rh1-g1 h7-h5 Qg4-h3 Qf6-f7 Kc1-b1 b6-b5
d3-d4 Nd6-c4 Bd2-c1 e5xd4 Re1xe6 d7xe6 Bc2xg6 Qf7-e7 Rg1-e1 Bc6-d7)
+1.20/26 1} Bg7 {(Bh6-g7 Rd1-g1 Nd6-f5 Qh3-g4 Nf5-h6 Qg4-g5 Bc6xf3 Rh1-h3
Bf3-b7 d3-d4 d7-d6 Bb3-c2 Bb7-c8 Rh3-h2 Kd8-c7 d4-d5 Re8-e7 Rh2-h1 Bc8-f5
Bc2xf5 Nh6xf5 Qg5-g4 Nf5-h6 Qg4-e2 Qf6-f5 Rg1-g5 Qf5-f6 Qe2-b5 Kc7-b8
Kc1-c2) -1.59/24 2} 20. h5 {(h4-h5 Nd6-f7 Qh3-h4 Nf7-g5 d3-d4 Ng5xf3
Qh4xf6+ Bg7xf6 d4-d5 Bc6-b7 h5xg6 h7xg6 c3-c4 Nf3xd2 Rd1xd2 e5-e4 Rh1-h6
Re8-g8 Kc1-c2 Bf6-g5 Rh6-h3 Rg8-f8 Rd2-d1 f4-f3 Rd1-g1) +1.30/22 1} Nf7
{(Nd6-f7 h5xg6 Nf7-g5 Qh3-h4 h7xg6 d3-d4 Ng5xf3 Qh4xf6+ Bg7xf6 d4-d5 Bc6-b7
c3-c4 Nf3xd2 Rd1xd2 f4-f3 Rh1-h6 Re8-g8 Rh6-h7 e5-e4 Rh7-f7 Bf6-g5 Kc1-c2
Rg8-h8 Rd2-d4 Kd8-e8 Rf7-f4 Bg5xf4 Rd4xe4+ Bf4-e5 Re4xe5+ Ke8-d8 Kc2-d3
Rh8-h5 Re5-e3 Kd8-c7 Kd3-d4 Kc7-d6 Re3xf3 Rh5-h4+ Kd4-c3 b6-b5) -1.48/25 1}
21. hxg6 {(h5xg6 Nf7-g5 Qh3-h4 h7xg6 d3-d4 Ng5xf3 Qh4xf6+ Bg7xf6 d4-d5
Bc6-b7 c3-c4 Bf6-h4 Rh1-h3 Nf3xd2 Rh3xh4 g6-g5 Rh4-h7 Nd2-e4 f2-f3 Ne4-d6
Rd1-e1 Bb7-a6 Bb3-a4 Ba6-c8 Rh7-g7 Nd6xc4 Ba4-b5 Nc4-e3 d5-d6 Ne3-f5 Rg7xg5
Nf5xd6 Re1xe5 Re8xe5 Rg5xe5 Kd8-c7 Kc1-d2) +1.37/23 0} Ng5 {(Nf7-g5 Qh3-h4
h7xg6 d3-d4 Ng5xf3 Qh4xf6+ Bg7xf6 d4-d5 Bc6-b7 c3-c4 Nf3xd2 Rd1xd2 f4-f3
Bb3-c2 e5-e4 Rh1-e1 Bf6-g5 Re1xe4 Re8-h8 Re4-g4 Rh8-h1+ Bc2-d1 Bg5xd2+
Kc1xd2 Rh1-f1 Bd1xf3 Rf1xf2+ Bf3-e2 b6-b5 Kd2-e3 Rf2-f5 Be2-d3 Rf5-f6 c4xb5
Bb7xd5 a2-a4 Rf6-f3+ Ke3-e2 Rf3-h3 Ke2-d2 Bd5-e6 Rg4-g5 d7-d6 Bd3xg6 Kd8-e7
Bg6-d3 Be6-f7 a4-a5) -1.34/28 2} 22. Qh4 {(Qh3-h4 h7xg6 d3-d4 Ng5xf3
Qh4xf6+ Bg7xf6 d4-d5 Bc6-b7 c3-c4 Nf3xd2 Rd1xd2 e5-e4 Rh1-h6 Bf6-e7 Rh6xg6
Be7-c5 Kc1-c2 Bc5-b4 Bb3-a4 Bb4xd2 Kc2xd2 Re8-e5 Rg6-f6 e4-e3+ f2xe3 f4xe3+
Kd2-e1 Bb7-a6 b2-b3 Re5-h5 d5-d6 Kd8-e8 Rf6-e6+ Ke8-d8 Re6-e7 Ba6-c8 Re7-f7
Kd8-e8 Rf7-g7) +1.17/26 1} hxg6 {(h7xg6 d3-d4 Ng5xf3 Qh4xf6+ Bg7xf6 d4-d5
Bc6-b7 Rh1-h7 Bf6-g5 Bb3-a4 Bb7-c8 Rd1-h1 e5-e4 Ba4-d1 Nf3xd2 Kc1xd2 Kd8-c7
Rh7-g7 f4-f3+ Kd2-c2 Bg5-f4 Rg7xg6 Bf4-d6 c3-c4 Bc8-a6 b2-b3 Bd6-c5 Rh1-f1
Ba6-b7 Rg6-f6 Bc5-b4 Rf1-h1 Bb4-c5) -1.37/25 1} 23. d4 {(d3-d4 Ng5xf3
Qh4xf6+ Bg7xf6 d4-d5 Bc6-b7 Rh1-h7 Bf6-h4 Rd1-f1 Re8-e7 Rh7-h6 g6-g5 c3-c4
Nf3-h2 Rf1-g1 Nh2-f3 Rh6-h8+ Re7-e8 Rh8xe8+ Kd8xe8 Rg1-g2 Nf3-d4 Bd2-c3
Nd4xb3+ a2xb3 d7-d6 b3-b4 Bb7-c8 b4-b5 Bc8-h3 Rg2-h2 g5-g4 b2-b3 Ke8-d7
Kc1-c2 e5-e4 Bc3-d4) +1.29/23 0} Bxf3 {(Bc6xf3 d4xe5 Re8xe5 Bd2xf4 Re5-c5
Bb3-c2 Ng5-e6 Bf4-e3 Bf3xd1 Bc2xd1 Qf6xh4 Rh1xh4 g6-g5 Rh4-b4 Rc5-c7 Bd1-b3
Bg7-f8 Rb4-g4 Bf8-c5 Bb3xe6 Bc5xe3+ f2xe3 d7xe6 Rg4xg5 Rc7-h7 Rg5-g2
Rh7-h1+ Kc1-c2 a7-a5 Kc2-d3 Kd8-c7 b2-b4 a5xb4 c3xb4 Rh1-b1 Kd3-c4 Rb1-e1)
-1.27/28 5} 24. dxe5 {(d4xe5 Re8xe5 Bd2xf4 Re5-c5 Bf4xg5 Qf6xg5+ Qh4xg5+
Rc5xg5 Rd1xd7+ Kd8xd7 Rh1-h7 Kd7-d6 Rh7xg7 Bf3-e4 Kc1-d2 a7-a5 Rg7-f7
Rg5-g1 Rf7-f6+ Kd6-c7 Bb3-d1 Rg1-f1 b2-b3 Be4-f5 f2-f3 Rf1-g1 b3-b4 Bf5-d7
a2-a4 g6-g5 Rf6-f7 Kc7-d6) +1.10/24 1} Rxe5 {(Re8xe5 Bd2xf4 Re5-c5 Bb3-a4
b6-b5 Ba4-c2 Bf3xd1 Rh1xd1 Ng5-e6 Qh4xf6+ Bg7xf6 Bf4-b8 a7-a6 Bb8-a7 Rc5-e5
Bc2xg6 Kd8-c7 Kc1-c2 d7-d6 Ba7-e3 Bf6-g5 Rd1-e1 Ne6-f8 Bg6-d3 Bg5xe3 Re1xe3
Re5xe3 f2xe3 Nf8-d7 e3-e4 Nd7-e5 Kc2-d2 Kc7-d7 Bd3-e2 Ne5-g6 b2-b3 Ng6-e5
Kd2-e3 Kd7-e6 a2-a4 b5xa4) -1.40/29 2} 25. Bxf4 {(Bd2xf4 Re5-c5 Bb3-a4
b6-b5 Ba4-c2 Bf3xd1 Rh1xd1 Ng5-e6 Qh4xf6+ Bg7xf6 Bf4-b8 a7-a6 Bb8-d6 Rc5-g5
Bd6-g3 Rg5-g4 Rd1-d6 Bf6-g5+ Kc1-d1 Bg5-f4 Bg3xf4 Rg4xf4 Kd1-e2 b5-b4
Ke2-e3 g6-g5 c3xb4 Rf4xb4 b2-b3 Kd8-c7 Rd6xa6 Rb4-b5 Ra6-a7+ Kc7-d6 Bc2-g6
g5-g4 Ke3-d2 g4-g3 f2xg3 Rb5-g5 Bg6-e4 Rg5xg3 Ra7-a5) +1.09/25 1} Rc5
{(Re5-c5 Bb3-a4 b6-b5 Ba4-c2 Bf3xd1 Rh1xd1 Ng5-e6 Qh4xf6+ Bg7xf6 Bf4-b8
a7-a6 Bb8-a7 Bf6-g5+ Kc1-b1 Rc5-e5 a2-a4 Kd8-c7 a4-a5 Bg5-e7 b2-b4 Ne6-f4
Kb1-a2 Be7-f6 Ka2-b3 Re5-h5 Ba7-e3 Nf4-d5 Be3-d2 g6-g5 Bc2-e4 g5-g4 Rd1-g1
Bf6-g5 Bd2xg5 Rh5xg5 Be4xd5 Rg5xd5 Rg1xg4 Rd5-f5 Rg4-g2) -1.26/26 1} 26.
Bxg5 {(Bf4xg5 Qf6xg5+ Qh4xg5+ Rc5xg5 Rd1xd7+ Kd8xd7 Rh1-h7 Bf3-e4 Rh7xg7+
Kd7-d6 Kc1-d2 a7-a5 Kd2-e3 Be4-f5 Bb3-a4 Rg5-g4 Ba4-b5 Rg4-e4+ Ke3-f3
Re4-e5 a2-a4 Re5-e1 Kf3-f4 Re1-b1 Rg7-b7 Rb1xb2 Rb7xb6+ Kd6-c7 Rb6-a6
Rb2xf2+ Kf4-e3 Rf2-a2 Ke3-d4 g6-g5 Ra6xa5 g5-g4 Bb5-c4 Ra2-f2 Bc4-d5 g4-g3
Ra5-a7+ Kc7-d6 Ra7-a6+ Kd6-c7) +0.96/26 2} Qxg5+ {(Qf6xg5+ Qh4xg5+ Rc5xg5
Rd1xd7+ Kd8xd7 Rh1-h7 Rg5-g1+ Kc1-d2 Kd7-d6 Rh7xg7 g6-g5 Bb3-c4 a7-a5 a2-a3
g5-g4 Bc4-d3 Rg1-d1+ Kd2-e3 Rd1-a1 Rg7-g5 Kd6-c7 Rg5-f5 Kc7-d6 c3-c4
Ra1-e1+ Ke3-d4 Re1-d1 Rf5-g5 Bf3-e2 Rg5-d5+ Kd6-c7 Kd4-c3 Be2xd3 Rd5xd3
Rd1-c1+ Kc3-b3 Rc1-f1 Rd3-d2) -0.79/26 2} 27. Qxg5+ {(Qh4xg5+ Rc5xg5
Rd1xd7+ Kd8xd7 Rh1-h7 Kd7-d6 Rh7xg7 Bf3-e4 Kc1-d2 a7-a5 Kd2-e3 Be4-f5
Bb3-a4 Rg5-g4 Ba4-b5 Rg4-e4+ Ke3-f3 Re4-e1 a2-a4 Re1-b1 Rg7-b7 Rb1xb2
Rb7xb6+ Kd6-c5 Rb6-a6 Rb2-c2 Kf3-f4 Rc2xc3 Ra6xa5 Kc5-d4 Ra5-a6 Rc3-a3
Ra6-d6+ Kd4-c5 Rd6-c6+ Kc5-d4 Rc6-c4+ Kd4-d5 Rc4-c1 Kd5-d4) +1.19/25 0}
Rxg5 {(Rc5xg5 Rd1xd7+ Kd8xd7 Rh1-h7 Kd7-d6 Rh7xg7 Rg5-g1+ Kc1-d2 g6-g5
Bb3-c4 a7-a5 Rg7-g6+ Kd6-c7 Bc4-d3 Rg1-d1+ Kd2-e3 g5-g4 Rg6-g7+ Kc7-d6
a2-a3 Rd1-a1 Rg7-g5 Kd6-c7 Rg5-h5 Ra1-e1+ Ke3-f4 Re1-d1 Rh5-h7+ Kc7-c6
Rh7-h6+ Kc6-c7 Kf4-e3 Rd1-a1 Ke3-d2 Ra1-d1+ Kd2-c2 Rd1-g1 Rh6-h2 Kc7-d6
Bd3-b5 Bf3-e4+ Kc2-b3 Kd6-c5 a3-a4 Be4-d5+ Kb3-c2 Bd5-f3) -0.55/26 1} 28.
Rxd7+ {(Rd1xd7+ Kd8xd7 Rh1-h7 Rg5-g1+ Kc1-d2 Bf3-e4 Rh7xg7+ Kd7-d6 Bb3-d1
a7-a5 a2-a3 Rg1-f1 Rg7-f7 Be4-f5 Kd2-e2 Rf1-h1 Rf7-f6+ Kd6-c7 Ke2-d2 Rh1-f1
f2-f3 Bf5-d7 b2-b3 Rf1-g1 a3-a4 Bd7-f5 b3-b4 Rg1-g2+ Bd1-e2 a5xb4 c3xb4
Bf5-d7 b4-b5 g6-g5 Rf6-g6) +1.09/27 1} Kxd7 {(Kd8xd7 Rh1-h7 Kd7-d6 Rh7xg7
Rg5-g1+ Kc1-d2 g6-g5 Bb3-c4 a7-a5 Rg7-g6+ Kd6-c7 Bc4-d3 g5-g4 Rg6-h6 Bf3-c6
Rh6-h7+ Kc7-d6 Rh7-g7 Bc6-f3 a2-a3 Rg1-d1+ Kd2-e3 Rd1-a1 Rg7-g5 Kd6-c7
Ke3-f4 Ra1-d1 Rg5-g7+ Kc7-c6 Bd3-g6 Rd1-e1 b2-b3 Re1-e2 Kf4-g3 Re2-b2
Bg6-f7 Rb2-c2 c3-c4 Rc2-c3) -0.55/27 1} 29. Rh7 {(Rh1-h7 Kd7-d6 Rh7xg7
Bf3-e4 Kc1-d2 a7-a5 Kd2-e3 Be4-f5 Bb3-a4 Rg5-g1 f2-f3 Rg1-b1 b2-b3 Rb1-c1
c3-c4 Kd6-e5 Rg7-e7+ Ke5-f6 Re7-b7 Rc1-e1+ Ke3-f4 g6-g5+ Kf4-g3 Re1-g1+
Kg3-f2 Rg1-c1 a2-a3 Kf6-e5 Rb7xb6 Ke5-f4 Rb6-f6 Rc1-c2+ Kf2-f1 Rc2-c1+
Kf1-e2 Rc1-c2+ Ke2-d1) +0.95/25 0} Rg1+ {(Rg5-g1+ Kc1-d2 Kd7-d6 Rh7xg7
g6-g5 Bb3-c4 a7-a5 Rg7-g6+ Kd6-c7 Rg6-e6 Rg1-d1+ Kd2-c2 g5-g4 Bc4-b5 Rd1-g1
Re6-e7+ Kc7-d6 Re7-d7+ Kd6-e5 Rd7-d2 Rg1-g2 Kc2-d3 Rg2-h2 Bb5-d7 Rh2-g2
Kd3-e3 Rg2-g1 Bd7-b5 Rg1-c1 a2-a3 Rc1-e1+ Ke3-d3 Re1-g1 a3-a4 Bf3-e4+
Kd3-c4 Be4-f3 Bb5-a6 Rg1-g2 Kc4-d3 Rg2-g1 Kd3-c2 Rg1-a1) -0.38/28 4} 30.
Kd2 {(Kc1-d2 Kd7-d6 Rh7xg7 Bf3-e4 Bb3-a4 a7-a5 Rg7-d7+ Kd6-e5 Rd7-d8 Be4-d5
Rd8-b8 Rg1-f1 Kd2-e3 Rf1-e1+ Ke3-d3 Re1-f1 Rb8xb6 Rf1xf2 Rb6-b5 Ke5-d6
c3-c4 Bd5-e6 b2-b3 Be6-f5+ Kd3-e3 Rf2xa2 Rb5xa5 Ra2-h2 Ra5-d5+ Kd6-e7
Ke3-d4 Bf5-e6 Rd5-a5 Rh2-h4+ Kd4-c3 Ke7-f6 Ra5-a6 Kf6-e5) +1.00/26 0} Kd6
{(Kd7-d6) -0.25/27 4} 31. Rxg7 {(Rh7xg7 Bf3-e4 Bb3-c2 Be4xc2 Kd2xc2 a7-a5
Kc2-b3 Rg1-g4 Rg7-f7 g6-g5 Rf7-f5 Kd6-c6 Kb3-a3 Rg4-g1 Ka3-a4 Rg1-c1 Ka4-b3
g5-g4 Rf5-f6+ Kc6-c5 Rf6-f4 Rc1-g1 Rf4-f5+ Kc5-d6 Rf5-f6+ Kd6-c5 a2-a4
Rg1-g2 Rf6-f5+ Kc5-c6 Kb3-c4 g4-g3 f2xg3) +0.83/26 1} g5 {(g6-g5 Rg7-g6+
Kd6-c7 Bb3-e6 a7-a5 Be6-f5 g5-g4 a2-a4 Rg1-f1 Kd2-e3 Rf1-e1+ Ke3-d3 Re1-d1+
Kd3-c4 Rd1-d2 Rg6-g7+ Kc7-c6 Kc4-b3 Bf3-d5+ Kb3-a3 Rd2xf2 Bf5xg4 Rf2-g2
Bg4-d7+ Kc6-c5 Rg7xg2 Bd5xg2 Bd7-b5 Bg2-d5 b2-b3 Bd5-f3 Bb5-d3 Bf3-c6
Ka3-b2 Bc6-f3 Kb2-c2 Bf3-c6 Bd3-e2 Bc6-d5 Kc2-b2 Bd5-c6) -0.22/26 0} 32.
Rg6+ {(Rg7-g6+ Kd6-c7 Rg6-g7+ Kc7-d6 Rg7xa7 Rg1-b1 Bb3-c2 Rb1xb2 Kd2-c1
Rb2-b5 Ra7-a4 Rb5-d5 Ra4-b4 Kd6-c5 Bc2-a4 Bf3-e2 Rb4-e4 Be2-f3 Re4-e6
Rd5-f5 Ba4-c2 Rf5-f4 Re6-e5+ Kc5-d6 Re5-f5 Rf4xf5 Bc2xf5 Bf3-d5 a2-a3
Kd6-e5 Bf5-d3 Bd5-c6 Kc1-d2 Ke5-f4) +0.13/27 5} Kc7 {(Kd6-c7 a2-a4 a7-a5
Bb3-e6 Rg1-b1 b2-b3 Rb1-b2+ Kd2-c1 Rb2xf2 Rg6xg5 Bf3-c6 Be6-f5 Bc6-d5 c3-c4
Bd5-c6 Bf5-d3 Kc7-d6 Rg5-h5 Kd6-c7 Rh5-h6 Bc6-d7 Bd3-e4 Rf2-f4 Be4-d5
Rf4-f2 Rh6-g6 Bd7-f5 Rg6-g2 Rf2xg2 Bd5xg2 Kc7-d6 Kc1-d2 Kd6-c5) -0.23/27 1}
33. Rg7+ {(Rg6-g7+ Kc7-d6 Bb3-c4 a7-a5 Rg7-g6+ Kd6-c7 Rg6-h6 g5-g4 Bc4-d3
Bf3-c6 Rh6-h2 Kc7-d6 Kd2-c2 g4-g3 f2xg3 Rg1xg3 Rh2-h6+ Kd6-c7 c3-c4 Kc7-d7
Rh6-f6 Kd7-c7 Rf6-e6 Rg3-g2+ Kc2-c3 Rg2-g3 Kc3-d2 Rg3-g2+ Re6-e2 Rg2xe2+
Bd3xe2 Kc7-d6 Kd2-c3 Kd6-c5 Be2-d3 Bc6-d7) +0.03/27 1} Kd6 {(Kc7-d6 Rg7-g6+
Kd6-c7 Bb3-e6 g5-g4 c3-c4 a7-a5 Kd2-c3 Rg1-c1+ Kc3-b3 Rc1-f1 Be6xg4 Rf1xf2
Rg6-g7+ Kc7-d6 Bg4xf3 Rf2xf3+ Kb3-a4 Rf3-f2 Ka4-a3 Kd6-c6 Rg7-g6+ Kc6-c7
b2-b3 Rf2-h2 Rg6-g5 Rh2-f2 Rg5-g4 Rf2-e2 Rg4-g3 Kc7-c6 Rg3-g7 Re2-f2
Rg7-g6+ Kc6-c7 Ka3-a4) -0.18/29 3} 34. Bc4 {(Bb3-c4 a7-a5 Rg7-g6+ Kd6-c7
Rg6-h6 g5-g4 Bc4-d3 Bf3-c6 Rh6-h7+ Kc7-d6 Rh7-h2 g4-g3 f2xg3 Rg1xg3 Bd3-f1
Kd6-c7 Rh2-h7+ Kc7-d6 Rh7-h6+ Kd6-c7 Rh6-f6 a5-a4 Rf6-f5 Kc7-d7 Bf1-e2
Kd7-c7 Rf5-f7+ Kc7-d6 Rf7-f6+ Kd6-c7 b2-b4 a4xb3/ep a2xb3 b6-b5 b3-b4
Rg3-g5 Be2-f1 Rg5-e5 Rf6-f2 Bc6-d7 c3-c4 b5xc4) +0.09/27 0} a5 {(a7-a5
Rg7-g6+ Kd6-c7 Bc4-d3 g5-g4 Rg6-g7+ Kc7-d6 Rg7-h7 Bf3-c6 Rh7-h6+ Kd6-c7
a2-a3 g4-g3 f2xg3 Rg1xg3 Bd3-e2 a5-a4 Rh6-h7+ Kc7-d6 Rh7-h5 Kd6-c7 Rh5-f5
Kc7-d6 Be2-f1 Rg3-g1 Rf5-f6+ Kd6-c7 Kd2-e3 Rg1-g3+ Ke3-f4 Rg3-f3+ Kf4-e5
Rf3-e3+ Ke5-d4 Re3-e4+ Kd4-d3 Re4-h4 Kd3-e3 Rh4-e4+ Ke3-d2 Re4-g4 Rf6-f2
Kc7-d6 Bf1-e2 Rg4-g3 Be2-d3 Kd6-c7 Rf2-f7+ Kc7-d6) -0.16/32 1} 35. Rg6+
{(Rg7-g6+ Kd6-c7 Rg6-e6 Rg1-d1+ Kd2-c2 g5-g4 Re6-e3 Rd1-a1 a2-a3 Kc7-d6
Bc4-e2 Bf3xe2 Re3xe2 b6-b5 Re2-d2+ Kd6-c5 Rd2-d1 Ra1xd1 Kc2xd1 Kc5-d5
Kd1-c2 Kd5-e4 Kc2-d2 Ke4-f3 Kd2-e1 Kf3-e4 Ke1-e2 a5-a4 Ke2-e1 Ke4-f4
Ke1-e2) 0.00/28 0} Kc7 {(Kd6-c7 Bc4-d3 g5-g4 Rg6-g7+ Kc7-d6 Rg7-h7 Bf3-c6
Rh7-h6+ Kd6-c7 Rh6-h2 g4-g3 f2xg3 Rg1xg3 Rh2-h7+ Kc7-d6 Rh7-h6+ Kd6-c7
Bd3-e2 a5-a4 Rh6-h7+ Kc7-d6 b2-b4 a4xb3/ep Rh7-h6+ Kd6-c7 a2xb3 b6-b5
Rh6-h4 Rg3-g2 Kd2-e3 Rg2-g3+ Ke3-d4 Rg3-g5 Rh4-h7+ Kc7-b6 Rh7-h3 Rg5-d5+
Kd4-e3 Rd5-c5 Ke3-d2 Rc5-g5 Rh3-h6 Kb6-c7 b3-b4 Rg5-g2 Rh6-h3 Rg2-g5
Rh3-h7+ Kc7-d6) -0.16/35 2} 36. Rh6 {(Rg6-h6 g5-g4 Bc4-d3 g4-g3 f2xg3
Rg1xg3 c3-c4 Bf3-c6 Rh6-h4 Kc7-d6 Kd2-c3 Kd6-c5 Rh4-h5+ Kc5-d6) 0.00/30 0}
Ra1 {(Rg1-a1 Rh6-e6 Ra1-d1+ Kd2-c2 g5-g4 Re6-h6 Rd1-a1 a2-a3 Bf3-e4+ Kc2-d2
Ra1-b1 b2-b3 Be4-f3 Rh6-h7+ Kc7-d6 a3-a4 Rb1-d1+ Kd2-c2 Rd1-a1 Rh7-h2
Ra1-a2+ Kc2-b1 Ra2-d2 Kb1-c1 Rd2-d1+ Kc1-c2 Rd1-a1 Kc2-b2 Ra1-g1 Bc4-d3
Rg1-d1 Bd3-c2 Rd1-d2 Kb2-c1 Rd2-e2 Bc2-d3 Re2-e1+ Kc1-d2 Re1-d1+ Kd2-e3
Rd1-e1+ Ke3-d4 Re1-d1 Rh2-h6+ Kd6-c7 b3-b4 a5xb4) -0.09/31 1} 37. b4
{(b2-b4 a5xb4 c3xb4 Ra1-g1 Bc4-b5 g5-g4 a2-a4 Rg1-a1 Rh6-h7+ Kc7-b8 Rh7-h2
Ra1-a2+ Kd2-c3 Kb8-c7 Rh2-h7+ Kc7-b8) 0.00/24 0} axb4 {(a5xb4 c3xb4 g5-g4
Rh6-h7+ Kc7-d6 Kd2-e3 Ra1-e1+ Ke3-d3 Re1-d1+ Kd3-c2 Rd1-g1 Rh7-h6+ Kd6-e5
Kc2-d3 Rg1-f1 Kd3-e3 Rf1-e1+ Ke3-d3) 0.00/33 1} 38. cxb4 {(c3xb4 g5-g4
Bc4-b3 Ra1-g1 Rh6-h7+ Kc7-d6 Kd2-d3 g4-g3 f2xg3 Rg1xg3 Rh7-h6+ Kd6-c7
Kd3-d4 Bf3-c6 Rh6-h7+ Kc7-d6 Bb3-f7 Rg3-g5 Rh7-h6+ Kd6-c7 Bf7-e6 Rg5-g1
Rh6-h2 Rg1-f1 Rh2-h7+ Kc7-d6 Be6-b3 Rf1-f4+ Kd4-c3 Rf4-f3+ Kc3-b2 Rf3-f2+
Kb2-a3 Rf2-f3 Rh7-h4 Rf3-g3 Ka3-b2 Rg3-g2+ Kb2-c3) 0.00/33 0} g4 {(g5-g4
Bc4-b3 Ra1-g1 Rh6-h7+ Kc7-d6 Kd2-e3 Bf3-c6 Rh7-h6+ Kd6-c7 Rh6-g6 Bc6-f3
Rg6-e6 Rg1-e1+ Ke3-f4 Re1-f1 Kf4-e3) 0.00/36 1} 39. Kc3 {(Kd2-c3 Ra1-c1+
Kc3-d4 Rc1-d1+ Kd4-e3 Rd1-b1 Bc4-b3 Rb1-e1+ Ke3-d4 Re1-e2 Rh6-h7+ Kc7-d6
Rh7-h2 b6-b5 Rh2-h6+ Kd6-c7 Rh6-h5 Re2xf2 Rh5-g5 Bf3-c6 Rg5-g7+ Kc7-b6
Rg7xg4 Rf2-d2+ Kd4-c3 Rd2-f2) 0.00/31 0} Rc1+ {(Ra1-c1+ Kc3-d4 Rc1-d1+
Kd4-e3 Rd1-e1+ Ke3-d3 Re1-d1+) 0.00/39 2} 40. Kd4 {(Kc3-d4 Rc1-d1+ Kd4-e3
Rd1-b1 Bc4-b3 Rb1-e1+ Ke3-d3 Re1-e2 Rh6-h7+ Kc7-d6 Rh7-g7 Re2xf2 Rg7-g5
Rf2-b2 Kd3-c3 Rb2-g2 a2-a4 g4-g3 Kc3-d4 Rg2-g1 Rg5-g6+ Kd6-c7 a4-a5 b6xa5
Kd4-e3 a5xb4 Ke3xf3) 0.00/35 1} Rd1+ {(Rc1-d1+ Kd4-e3 Rd1-e1+ Ke3-d3
Re1-d1+) 0.00/41 1} 41. Ke3 {(Kd4-e3 Rd1-b1 Bc4-b3 Rb1-e1+ Ke3-d3 Re1-e2
Rh6-h2 b6-b5 Kd3-d4 Kc7-d6 Rh2-h6+ Kd6-c7 Rh6-h2) 0.00/36 0} Re1+ {(Rd1-e1+
Ke3-d3 Re1-b1 b4-b5 Rb1-f1 Kd3-e3 Rf1-e1+ Ke3-d4 Re1-d1+ Kd4-c3 Rd1-c1+
Kc3-d4) 0.00/38 2} 42. Kd4 {(Ke3-d4 Re1-d1+ Kd4-e3 Rd1-e1+ Ke3-d2 Re1-b1
Kd2-c3 Rb1-c1+ Kc3-d4) 0.00/39 1} Rd1+ {(Re1-d1+ Kd4-e3 Rd1-e1+ Ke3-d3
Re1-b1 b4-b5 Rb1-d1+ Kd3-c3 Rd1-c1+ Kc3-d4 Rc1-d1+ Kd4-e5 Rd1-e1+ Ke5-d4)
0.00/42 1} 43. Ke3 {(Kd4-e3 Rd1-e1+ Ke3-d3 Re1-d1+ Kd3-c2 Rd1-g1 Bc4-b5
Rg1-g2 Rh6-h7+ Kc7-d6 Rh7-d7+ Kd6-e5 Rd7-d2 g4-g3 f2xg3 Rg2xg3 a2-a4 Rg3-g2
Rd2xg2 Bf3xg2) 0.00/36 0} Re1+ {(Rd1-e1+ Ke3-d2 Re1-b1 Rh6-h7+ Kc7-d6
Bc4-b3 Rb1-b2+ Kd2-e3 Rb2-e2+ Ke3-f4 Re2-e4+ Kf4-g3 Re4xb4 Rh7-g7 Kd6-c5
Bb3-e6 Rb4-b2 Be6xg4 Bf3xg4 Rg7xg4) 0.00/38 2} 44. Kd2 {(Ke3-d2 Re1-b1
Rh6-h7+ Kc7-d6 Kd2-c3 Rb1-c1+ Kc3-d4 Rc1-d1+ Kd4-e3 Rd1-e1+ Ke3-d3 Re1-f1
Rh7-g7 Rf1xf2 Kd3-e3 Rf2-g2 Rg7-g6+ Kd6-c7 a2-a3 Rg2-c2 Bc4-e6 Rc2-c3+
Ke3-f4 Rc3xa3 Be6xg4 Bf3xg4 Kf4xg4) 0.00/36 1} Rb1 {(Re1-b1 Rh6-h7+ Kc7-d6
Bc4-b3 Rb1-b2+ Kd2-e3 Rb2-e2+ Ke3-f4 Re2-e4+ Kf4-g3 Re4xb4 Rh7-g7 Kd6-c5
Bb3-e6 Rb4-b2 Be6xg4 Bf3xg4 Rg7xg4) 0.00/47 1} 45. Rh7+ {(Rh6-h7+ Kc7-d6
Kd2-c3 Rb1-c1+ Kc3-d4 Rc1-d1+ Kd4-e3 Rd1-e1+ Ke3-d4) 0.00/37 0} Kd6
{(Kc7-d6 Kd2-c3 Rb1-c1+ Kc3-d4 Rc1-d1+ Kd4-c3) 0.00/41 2} 46. Rh6+
{(Rh7-h6+ Kd6-e5 Kd2-e3 Rb1xb4 Bc4-b3 Rb4-e4+ Ke3-d3 Ke5-f5 Rh6xb6 Re4-e2
Bb3-d1 Re2xf2 Bd1xf3 Rf2xf3+) 0.00/40 0} Ke5 {(Kd6-e5 Kd2-e3 Rb1xb4 Bc4-b3
Rb4-e4+ Ke3-d3 Ke5-f5 Bb3-d5 Re4-f4 Bd5xf3 Rf4xf3+ Kd3-e2 Rf3-a3 Rh6xb6)
0.00/46 1} 47. Ke3 {(Kd2-e3 Rb1xb4 Bc4-b3 Rb4-e4+ Ke3-d3 Ke5-f5 Rh6xb6
Re4-e2 Bb3-d1 Re2xf2 Bd1xf3 Rf2xf3+) 0.00/39 0} Rxb4 {(Rb1xb4 Bc4-b3
Rb4-e4+ Ke3-d3 Ke5-f5 Rh6xb6 Re4-e2 Rb6-b5+ Kf5-f4 Rb5-b4+ Kf4-g5 Bb3-d5
Re2xf2 Bd5xf3 Rf2xf3+) 0.00/47 1} 48. Bb3 {(Bc4-b3 Rb4-e4+ Ke3-d3 Ke5-f5
Rh6xb6 Re4-e2 Bb3-d1 Re2xf2 Bd1xf3 Rf2xf3+) 0.00/44 0} Re4+ {(Rb4-e4+
Ke3-d3 Ke5-f5 Rh6xb6 Re4-e2 Rb6-b5+ Kf5-f4 Rb5-b4+ Kf4-g5 Bb3-d5 Re2xf2
Bd5xf3 Rf2xf3+) 0.00/49 1} 49. Kd3 {(Ke3-d3 b6-b5 Rh6-e6+ Ke5-f5 Re6xe4
Bf3xe4+ Kd3-d4 Kf5-f4 Kd4-c5 Kf4-f3 Bb3-d1+ Kf3xf2) 0.00/44 1} Kf5 {(Ke5-f5
Rh6xb6 Re4-e2 Rb6-b5+ Kf5-f4 Rb5-b4+ Kf4-g5 Bb3-d5 Re2xf2 Bd5xf3 Rf2xf3+)
0.00/53 1} 50. Rxb6 {(Rh6xb6 Re4-e2 Rb6-b5+ Kf5-f4 Rb5-b4+ Kf4-g5 Bb3-d1
Re2xf2 Bd1xf3 Rf2xf3+) 0.00/52 0} Re2 {(Re4-e2 Rb6-b5+ Kf5-f4 Rb5-b4+
Kf4-g5 Bb3-d5 Re2xf2 Bd5xf3 Rf2xf3+) 0.00/56 2} 51. Rb5+ {(Rb6-b5+ Kf5-f4
Rb5-b4+ Kf4-g5 Bb3-d1 Re2xf2 Bd1xf3 Rf2xf3+) 0.00/50 0} Kf4 {(Kf5-f4
Rb5-b4+ Kf4-e5 Bb3-d1 Re2xf2 Bd1xf3 Rf2xf3+) 0.00/50 1} 52. Rb4+ {(Rb5-b4+
Kf4-g5 Bb3-d1 Re2xf2 Bd1xf3 Rf2xf3+) 0.00/50 0} Ke5 {(Kf4-e5 Bb3-d1 Re2xf2
Bd1xf3 Rf2xf3+) 0.00/59 1} 53. Bd1 {(Bb3-d1 Re2xf2 Bd1xf3 Rf2xf3+) 0.00/51
0} Rxf2 {(Re2xf2 Bd1xf3 Rf2xf3+) 0.00/58 4} 54. Bxf3 {(Bd1xf3 Rf2xf3+)
0.00/71 0} gxf3 {(g4xf3) 0.00/95 1} 55. Ke3 {(Kd3-e3 Rf2xa2 Rb4-b5+ Ke5-d6
Ke3xf3 Ra2-a1 Kf3-e3 Kd6-c7 Ke3-e4 Ra1-e1+ Ke4-d4 Kc7-d6 Rb5-g5 Re1-d1+
Kd4-e4 Rd1-f1 Rg5-g6+ Kd6-c5 Rg6-g2 Rf1-e1+ Ke4-d3 Kc5-d6 Rg2-g5 Re1-b1
Kd3-e4 Rb1-b4+ Ke4-f5 Rb4-b5+ Kf5-f4 Rb5xg5 Kf4xg5 Kd6-c5 Kg5-g4 Kc5-c4
Kg4-f5 Kc4-d5 Kf5-g5 Kd5-e4 Kg5-f6 Ke4-d4 Kf6-e6 Kd4-d3 Ke6-f5) 0.00/41 0}
Re2+ {(Rf2-e2+ Ke3xf3 Re2xa2 Rb4-g4 Ra2-a3+ Kf3-f2 Ke5-d5 Rg4-g5+ Kd5-d4
Kf2-g2 Kd4-e4 Rg5-g4+ Ke4-f5 Rg4-g8 Kf5-e4 Rg8-g5 Ra3-a2+ Kg2-g3 Ra2-a3+
Kg3-h4 Ke4-d4 Rg5-g2 Kd4-e4 Rg2-g1 Ra3-f3 Kh4-g4 Rf3-f2 Rg1-e1+ Ke4-d4
Kg4-h3 Rf2-b2 Re1-d1+ Kd4-e4 Rd1-g1 Rb2-b3+ Rg1-g3 Rb3xg3+ Kh3xg3 Ke4-e5
Kg3-g2 Ke5-e4 Kg2-f2 Ke4-d5 Kf2-e3 Kd5-c5 Ke3-d3 Kc5-d6 Kd3-d4 Kd6-c7
Kd4-e5) 0.00/47 1} 56. Kxf3 {(Ke3xf3 Re2xa2 Rb4-b5+ Ke5-f6 Kf3-e4 Kf6-e6
Rb5-b4 Ra2-a3 Rb4-b6+ Ke6-d7 Rb6-b5 Kd7-c6 Rb5-b1 Kc6-d6 Rb1-b5 Ra3-c3
Rb5-b1) 0.00/45 0} Rxa2 {(Re2xa2 Rb4-g4 Ra2-a3+ Kf3-f2 Ke5-d5 Rg4-g5+
Kd5-d4 Rg5-g7 Kd4-e4 Kf2-e2 Ra3-a2+ Ke2-e1 Ra2-a3 Rg7-g4+ Ke4-f3 Rg4-g8
Ra3-a2 Rg8-f8+ Kf3-e3 Rf8-e8+ Ke3-d3 Re8-d8+ Kd3-c3 Rd8-b8 Ra2-a1+ Ke1-f2
Kc3-c4 Kf2-e3 Kc4-c3 Rb8-c8+ Kc3-b4 Rc8-c6 Ra1-e1+ Ke3-d2 Re1-a1 Rc6-c1
Ra1-a3 Kd2-c2 Kb4-c5 Kc2-b1+ Kc5-b5 Rc1-g1 Kb5-c5) 0.00/53 1 Arena
Adjudication (Tablebases)} 1/2-1/2


Example after:
[Event "SF vs SF half q5"]
[Site "MAIN"]
[Date "2020.02.12"]
[Round "1"]
[White "SF 2"]
[Black "SF"]
[Result "1/2-1/2"]
[ECO "C27"]
[Opening "Vienna"]
[Time "03:34:57"]
[TimeControl "20+2"]
[Termination "adjudication"]
[PlyCount "112"]

1. e4 e5
2. Nc3 Nf6
3. Bc4 Nxe4
4. Qh5 Nd6
5. Bb3 Nc6
6. Nb5 g6
7. Qf3 f5
8. Qd5 {+1.17/22 1} Qf6 {-1.76/24 2}
9. Nxc7+ {+1.41/23 0} Kd8 {-1.90/24 2}
10. Nxa8 {+1.54/21 0} b6 {-1.83/25 1}
11. Qf3 {+1.50/22 1} Bb7 {-1.96/22 1}
12. h4 {+1.47/20 0} f4 {-1.90/25 4}
13. d3 {+1.44/23 2} Bh6 {-1.79/24 2}
14. Bd2 {+1.56/22 1} Re8 {-1.87/23 1}
15. O-O-O {+1.28/26 3} Nd4 {-1.89/26 1}
16. Qh3 {+1.20/25 1} Bxa8 {-1.69/24 1}
17. Nf3 {+1.43/23 0} Nxf3 {-1.67/25 2}
18. gxf3 {+1.16/23 1} Bc6 {-1.64/25 2}
19. c3 {+1.20/26 1} Bg7 {-1.59/24 2}
20. h5 {+1.30/22 1} Nf7 {-1.48/25 1}
21. hxg6 {+1.37/23 0} Ng5 {-1.34/28 2}
22. Qh4 {+1.17/26 1} hxg6 {-1.37/25 1}
23. d4 {+1.29/23 0} Bxf3 {-1.27/28 5}
24. dxe5 {+1.10/24 1} Rxe5 {-1.40/29 2}
25. Bxf4 {+1.09/25 1} Rc5 {-1.26/26 1}
26. Bxg5 {+0.96/26 2} Qxg5+ {-0.79/26 2}
27. Qxg5+ {+1.19/25 0} Rxg5 {-0.55/26 1}
28. Rxd7+ {+1.09/27 1} Kxd7 {-0.55/27 1}
29. Rh7 {+0.95/25 0} Rg1+ {-0.38/28 4}
30. Kd2 {+1.00/26 0} Kd6 {-0.25/27 4}
31. Rxg7 {+0.83/26 1} g5 {-0.22/26 0}
32. Rg6+ {+0.13/27 5} Kc7 {-0.23/27 1}
33. Rg7+ {+0.03/27 1} Kd6 {-0.18/29 3}
34. Bc4 {+0.09/27 0} a5 {-0.16/32 1}
35. Rg6+ {0.00/28 0} Kc7 {-0.16/35 2}
36. Rh6 {0.00/30 0} Ra1 {-0.09/31 1}
37. b4 {0.00/24 0} axb4 {0.00/33 1}
38. cxb4 {0.00/33 0} g4 {0.00/36 1}
39. Kc3 {0.00/31 0} Rc1+ {0.00/39 2}
40. Kd4 {0.00/35 1} Rd1+ {0.00/41 1}
41. Ke3 {0.00/36 0} Re1+ {0.00/38 2}
42. Kd4 {0.00/39 1} Rd1+ {0.00/42 1}
43. Ke3 {0.00/36 0} Re1+ {0.00/38 2}
44. Kd2 {0.00/36 1} Rb1 {0.00/47 1}
45. Rh7+ {0.00/37 0} Kd6 {0.00/41 2}
46. Rh6+ {0.00/40 0} Ke5 {0.00/46 1}
47. Ke3 {0.00/39 0} Rxb4 {0.00/47 1}
48. Bb3 {0.00/44 0} Re4+ {0.00/49 1}
49. Kd3 {0.00/44 1} Kf5 {0.00/53 1}
50. Rxb6 {0.00/52 0} Re2 {0.00/56 2}
51. Rb5+ {0.00/50 0} Kf4 {0.00/50 1}
52. Rb4+ {0.00/50 0} Ke5 {0.00/59 1}
53. Bd1 {0.00/51 0} Rxf2 {0.00/58 4}
54. Bxf3 {0.00/71 0} gxf3 {0.00/95 1}
55. Ke3 {0.00/41 0} Re2+ {0.00/47 1}
56. Kxf3 {0.00/45 0} Rxa2 {0.00/53 1 Arena Adjudication} 1/2-1/2

Re: Time handicap tournament. LC0 and SF11

Posted: Mon Feb 17, 2020 4:08 pm
by mmt
Thanks for looking! Definitely have to clean look at these time forfeit games. I can create cleaned-up versions of the pgn files without PV.
Guenther wrote:
Mon Feb 17, 2020 8:45 am
A first quick look also revealed that (as I supposed) that most of the times SF/2 wins vs. SF were only due to crappy openings starting with +2 or so.
A large book has its minuses. Since both sides are played, it shouldn't be too big of a deal for getting the right Elo (Ordo) difference.
Guenther wrote:
Mon Feb 17, 2020 8:45 am
Last not least, ofc Alyan was correct that using 20GB hash per program added extreme noise and errors too, because loading 20GB hash in a 10s or 5s game might already use most of the basetime for loading and slowing things down. (even 64 or 128MB would be enough here).
No, this is not the case with Arena. I can set up a 1s/0 tournament with a 20 GB cache and it works fine, whether the engines are loaded beforehand or not.
Guenther wrote:
Mon Feb 17, 2020 8:45 am
For this experiment cutechess-cli would be the only serious and scientific way to do it. (plus a much better, balanced and not too much plies opening file)
Yeah, I will try Cute Chess CLI. Any large book recommendations?

Re: Time handicap tournament. LC0 and SF11

Posted: Mon Feb 17, 2020 4:17 pm
by mmt
Guenther wrote:
Mon Feb 17, 2020 3:10 pm
At least it motivated me to create another regex macro ;-)

Below are the pgn files transformed into sth more readable and usable and size 1:10.
(I left the 700 'forfeits on time lines' in one match in)
Thanks. This one match should be discarded anyway because it's the one with mistaken time controls. I see a total of 3 other forfeits on time in all other matches.

Re: Time handicap tournament. LC0 and SF11

Posted: Tue Feb 18, 2020 12:58 pm
by Guenther
mmt wrote:
Mon Feb 17, 2020 4:08 pm
Thanks for looking! Definitely have to clean look at these time forfeit games. I can create cleaned-up versions of the pgn files without PV.
Guenther wrote:
Mon Feb 17, 2020 8:45 am
A first quick look also revealed that (as I supposed) that most of the times SF/2 wins vs. SF were only due to crappy openings starting with +2 or so.
A large book has its minuses. Since both sides are played, it shouldn't be too big of a deal for getting the right Elo (Ordo) difference.
This is wrong and will defeat the whole purpose of the test. Bad openings cannot be cured by playing them for both sides.
That is a widely spread illogical opinion. The only thing it does, is to help the suspected weaker program to push its score
further to equality.

Also long openings generally make those statistically tests unreliable for various reasons.
I noticed e.g. 120+ and 140+ games with early 3 time reps below move 30! in your pgn files. (same effect, pushing the weaker one towards equality)
Sometimes even directly after book end. IMO long books should be abolished at all for serious and statistical tests.
mmt wrote:
Mon Feb 17, 2020 4:08 pm
Guenther wrote:
Mon Feb 17, 2020 8:45 am
Last not least, ofc Alyan was correct that using 20GB hash per program added extreme noise and errors too, because loading 20GB hash in a 10s or 5s game might already use most of the basetime for loading and slowing things down. (even 64 or 128MB would be enough here).
No, this is not the case with Arena. I can set up a 1s/0 tournament with a 20 GB cache and it works fine, whether the engines are loaded beforehand or not.
Well, in one file there were seven unterminated games, which Arena replayed automatically for you, in some of them the second program may be
crashed, or didn't move at all for the first move after book end.
mmt wrote:
Mon Feb 17, 2020 4:08 pm
Guenther wrote:
Mon Feb 17, 2020 8:45 am
For this experiment cutechess-cli would be the only serious and scientific way to do it. (plus a much better, balanced and not too much plies opening file)
Yeah, I will try Cute Chess CLI. Any large book recommendations?
No, see above, I would use a 6-12 (at max!) plies opening file.

Moreover I have some doubts now how to set the tc at all. Most programs now have too clever time management and I noticed that
in crucial positions sometimes the program, which should use half of the time actually used more time than the other.

Just for curiosity I did a match myself until today between SF and SFx2 (half time) on my slow hardware with 1 cpu each, at a very fast tc with given time per move. 1move/0.5s vs. 1move/0.25s (128MB) in cutechess-cli with a 6 plies general book and the diff was around 160 rating points.
(Need to calculate average depth for midgame to compare)

Even here I find artifacts of assymmetric time usage and I am not sure how much noise this adds to the outcome.

There is also an effect I completely forgot (but it is very rare) and we talked about long ago here.
Sometimes it is even negative to see further than your opponent, because you see more and more how worse your position might become
and defend against something the other won't see at all and play suboptimal against the time handicapped program until it even wins.

This and the assymmetric time usage explains most of the wins the handicapped version can make against the non-handicapped one.
(plus bad book lines ofc)

Moreover there is contempt in SF too, which also will influence this test...