Lc0 v18 rc2 for 2080 Syzygy implementation

[Laskos]

The engine runs smoothly with new CUDA drivers.

The time control used is always 0.2s/move, Lc0 is on GTX 1060, SF8 is on 4 i7 cores.

First, now with both WDL and DTZ usage, Lc0 should convert any 6-men won positions at root with my 6-men Syzygy on SSD. Not so, on 6-men suite of hard wins at root, it missed 7 conversions out of 100. Here is the PGN, the problem seems to lie in some bug with 3-fold repetition:

[pgn] [Event "?"] [Site "?"] [Date "2018.10.01"] [Round "9"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [GameDuration "00:00:05"] [GameEndTime "2018-10-01T15:46:13.024 GTB Daylight Time"] [GameStartTime "2018-10-01T15:46:07.445 GTB Daylight Time"] [PlyCount "40"] [SetUp "1"] [TimeControl "0.2/move"] [FEN "3B4/8/8/8/8/4P1Q1/1p6/K1k5 w - - 0 1"] 1. Ka2 b1=Q+ {-128.51/16 0.20s} 2. Ka3 Qd3+ {-128.51/18 0.20s} 3. Kb4 Qxd8 {-128.51/17 0.20s} 4. Qe1+ Kb2 {-128.51/21 0.20s} 5. Qe2+ {+5.40/4 0.20s} 5. ... Ka1 {-128.51/21 0.20s} 6. e4 {+128.00/2 0.25s} 6. ... Qd6+ { -128.51/18 0.20s} 7. Kc3 {+4.03/4 0.20s} 7. ... Qa3+ {-128.51/16 0.20s} 8. Kd2 {+3.68/5 0.20s} 8. ... Qc1+ {-128.51/20 0.20s} 9. Kd3 Qa3+ { -128.51/21 0.20s} 10. Kd4 {+2.67/5 0.20s} 10. ... Qd6+ {-128.51/22 0.20s} 11. Ke3 {+3.67/6 0.20s} 11. ... Qh6+ {-128.51/22 0.20s} 12. Kf2 {+3.24/5 0.20s} 12. ... Qh2+ {-128.51/23 0.20s} 13. Ke1 {+3.61/5 0.20s} 13. ... Qg1+ {-128.51/26 0.20s} 14. Kd2 {+5.06/5 0.20s} 14. ... Qc1+ {-128.51/127 0.001s} 15. Kd3 Qa3+ {-128.51/127 0.001s} 16. Kd4 Qd6+ {-128.51/127 0.001s } 17. Ke3 Qh6+ {-128.51/127 0.001s} 18. Kd3 Qd6+ {-128.51/22 0.20s} 19. Kc2 {+2.72/5 0.21s} 19. ... Qc5+ {-128.51/16 0.20s} 20. Kd2 {+2.67/6 0.20s } 20. ... Qc1+ {-128.51/127 0.001s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.01"] [Round "27"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [GameDuration "00:00:09"] [GameEndTime "2018-10-01T15:51:09.546 GTB Daylight Time"] [GameStartTime "2018-10-01T15:51:00.047 GTB Daylight Time"] [PlyCount "63"] [SetUp "1"] [TimeControl "0.2/move"] [FEN "8/8/2B5/8/b7/1KP5/2Q5/k7 w - - 0 1"] 1. Ka3 Bxc2 {-128.51/33 0.20s} 2. c4 {+128.00/1 0.019s} 2. ... Bb1 { -128.51/36 0.20s} 3. c5 {+128.00/1 0.019s} 3. ... Be4 {-128.51/37 0.20s} 4. Bb5 Bd5 {-128.51/31 0.20s} 5. Bd3 Be4 {-128.51/31 0.20s} 6. Bb5 { +5.89/5 0.20s} 6. ... Bd5 {-128.51/57 0.20s} 7. Bd3 {+4.76/2 0.019s} 7. ... Be4 {-128.51/62 0.20s} 8. Bc4 Kb1 {-128.51/31 0.20s} 9. Kb3 Bh1 { -128.51/32 0.20s} 10. Kc3 {+6.05/4 0.20s} 10. ... Bc6 {-128.51/27 0.20s} 11. Be6 {+5.68/4 0.20s} 11. ... Be8 {-128.51/30 0.20s} 12. Bd5 {+7.42/4 0.20s} 12. ... Kc1 {-128.51/30 0.20s} 13. Bf3 {+7.10/5 0.20s} 13. ... Bd7 {-128.51/29 0.20s} 14. Be2 {+6.79/5 0.20s} 14. ... Bc6 {-128.51/32 0.20s} 15. Bg4 {+5.64/5 0.20s} 15. ... Kb1 {-128.51/30 0.20s} 16. Be6 {+5.16/5 0.021s} 16. ... Be8 {-128.51/60 0.20s} 17. Bd5 {+8.48/2 0.018s} 17. ... Kc1 {-128.51/62 0.20s} 18. Bb7 Bh5 {-128.51/28 0.20s} 19. Be4 {+10.15/4 0.20s} 19. ... Be8 {-128.51/34 0.20s} 20. Bd5 {+6.37/5 0.21s} 20. ... Kd1 {-128.51/31 0.20s} 21. Bc4 {+6.37/5 0.019s} 21. ... Bc6 {-128.51/31 0.20s} 22. Ba6 {+6.42/2 0.015s} 22. ... Bg2 {-128.51/25 0.20s} 23. Bb5 {+5.47/4 0.20s} 23. ... Kc1 {-128.51/31 0.20s} 24. Bd7 {+5.17/4 0.20s} 24. ... Bf3 {-128.51/26 0.20s} 25. Be6 {+4.71/4 0.20s} 25. ... Bc6 {-128.51/32 0.20s} 26. Bb3 {+4.16/4 0.20s} 26. ... Be8 {-128.51/32 0.20s} 27. Bd5 {+6.52/4 0.20s} 27. ... Kd1 {-128.51/127 0.001s} 28. Bc4 {+4.37/4 0.019s} 28. ... Bc6 {-128.51/127 0.001s} 29. Ba6 {+3.10/3 0.018s} 29. ... Bf3 {-128.51/34 0.20s} 30. Bb5 {+3.63/4 0.20s} 30. ... Bb7 {-128.51/38 0.20s} 31. Bd3 { +4.10/5 0.20s} 31. ... Bc6 {-128.51/34 0.20s} 32. Ba6 {0.00/2 0.022s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.01"] [Round "67"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [GameDuration "00:00:03"] [GameEndTime "2018-10-01T15:59:38.265 GTB Daylight Time"] [GameStartTime "2018-10-01T15:59:35.105 GTB Daylight Time"] [PlyCount "19"] [SetUp "1"] [TimeControl "0.2/move"] [FEN "8/8/1pk5/K7/8/8/Qr6/4Q3 w - - 0 1"] 1. Ka4 Rxa2+ {-128.51/34 0.20s} 2. Kb3 {+5.66/4 0.21s} 2. ... Ra7 { -128.51/35 0.20s} 3. Kc3 {+5.56/4 0.20s} 3. ... Ra3+ {-128.51/35 0.20s} 4. Kb4 {+7.09/5 0.21s} 4. ... Ra7 {-128.51/35 0.20s} 5. Kc3 {+6.42/2 0.020s} 5. ... Ra3+ {-128.51/127 0.002s} 6. Kd2 Ra2+ {-128.51/34 0.20s} 7. Kd3 { +5.17/4 0.23s} 7. ... Ra3+ {-128.51/34 0.20s} 8. Kc2 {+5.25/5 0.21s} 8. ... Ra2+ {-128.51/43 0.20s} 9. Kb3 {+5.39/4 0.20s} 9. ... Ra7 { -128.51/112 0.20s} 10. Kc3 {0.00/2 0.20s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.01"] [Round "73"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [GameDuration "00:00:09"] [GameEndTime "2018-10-01T16:01:16.205 GTB Daylight Time"] [GameStartTime "2018-10-01T16:01:06.643 GTB Daylight Time"] [PlyCount "65"] [SetUp "1"] [TimeControl "0.2/move"] [FEN "Q7/8/7q/8/2B5/3K4/n7/2k5 w - - 0 1"] 1. Qa3+ Kd1 {-128.51/29 0.20s} 2. Qa4+ Ke1 {-128.51/34 0.20s} 3. Qe8+ Kf2 {-128.51/33 0.20s} 4. Qe2+ Kg3 {-128.51/35 0.20s} 5. Qe1+ Kh2 {-128.51/34 0.20s} 6. Qe5+ {+1.81/5 0.20s} 6. ... Kg2 {-128.51/36 0.20s} 7. Bd5+ { +1.45/6 0.20s} 7. ... Kf2 {-128.51/32 0.20s} 8. Qe2+ {+2.20/6 0.047s} 8. ... Kg3 {-128.51/33 0.20s} 9. Qe1+ {+2.51/3 0.042s} 9. ... Kg4 { -128.51/31 0.20s} 10. Be6+ {+3.05/2 0.024s} 10. ... Kh5 {-128.51/34 0.20s} 11. Bf7+ {+1.41/5 0.20s} 11. ... Kg4 {-128.51/13 0.20s} 12. Be6+ {+1.28/5 0.042s} 12. ... Kh5 {-128.51/127 0.001s} 13. Qh1+ {+0.97/6 0.21s} 13. ... Kg6 {-128.51/27 0.20s} 14. Qe4+ {+0.78/5 0.086s} 14. ... Kg7 {-128.51/16 0.20s} 15. Qb7+ {+1.60/6 0.055s} 15. ... Kh8 {-128.51/18 0.20s} 16. Qb8+ { +1.74/5 0.21s} 16. ... Kg7 {-128.51/14 0.20s} 17. Qc7+ {+1.69/6 0.20s} 17. ... Kf6 {-128.51/30 0.20s} 18. Qd8+ {+1.68/6 0.20s} 18. ... Kg6 { -128.51/31 0.20s} 19. Qe8+ {+0.86/6 0.089s} 19. ... Kf6 {-128.51/31 0.20s} 20. Qf7+ {+1.05/6 0.20s} 20. ... Ke5 {-128.51/25 0.20s} 21. Qc7+ {+0.96/6 0.20s} 21. ... Kf6 {-128.51/127 0.001s} 22. Qf7+ {+0.50/4 0.10s} 22. ... Ke5 {-128.51/127 0.002s} 23. Qf5+ {+0.12/4 0.026s} 23. ... Kd6 {-128.51/28 0.20s} 24. Qd5+ {+0.23/5 0.032s} 24. ... Ke7 {-128.51/30 0.20s} 25. Qc5+ { +1.56/4 0.21s} 25. ... Ke8 {-128.51/28 0.20s} 26. Qc6+ {+1.01/5 0.20s} 26. ... Ke7 {-128.51/27 0.20s} 27. Qd7+ {+1.19/5 0.21s} 27. ... Kf6 { -128.51/13 0.20s} 28. Qd4+ {+0.98/5 0.20s} 28. ... Kg6 {-128.51/15 0.20s} 29. Qe4+ {+0.63/5 0.034s} 29. ... Kg7 {-128.51/127 0.001s} 30. Qb7+ { +0.23/4 0.081s} 30. ... Kh8 {-128.51/127 0.001s} 31. Qc8+ {+0.54/5 0.20s} 31. ... Kg7 {-128.51/20 0.20s} 32. Qd7+ {+1.12/4 0.20s} 32. ... Kf6 { -128.51/127 0.001s} 33. Qf7+ {0.00/2 0.094s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.01"] [Round "95"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [GameDuration "00:00:07"] [GameEndTime "2018-10-01T16:07:15.217 GTB Daylight Time"] [GameStartTime "2018-10-01T16:07:08.061 GTB Daylight Time"] [PlyCount "58"] [SetUp "1"] [TimeControl "0.2/move"] [FEN "8/RK6/3n4/3k4/2R5/4P3/8/8 w - - 0 1"] 1. Ka6 Nxc4 {-128.51/21 0.20s} 2. Rd7+ Ke5 {-128.51/25 0.20s} 3. Re7+ Kf6 {-128.51/27 0.20s} 4. Re8 Nd6 {-128.51/29 0.20s} 5. Rh8 {+2.18/5 0.20s} 5. ... Ke5 {-128.51/32 0.20s} 6. Rd8 {+6.01/4 0.020s} 6. ... Nc4 { -128.51/30 0.20s} 7. Re8+ {+3.08/3 0.016s} 7. ... Kd5 {-128.51/30 0.20s} 8. Rd8+ {+2.78/5 0.20s} 8. ... Ke5 {-128.51/127 0.002s} 9. Re8+ {0.00/4 0.020s} 9. ... Kd5 {-128.51/79 0.20s} 10. Ka7 {+1.50/3 0.020s} 10. ... Nd6 {-128.51/31 0.20s} 11. Rd8 {+4.31/2 0.016s} 11. ... Ke5 {-128.51/30 0.20s} 12. Kb8 {+4.88/5 0.20s} 12. ... Nf5 {-128.51/31 0.20s} 13. Re8+ {+7.02/5 0.020s} 13. ... Kd6 {-128.51/33 0.20s} 14. Kb7 {+7.65/5 0.018s} 14. ... Kd7 {-128.51/30 0.20s} 15. Re5 {+2.29/5 0.020s} 15. ... Nd6+ {-128.51/31 0.20s} 16. Kb8 {+3.79/5 0.20s} 16. ... Kc6 {-128.51/30 0.20s} 17. Re7 { +3.54/5 0.20s} 17. ... Nf5 {-128.51/22 0.20s} 18. Re8 {+3.87/5 0.20s} 18. ... Kd6 {-128.51/127 0.002s} 19. Kb7 {+4.09/4 0.001s} 19. ... Kd7 { -128.51/127 0.001s} 20. Re5 {+1.93/5 0.019s} 20. ... Nd6+ {-128.51/127 0.001s} 21. Ka7 {+2.71/3 0.020s} 21. ... Kc6 {-128.51/33 0.20s} 22. Kb8 { +3.90/5 0.20s} 22. ... Kd7 {-128.51/30 0.20s} 23. Ra5 {+3.58/4 0.20s} 23. ... Kc6 {-128.51/29 0.20s} 24. Re5 {+3.47/5 0.20s} 24. ... Kd7 { -128.51/127 0.002s} 25. Rg5 {+3.23/4 0.026s} 25. ... Kc6 {-128.51/31 0.20s } 26. Rg6 {+1.95/4 0.20s} 26. ... Kd5 {-128.51/22 0.20s} 27. Rg4 {+1.58/4 0.023s} 27. ... Ke5 {-128.51/26 0.20s} 28. Rg8 {+1.02/2 0.019s} 28. ... Nf5 {-128.51/28 0.20s} 29. Re8+ Kd6 {-128.51/127 0.001s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.01"] [Round "111"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [GameDuration "00:00:03"] [GameEndTime "2018-10-01T16:10:33.329 GTB Daylight Time"] [GameStartTime "2018-10-01T16:10:30.223 GTB Daylight Time"] [PlyCount "26"] [SetUp "1"] [TimeControl "0.2/move"] [FEN "Q5Q1/1r6/2k4q/8/8/8/8/1K6 w - - 0 1"] 1. Kc2 Qh2+ {-128.51/23 0.20s} 2. Kd3 Qd6+ {-128.51/12 0.20s} 3. Ke4 Qb4+ {-128.51/24 0.20s} 4. Kf3 Qc3+ {-128.51/22 0.20s} 5. Kg4 {+1.35/5 0.20s} 5. ... Qd4+ {-128.51/16 0.20s} 6. Kh3 {+1.41/5 0.026s} 6. ... Qe3+ { -128.51/26 0.20s} 7. Kh4 {+2.32/4 0.024s} 7. ... Qe1+ {-128.51/26 0.20s} 8. Qg3 {+1.83/2 0.016s} 8. ... Qe7+ {-128.51/27 0.20s} 9. Qg5 Qe1+ { -128.51/25 0.20s} 10. Qg3 {+0.88/5 0.20s} 10. ... Qe7+ {-128.51/127 0.002s } 11. Qg5 {+0.60/4 0.024s} 11. ... Qe1+ {-128.51/127 0.002s} 12. Kg4 { +0.49/5 0.21s} 12. ... Qd1+ {-128.51/25 0.20s} 13. Kh4 {+0.17/5 0.20s} 13. ... Qe1+ {-128.51/127 0.002s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.01"] [Round "195"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [GameDuration "00:00:08"] [GameEndTime "2018-10-01T16:34:58.900 GTB Daylight Time"] [GameStartTime "2018-10-01T16:34:50.496 GTB Daylight Time"] [PlyCount "59"] [SetUp "1"] [TimeControl "0.2/move"] [FEN "8/8/7q/8/8/qQ1K4/B7/2k5 w - - 0 1"] 1. Qxa3+ {+128.00/1 0.026s} 1. ... Kd1 {-128.51/32 0.20s} 2. Qa4+ {+2.48/6 0.20s} 2. ... Ke1 {-M98/33 0.20s} 3. Qe4+ {+2.37/8 0.20s} 3. ... Kf2 { -M72/34 0.20s} 4. Qe2+ {+2.03/8 0.048s} 4. ... Kg3 {-M72/35 0.20s} 5. Qe1+ {+2.47/7 0.022s} 5. ... Kh2 {-M68/37 0.20s} 6. Qe5+ {+2.51/7 0.20s} 6. ... Kg2 {-M62/38 0.20s} 7. Bd5+ {+2.35/7 0.20s} 7. ... Kf2 {-M56/38 0.20s} 8. Qe2+ {+2.76/6 0.025s} 8. ... Kg3 {-M52/40 0.20s} 9. Qe1+ {+2.29/6 0.021s} 9. ... Kg4 {-M50/41 0.20s} 10. Be6+ {+1.86/5 0.022s} 10. ... Kh5 {-M48/41 0.20s} 11. Bf7+ {+2.13/6 0.20s} 11. ... Kg4 {-128.51/13 0.20s} 12. Be6+ { +1.65/6 0.022s} 12. ... Kh5 {-128.51/127 0.001s} 13. Qe2+ {+1.41/6 0.20s} 13. ... Kg6 {-M46/42 0.20s} 14. Qe4+ {+1.81/6 0.020s} 14. ... Kg7 {-M44/41 0.20s} 15. Qb7+ {+1.58/5 0.019s} 15. ... Kh8 {-M42/42 0.20s} 16. Qa8+ { +2.34/6 0.20s} 16. ... Kg7 {-M40/43 0.20s} 17. Qg8+ {+1.61/7 0.20s} 17. ... Kf6 {-M38/43 0.20s} 18. Qd8+ {+1.94/6 0.20s} 18. ... Kg6 {-M40/40 0.20s} 19. Qe8+ {+1.26/6 0.030s} 19. ... Kf6 {-M38/45 0.20s} 20. Qd8+ { +1.88/5 0.20s} 20. ... Kg6 {-128.51/127 0.001s} 21. Qe8+ {+1.23/6 0.020s} 21. ... Kf6 {-128.51/127 0.001s} 22. Qf7+ {+1.77/5 0.001s} 22. ... Ke5 { -M36/44 0.20s} 23. Qc7+ {+1.38/5 0.20s} 23. ... Kf6 {-128.51/15 0.20s} 24. Qf7+ {+1.00/6 0.20s} 24. ... Ke5 {-128.51/127 0.001s} 25. Qf5+ {+0.76/6 0.022s} 25. ... Kd6 {-M34/45 0.20s} 26. Qd5+ {+0.72/6 0.021s} 26. ... Ke7 {-M32/45 0.20s} 27. Qc5+ {+1.44/4 0.20s} 27. ... Ke8 {-M34/41 0.20s} 28. Qc6+ {+1.65/6 0.20s} 28. ... Ke7 {-M32/45 0.20s} 29. Qc7+ {+1.31/5 0.20s} 29. ... Kf6 {-128.51/127 0.001s} 30. Qf7+ {0.00/2 0.031s, Draw by 3-fold repetition} 1/2-1/2 [/pgn]

Both engines had 6-men Syzygy and SF8 is known to convert all root TB positions. I didn't check the PGN thoroughly to see what exactly happens.



Second test: 6-men hard wins positions, Lc0 enabled with 5-men Syzygy against no-TBs Lc0 (both v18):

Code: Select all

Score of lc0_v18_Syzygy vs lc0_v18: 76 - 35 - 89  [0.603] 200
Elo difference: 72.25 +/- 36.09
Finished match

Not dramatically different from what I get with regular engines. One might expect that Lc0 would profit more from Syzygy because of its terrible end-play. Those are very close to root TB position, so WDL are not probed much, so third test, where WDL is probed a lot:



"Peasant Revolt" Endgame Chess
http://talkchess.com/forum3/viewtopic.p ... =0&t=65065

[d]1nn1k1n1/4p3/8/8/8/PPPPPPPP/8/4K3 w - - 0 1
This starting position (and a built opening suite from it) is very sensitive to TBs, Komodo gets a boost of some 75 Elo points due to Syzygy 6-men.

Lc0 Syzygy-6 against Lc0 no-TBs

Code: Select all

Score of lc0_v18_Syzygy vs lc0_v18: 67 - 37 - 96  [0.575] 200
Elo difference: 52.51 +/- 34.82
Finished match

Not very impressive, no more improvement than regular engines get from Syzygy. Maybe because there are few hits on TBs, speed being 1000 times lower for Lc0 compared to a regular engine.

[Laskos]

Forgot to mention, I use no adjudications at all, the games are played till the end.

[can00336]

Thanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe

[Laskos]

Thanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe

No, still misses wins as 3-fold rep. at steady rate.

[crem]

Thanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe

No, still misses wins as 3-fold rep. at steady rate.

Hi Laskos,

We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe

Thanks!

[Laskos]

Thanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe

No, still misses wins as 3-fold rep. at steady rate.

Hi Laskos,

We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe

Thanks!

In an hour or two I will post the results on 100 hard 6-men positions.
With the original v18rc2 2080 version, it misses wins due to 3-fold repetition also at much longer 2.0s/move time control, and on easy 6-men wins too, at roughly the same rate, so it's a pretty serious bug.

[Laskos]

Thanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe

No, still misses wins as 3-fold rep. at steady rate.

Hi Laskos,

We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe

Thanks!

Sill misses by 3-fold, seems at lower rate, only 3 misses out of 100 positions. Time control for Lc0 is 0.5s/move, but as I already wrote, it seems to not matter, and even on easy 6-men wins it misses at pretty much the same rate. Here is the PGN of misses:

[pgn] [Event "?"] [Site "?"] [Date "2018.10.02"] [Round "49"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [BlackTimeControl "0.2/move"] [GameDuration "00:00:04"] [GameEndTime "2018-10-02T10:33:45.395 GTB Daylight Time"] [GameStartTime "2018-10-02T10:33:41.026 GTB Daylight Time"] [PlyCount "20"] [SetUp "1"] [WhiteTimeControl "0.5/move"] [FEN "8/8/1pk5/K7/8/8/Qr6/4Q3 w - - 0 1"] 1. Ka4 Rxa2+ {-128.51/35 0.20s} 2. Kb3 {+7.20/5 0.50s} 2. ... Ra6 { -128.51/31 0.20s} 3. Kb4 {+7.91/5 0.49s} 3. ... Ra7 {-128.51/33 0.20s} 4. Kc3 {+5.36/4 0.027s} 4. ... Ra3+ {-128.51/33 0.20s} 5. Kb4 {+8.56/5 0.50s} 5. ... Ra7 {-128.51/127 0.002s} 6. Kc3 {+4.75/3 0.022s} 6. ... Ra3+ { -128.51/127 0.001s} 7. Kd2 {+5.06/2 0.016s} 7. ... Ra2+ {-128.51/30 0.20s} 8. Kd3 {+5.39/5 0.50s} 8. ... Ra3+ {-128.51/34 0.20s} 9. Kc2 {+5.71/6 0.52s} 9. ... Ra2+ {-128.51/38 0.20s} 10. Kc3 {+0.63/3 0.041s} 10. ... Ra3+ {-128.51/127 0.001s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.02"] [Round "145"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [BlackTimeControl "0.2/move"] [GameDuration "00:01:27"] [GameEndTime "2018-10-02T11:17:06.373 GTB Daylight Time"] [GameStartTime "2018-10-02T11:15:39.176 GTB Daylight Time"] [PlyCount "270"] [SetUp "1"] [WhiteTimeControl "0.5/move"] [FEN "8/8/8/Q7/8/8/1Pp1P3/1k1K4 w - - 0 1"] 1. Kd2 c1=Q+ {-128.51/15 0.20s} 2. Kd3 Qc2+ {-128.51/17 0.20s} 3. Ke3 Qc1+ {-128.51/19 0.20s} 4. Ke4 {+4.30/5 0.44s} 4. ... Qc4+ {-128.51/15 0.20s} 5. Kf3 {+4.44/6 0.047s} 5. ... Qf7+ {-128.51/16 0.20s} 6. Kg3 {+3.85/6 0.50s} 6. ... Qg7+ {-128.51/17 0.20s} 7. Kf4 {+3.88/6 0.50s} 7. ... Qd4+ { -128.51/18 0.20s} 8. Kf3 {+4.42/7 0.096s} 8. ... Qf6+ {-128.51/20 0.20s} 9. Ke4 {+5.22/6 0.54s} 9. ... Qxb2 {-128.51/20 0.20s} 10. Qa6 Kc1 { -128.51/22 0.20s} 11. Kd5 Kd1 {-128.51/18 0.20s} 12. Qd3+ Ke1 {-128.51/22 0.20s} 13. e4 {+128.00/2 0.46s} 13. ... Qb7+ {-128.51/20 0.20s} 14. Ke6 { +3.34/5 0.50s} 14. ... Qc6+ {-128.51/21 0.20s} 15. Kf7 {+3.44/6 0.52s} 15. ... Qb7+ {-128.51/20 0.20s} 16. Kg6 {+3.67/6 0.50s} 16. ... Qc6+ { -128.51/21 0.20s} 17. Kg5 {+3.75/6 0.47s} 17. ... Qc5+ {-128.51/21 0.20s} 18. Qd5 {+7.17/5 0.51s} 18. ... Qe7+ {-128.51/22 0.20s} 19. Kg6 {+6.02/6 0.50s} 19. ... Qe8+ {-128.51/21 0.20s} 20. Kf6 {+4.76/6 0.50s} 20. ... Qf8+ {-128.51/19 0.20s} 21. Kg5 {+4.43/6 0.51s} 21. ... Qe7+ {-128.51/41 0.20s} 22. Kg4 {+4.01/6 0.083s} 22. ... Qg7+ {-128.51/19 0.20s} 23. Kf5 { +4.55/6 0.50s} 23. ... Qf8+ {-128.51/19 0.20s} 24. Ke5 {+4.02/6 0.50s} 24. ... Qh8+ {-128.51/17 0.20s} 25. Ke6 {+4.58/5 0.47s} 25. ... Qh6+ { -128.51/22 0.20s} 26. Kd7 {+6.04/6 0.50s} 26. ... Qg7+ {-128.51/21 0.20s} 27. Kc8 {+6.48/6 0.50s} 27. ... Qc3+ {-128.51/23 0.20s} 28. Kb7 {+6.52/6 0.50s} 28. ... Qg7+ {-128.51/22 0.20s} 29. Ka6 {+7.96/6 0.54s} 29. ... Qf6+ {-128.51/19 0.20s} 30. Kb5 {+7.47/6 0.51s} 30. ... Qb2+ {-128.51/21 0.20s} 31. Ka4 {+7.65/5 0.48s} 31. ... Qc2+ {-128.51/17 0.20s} 32. Ka5 { +8.57/6 0.50s} 32. ... Qc3+ {-128.51/23 0.20s} 33. Ka4 {+6.60/6 0.51s} 33. ... Qc2+ {-128.51/39 0.20s} 34. Kb5 {+4.34/7 0.058s} 34. ... Qb2+ { -128.51/127 0.002s} 35. Kc6 {+2.17/3 0.036s} 35. ... Qh8 {-128.51/16 0.20s } 36. Qe6 {+4.63/5 0.51s} 36. ... Qc3+ {-128.51/19 0.20s} 37. Kd7 {+4.47/6 0.51s} 37. ... Qg7+ {-128.51/19 0.20s} 38. Kd6 {+4.51/7 0.50s} 38. ... Qd4+ {-128.51/20 0.20s} 39. Qd5 {+3.89/6 0.57s} 39. ... Qf6+ {-128.51/24 0.20s} 40. Kc7 {+3.61/6 0.56s} 40. ... Qc3+ {-128.51/22 0.20s} 41. Kb8 { +3.29/6 0.50s} 41. ... Qb2+ {-128.51/24 0.20s} 42. Kc8 {+3.22/6 0.50s} 42. ... Qc3+ {-128.51/48 0.20s} 43. Kd7 {+2.39/7 0.12s} 43. ... Qg7+ { -128.51/127 0.002s} 44. Kc6 {+1.70/4 0.069s} 44. ... Qh8 {-128.51/59 0.20s } 45. Qf5 {+1.92/2 0.026s} 45. ... Qa8+ {-128.51/16 0.20s} 46. Kb5 { +2.54/5 0.50s} 46. ... Qb7+ {-128.51/20 0.20s} 47. Kc5 {+1.57/6 0.078s} 47. ... Qc7+ {-128.51/19 0.20s} 48. Kd5 {+1.44/5 0.026s} 48. ... Qb7+ { -128.51/19 0.20s} 49. Ke6 {+1.83/6 0.46s} 49. ... Qb3+ {-128.51/19 0.20s} 50. Kf6 {+2.35/6 0.50s} 50. ... Qb6+ {-128.51/20 0.20s} 51. Kg5 {+2.17/6 0.47s} 51. ... Qd8+ {-128.51/18 0.20s} 52. Qf6 {+1.94/6 0.048s} 52. ... Qd2+ {-128.51/17 0.20s} 53. Qf4 {+1.32/6 0.022s} 53. ... Qd8+ {-128.51/18 0.20s} 54. Kh5 {+1.13/5 0.034s} 54. ... Qe8+ {-128.51/19 0.20s} 55. Kh4 { +1.07/4 0.026s} 55. ... Qd8+ {-128.51/20 0.20s} 56. Kh3 {+1.42/4 0.026s} 56. ... Qd7+ {-128.51/21 0.20s} 57. Kh2 {+0.77/2 0.018s} 57. ... Qh7+ { -128.51/20 0.20s} 58. Kg2 Qb7 {-128.51/22 0.20s} 59. Kf3 Qc6 {-128.51/20 0.20s} 60. Qg5 Qc3+ {-128.51/15 0.20s} 61. Qe3+ Kd1 {-128.51/17 0.20s} 62. e5 {+128.00/1 0.061s} 62. ... Qc6+ {-128.51/15 0.20s} 63. Qe4 {+12.25/5 0.45s} 63. ... Qe6 {-128.51/15 0.20s} 64. Kf4 {+10.02/5 0.50s} 64. ... Qf7+ {-128.51/16 0.20s} 65. Qf5 {+15.41/5 0.54s} 65. ... Qc7 {-128.51/17 0.20s} 66. Kg5 {+14.04/5 0.51s} 66. ... Qd8+ {-128.51/17 0.20s} 67. Qf6 { +19.64/5 0.50s} 67. ... Qd2+ {-128.51/15 0.20s} 68. Kg6 {+15.87/5 0.50s} 68. ... Qg2+ {-128.51/15 0.20s} 69. Kf7 {+13.02/5 0.46s} 69. ... Qb7+ { -128.51/18 0.20s} 70. Qe7 {+12.94/5 0.52s} 70. ... Qf3+ {-128.51/17 0.20s} 71. Ke8 {+16.37/5 0.50s} 71. ... Qh5+ {-128.51/18 0.20s} 72. Kd8 {+23.68/5 0.50s} 72. ... Qh1 {-128.51/20 0.20s} 73. Qd6+ {+27.20/4 0.50s} 73. ... Ke2 {-128.51/21 0.20s} 74. e6 {+128.00/2 0.50s} 74. ... Qa8+ {-128.51/21 0.20s} 75. Kd7 {+13.64/5 0.50s} 75. ... Qa4+ {-128.51/19 0.20s} 76. Qc6 { +16.29/5 0.50s} 76. ... Qd4+ {-128.51/18 0.20s} 77. Ke8 {+14.46/5 0.50s} 77. ... Qh8+ {-128.51/15 0.20s} 78. Kd7 {+12.86/6 0.49s} 78. ... Qd4+ { -128.51/40 0.20s} 79. Kc8 {+12.21/5 0.51s} 79. ... Qh8+ {-128.51/17 0.20s} 80. Kc7 {+12.02/5 0.50s} 80. ... Qg7+ {-128.51/16 0.20s} 81. Qd7 {+13.17/5 0.54s} 81. ... Qc3+ {-128.51/17 0.20s} 82. Kb6 {+16.91/5 0.50s} 82. ... Qe3+ {-128.51/18 0.20s} 83. Kb7 {+15.93/5 0.50s} 83. ... Qb3+ {-128.51/18 0.20s} 84. Kc7 {+12.30/5 0.50s} 84. ... Qc3+ {-128.51/42 0.20s} 85. Qc6 { +6.62/6 0.050s} 85. ... Qg7+ {-128.51/42 0.20s} 86. Kb6 {+10.55/3 0.024s} 86. ... Qd4+ {-128.51/16 0.20s} 87. Qc5 {+15.59/5 0.55s} 87. ... Qf6 { -128.51/17 0.20s} 88. Qd6 {+17.49/5 0.50s} 88. ... Qb2+ {-128.51/17 0.20s} 89. Kc7 {+19.47/5 0.51s} 89. ... Qc2+ {-128.51/17 0.20s} 90. Kd8 {+25.03/4 0.50s} 90. ... Qe4 {-128.51/17 0.20s} 91. e7 {+128.00/2 0.52s} 91. ... Qa8+ {-128.51/17 0.20s} 92. Kd7 {+18.34/5 0.50s} 92. ... Qb7+ {-128.51/13 0.20s} 93. Ke6 {+19.35/5 0.50s} 93. ... Qc8+ {-128.51/14 0.20s} 94. Kf7 { +17.35/5 0.50s} 94. ... Qf5+ {-128.51/13 0.20s} 95. Qf6 {+26.58/5 0.50s} 95. ... Qh5+ {-M112/14 0.20s} 96. Kf8 {+30.26/5 0.53s} 96. ... Qc5 { -128.51/17 0.20s} 97. Kg7 {+22.48/5 0.50s} 97. ... Qc7 {-128.51/13 0.20s} 98. Kf8 {+26.23/4 0.51s} 98. ... Qc5 {-128.51/61 0.20s} 99. Kf7 {+12.66/2 0.026s} 99. ... Qh5+ {-128.51/60 0.20s} 100. Kg7 {+7.38/3 0.026s} 100. ... Qg4+ {-M70/17 0.20s} 101. Kf8 {+28.90/4 0.51s} 101. ... Qb4 {-M60/20 0.20s} 102. Kf7 {+27.49/5 0.51s} 102. ... Qb7 {-128.51/15 0.20s} 103. Qe6+ {+15.65/4 0.51s} 103. ... Kd2 {-128.51/15 0.20s} 104. Kg6 {+20.09/4 0.49s} 104. ... Qg2+ {-128.51/16 0.20s} 105. Kh7 {+26.29/5 0.50s} 105. ... Qb7 { -128.51/15 0.20s} 106. Kh6 {+21.40/4 0.52s} 106. ... Qh1+ {-128.51/14 0.20s} 107. Kg7 {+19.46/5 0.50s} 107. ... Qg1+ {-128.51/14 0.20s} 108. Qg6 {+15.46/5 0.50s} 108. ... Qd4+ {-128.51/16 0.20s} 109. Qf6 {+18.71/5 0.48s } 109. ... Qg4+ {-128.51/17 0.20s} 110. Kf8 {+21.29/4 0.47s} 110. ... Qb4 {-128.51/17 0.20s} 111. Qe5 {+19.28/5 0.51s} 111. ... Kc2 {-128.51/18 0.20s} 112. Kf7 {+14.70/4 0.52s} 112. ... Qc4+ {-128.51/17 0.20s} 113. Qe6 {+16.22/5 0.50s} 113. ... Qf4+ {-128.51/15 0.20s} 114. Qf6 {+20.87/5 0.50s } 114. ... Qc7 {-128.51/18 0.20s} 115. Kf8 {+27.60/4 0.51s} 115. ... Qc5 { -128.51/19 0.20s} 116. Kg7 {+25.79/4 0.51s} 116. ... Qc7 {-128.51/19 0.20s } 117. Kh6 {+16.18/4 0.53s} 117. ... Qh2+ {-128.51/14 0.20s} 118. Kg7 { +14.99/5 0.47s} 118. ... Qc7 {-128.51/54 0.20s} 119. Kf7 {+7.68/6 0.15s} 119. ... Kd3 {-128.51/16 0.20s} 120. Kf8 {+18.85/4 0.51s} 120. ... Qc5 { -128.51/15 0.20s} 121. Kg7 {+13.07/5 0.51s} 121. ... Qc7 {-128.51/16 0.20s } 122. Kg6 {+8.85/4 0.53s} 122. ... Qg3+ {-M82/15 0.20s} 123. Kf7 { +10.67/5 0.51s} 123. ... Qc7 {-128.51/67 0.20s} 124. Qa6+ {+5.65/6 0.51s} 124. ... Kd4 {-128.51/15 0.20s} 125. Qa1+ {+5.98/5 0.54s} 125. ... Ke3 { -128.51/16 0.20s} 126. Qa3+ {+4.64/5 0.55s} 126. ... Ke2 {-128.51/17 0.20s } 127. Qa6+ {+3.75/5 0.51s} 127. ... Kf2 {-128.51/15 0.20s} 128. Qf6+ { +3.52/5 0.53s} 128. ... Ke1 {-M76/19 0.20s} 129. Kf8 {+4.14/5 0.51s} 129. ... Qc5 {-M74/21 0.20s} 130. Kf7 {+3.84/5 0.50s} 130. ... Qc7 { -128.51/51 0.20s} 131. Qd4 {+1.24/5 0.037s} 131. ... Qb7 {-M60/21 0.20s} 132. Qe5+ {+2.01/5 0.53s} 132. ... Kd1 {-128.51/15 0.20s} 133. Qa1+ { +1.71/5 0.50s} 133. ... Ke2 {-128.51/16 0.20s} 134. Qd4 {+1.25/5 0.031s} 134. ... Qf3+ {-128.51/27 0.20s} 135. Qf6 Qh5+ {-128.51/127 0.001s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.02"] [Round "163"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [BlackTimeControl "0.2/move"] [GameDuration "00:00:07"] [GameEndTime "2018-10-02T11:23:43.490 GTB Daylight Time"] [GameStartTime "2018-10-02T11:23:36.381 GTB Daylight Time"] [PlyCount "35"] [SetUp "1"] [WhiteTimeControl "0.5/move"] [FEN "b7/8/6B1/3K4/8/8/2P5/Bk6 w - - 0 1"] 1. Kd4 Bc6 {-128.51/24 0.20s} 2. Kc5 Ba4 {-128.51/26 0.20s} 3. c4+ { +128.00/1 0.052s} 3. ... Kxa1 {-128.51/32 0.20s} 4. Kb4 Bc6 {-128.51/36 0.20s} 5. Ka3 Bh1 {-128.51/38 0.20s} 6. c5 {+128.00/2 0.42s} 6. ... Bg2 { -128.51/38 0.20s} 7. Bf7 {+5.56/5 0.47s} 7. ... Bc6 {-128.51/34 0.20s} 8. Kb3 {+6.24/5 0.49s} 8. ... Kb1 {-128.51/37 0.20s} 9. Kc3 {+5.66/6 0.47s} 9. ... Kc1 {-128.51/34 0.20s} 10. Bh5 {+5.79/6 0.50s} 10. ... Kb1 { -128.51/26 0.20s} 11. Bf7 {+5.32/5 0.039s} 11. ... Kc1 {-128.51/55 0.20s} 12. Bc4 {+4.86/5 0.031s} 12. ... Kd1 {-128.51/31 0.20s} 13. Ba6 {+5.76/2 0.018s} 13. ... Bg2 {-128.51/32 0.20s} 14. Bd3 {+6.96/5 0.50s} 14. ... Bc6 {-128.51/34 0.20s} 15. Ba6 {+5.30/4 0.021s} 15. ... Bg2 {-128.51/62 0.20s} 16. Bb5 {+4.25/4 0.024s} 16. ... Bf3 {-128.51/30 0.20s} 17. Bd3 {+5.92/5 0.52s} 17. ... Bc6 {-128.51/95 0.20s} 18. Ba6 {0.00/2 0.025s, Draw by 3-fold repetition} 1/2-1/2 [/pgn]

[George Tsavdaris]

In an hour or two I will post the results on 100 hard 6-men positions.

Can you give me or post here your 100 hard 6-men positions?
I know that they are from Kirill's 361 longest 6-men TBs but i want to see which ones you are using.....

[Laskos]

In an hour or two I will post the results on 100 hard 6-men positions.

Can you give me or post here your 100 hard 6-men positions?
I know that they are from Kirill's 361 longest 6-men TBs but i want to see which ones you are using.....

Kirill longest 6-men TB wins are according to Nalimov's TBs, and many are actually draws. I will send my file of positions via PM, forgot to do it in the morning.

[jkiliani]

Thanks for finding a bug in the implementation. A proposed fix is available here: https://github.com/LeelaChessZero/lc0/pull/408
You can download a CUDA binary to try the fix here: https://ci.appveyor.com/api/buildjobs/g ... %2Flc0.exe

No, still misses wins as 3-fold rep. at steady rate.

Hi Laskos,

We have a fix for that https://github.com/LeelaChessZero/lc0/pull/410, which we have no way to check currently. Would it be possible for you to test it? The deadline for CCCC binary submission is today, and we'll have to release v0.18.1 today with a fix.
As a last resort we can push this patch blindly, but it would be nice to have it tested.
The CUDA build of Lc0 with this patch is https://ci.appveyor.com/api/buildjobs/3 ... %2Flc0.exe

Thanks!

Sill misses by 3-fold, seems at lower rate, only 3 misses out of 100 positions. Time control for Lc0 is 0.5s/move, but as I already wrote, it seems to not matter, and even on easy 6-men wins it misses at pretty much the same rate. Here is the PGN of misses:

[pgn] [Event "?"] [Site "?"] [Date "2018.10.02"] [Round "49"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [BlackTimeControl "0.2/move"] [GameDuration "00:00:04"] [GameEndTime "2018-10-02T10:33:45.395 GTB Daylight Time"] [GameStartTime "2018-10-02T10:33:41.026 GTB Daylight Time"] [PlyCount "20"] [SetUp "1"] [WhiteTimeControl "0.5/move"] [FEN "8/8/1pk5/K7/8/8/Qr6/4Q3 w - - 0 1"] 1. Ka4 Rxa2+ {-128.51/35 0.20s} 2. Kb3 {+7.20/5 0.50s} 2. ... Ra6 { -128.51/31 0.20s} 3. Kb4 {+7.91/5 0.49s} 3. ... Ra7 {-128.51/33 0.20s} 4. Kc3 {+5.36/4 0.027s} 4. ... Ra3+ {-128.51/33 0.20s} 5. Kb4 {+8.56/5 0.50s} 5. ... Ra7 {-128.51/127 0.002s} 6. Kc3 {+4.75/3 0.022s} 6. ... Ra3+ { -128.51/127 0.001s} 7. Kd2 {+5.06/2 0.016s} 7. ... Ra2+ {-128.51/30 0.20s} 8. Kd3 {+5.39/5 0.50s} 8. ... Ra3+ {-128.51/34 0.20s} 9. Kc2 {+5.71/6 0.52s} 9. ... Ra2+ {-128.51/38 0.20s} 10. Kc3 {+0.63/3 0.041s} 10. ... Ra3+ {-128.51/127 0.001s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.02"] [Round "145"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [BlackTimeControl "0.2/move"] [GameDuration "00:01:27"] [GameEndTime "2018-10-02T11:17:06.373 GTB Daylight Time"] [GameStartTime "2018-10-02T11:15:39.176 GTB Daylight Time"] [PlyCount "270"] [SetUp "1"] [WhiteTimeControl "0.5/move"] [FEN "8/8/8/Q7/8/8/1Pp1P3/1k1K4 w - - 0 1"] 1. Kd2 c1=Q+ {-128.51/15 0.20s} 2. Kd3 Qc2+ {-128.51/17 0.20s} 3. Ke3 Qc1+ {-128.51/19 0.20s} 4. Ke4 {+4.30/5 0.44s} 4. ... Qc4+ {-128.51/15 0.20s} 5. Kf3 {+4.44/6 0.047s} 5. ... Qf7+ {-128.51/16 0.20s} 6. Kg3 {+3.85/6 0.50s} 6. ... Qg7+ {-128.51/17 0.20s} 7. Kf4 {+3.88/6 0.50s} 7. ... Qd4+ { -128.51/18 0.20s} 8. Kf3 {+4.42/7 0.096s} 8. ... Qf6+ {-128.51/20 0.20s} 9. Ke4 {+5.22/6 0.54s} 9. ... Qxb2 {-128.51/20 0.20s} 10. Qa6 Kc1 { -128.51/22 0.20s} 11. Kd5 Kd1 {-128.51/18 0.20s} 12. Qd3+ Ke1 {-128.51/22 0.20s} 13. e4 {+128.00/2 0.46s} 13. ... Qb7+ {-128.51/20 0.20s} 14. Ke6 { +3.34/5 0.50s} 14. ... Qc6+ {-128.51/21 0.20s} 15. Kf7 {+3.44/6 0.52s} 15. ... Qb7+ {-128.51/20 0.20s} 16. Kg6 {+3.67/6 0.50s} 16. ... Qc6+ { -128.51/21 0.20s} 17. Kg5 {+3.75/6 0.47s} 17. ... Qc5+ {-128.51/21 0.20s} 18. Qd5 {+7.17/5 0.51s} 18. ... Qe7+ {-128.51/22 0.20s} 19. Kg6 {+6.02/6 0.50s} 19. ... Qe8+ {-128.51/21 0.20s} 20. Kf6 {+4.76/6 0.50s} 20. ... Qf8+ {-128.51/19 0.20s} 21. Kg5 {+4.43/6 0.51s} 21. ... Qe7+ {-128.51/41 0.20s} 22. Kg4 {+4.01/6 0.083s} 22. ... Qg7+ {-128.51/19 0.20s} 23. Kf5 { +4.55/6 0.50s} 23. ... Qf8+ {-128.51/19 0.20s} 24. Ke5 {+4.02/6 0.50s} 24. ... Qh8+ {-128.51/17 0.20s} 25. Ke6 {+4.58/5 0.47s} 25. ... Qh6+ { -128.51/22 0.20s} 26. Kd7 {+6.04/6 0.50s} 26. ... Qg7+ {-128.51/21 0.20s} 27. Kc8 {+6.48/6 0.50s} 27. ... Qc3+ {-128.51/23 0.20s} 28. Kb7 {+6.52/6 0.50s} 28. ... Qg7+ {-128.51/22 0.20s} 29. Ka6 {+7.96/6 0.54s} 29. ... Qf6+ {-128.51/19 0.20s} 30. Kb5 {+7.47/6 0.51s} 30. ... Qb2+ {-128.51/21 0.20s} 31. Ka4 {+7.65/5 0.48s} 31. ... Qc2+ {-128.51/17 0.20s} 32. Ka5 { +8.57/6 0.50s} 32. ... Qc3+ {-128.51/23 0.20s} 33. Ka4 {+6.60/6 0.51s} 33. ... Qc2+ {-128.51/39 0.20s} 34. Kb5 {+4.34/7 0.058s} 34. ... Qb2+ { -128.51/127 0.002s} 35. Kc6 {+2.17/3 0.036s} 35. ... Qh8 {-128.51/16 0.20s } 36. Qe6 {+4.63/5 0.51s} 36. ... Qc3+ {-128.51/19 0.20s} 37. Kd7 {+4.47/6 0.51s} 37. ... Qg7+ {-128.51/19 0.20s} 38. Kd6 {+4.51/7 0.50s} 38. ... Qd4+ {-128.51/20 0.20s} 39. Qd5 {+3.89/6 0.57s} 39. ... Qf6+ {-128.51/24 0.20s} 40. Kc7 {+3.61/6 0.56s} 40. ... Qc3+ {-128.51/22 0.20s} 41. Kb8 { +3.29/6 0.50s} 41. ... Qb2+ {-128.51/24 0.20s} 42. Kc8 {+3.22/6 0.50s} 42. ... Qc3+ {-128.51/48 0.20s} 43. Kd7 {+2.39/7 0.12s} 43. ... Qg7+ { -128.51/127 0.002s} 44. Kc6 {+1.70/4 0.069s} 44. ... Qh8 {-128.51/59 0.20s } 45. Qf5 {+1.92/2 0.026s} 45. ... Qa8+ {-128.51/16 0.20s} 46. Kb5 { +2.54/5 0.50s} 46. ... Qb7+ {-128.51/20 0.20s} 47. Kc5 {+1.57/6 0.078s} 47. ... Qc7+ {-128.51/19 0.20s} 48. Kd5 {+1.44/5 0.026s} 48. ... Qb7+ { -128.51/19 0.20s} 49. Ke6 {+1.83/6 0.46s} 49. ... Qb3+ {-128.51/19 0.20s} 50. Kf6 {+2.35/6 0.50s} 50. ... Qb6+ {-128.51/20 0.20s} 51. Kg5 {+2.17/6 0.47s} 51. ... Qd8+ {-128.51/18 0.20s} 52. Qf6 {+1.94/6 0.048s} 52. ... Qd2+ {-128.51/17 0.20s} 53. Qf4 {+1.32/6 0.022s} 53. ... Qd8+ {-128.51/18 0.20s} 54. Kh5 {+1.13/5 0.034s} 54. ... Qe8+ {-128.51/19 0.20s} 55. Kh4 { +1.07/4 0.026s} 55. ... Qd8+ {-128.51/20 0.20s} 56. Kh3 {+1.42/4 0.026s} 56. ... Qd7+ {-128.51/21 0.20s} 57. Kh2 {+0.77/2 0.018s} 57. ... Qh7+ { -128.51/20 0.20s} 58. Kg2 Qb7 {-128.51/22 0.20s} 59. Kf3 Qc6 {-128.51/20 0.20s} 60. Qg5 Qc3+ {-128.51/15 0.20s} 61. Qe3+ Kd1 {-128.51/17 0.20s} 62. e5 {+128.00/1 0.061s} 62. ... Qc6+ {-128.51/15 0.20s} 63. Qe4 {+12.25/5 0.45s} 63. ... Qe6 {-128.51/15 0.20s} 64. Kf4 {+10.02/5 0.50s} 64. ... Qf7+ {-128.51/16 0.20s} 65. Qf5 {+15.41/5 0.54s} 65. ... Qc7 {-128.51/17 0.20s} 66. Kg5 {+14.04/5 0.51s} 66. ... Qd8+ {-128.51/17 0.20s} 67. Qf6 { +19.64/5 0.50s} 67. ... Qd2+ {-128.51/15 0.20s} 68. Kg6 {+15.87/5 0.50s} 68. ... Qg2+ {-128.51/15 0.20s} 69. Kf7 {+13.02/5 0.46s} 69. ... Qb7+ { -128.51/18 0.20s} 70. Qe7 {+12.94/5 0.52s} 70. ... Qf3+ {-128.51/17 0.20s} 71. Ke8 {+16.37/5 0.50s} 71. ... Qh5+ {-128.51/18 0.20s} 72. Kd8 {+23.68/5 0.50s} 72. ... Qh1 {-128.51/20 0.20s} 73. Qd6+ {+27.20/4 0.50s} 73. ... Ke2 {-128.51/21 0.20s} 74. e6 {+128.00/2 0.50s} 74. ... Qa8+ {-128.51/21 0.20s} 75. Kd7 {+13.64/5 0.50s} 75. ... Qa4+ {-128.51/19 0.20s} 76. Qc6 { +16.29/5 0.50s} 76. ... Qd4+ {-128.51/18 0.20s} 77. Ke8 {+14.46/5 0.50s} 77. ... Qh8+ {-128.51/15 0.20s} 78. Kd7 {+12.86/6 0.49s} 78. ... Qd4+ { -128.51/40 0.20s} 79. Kc8 {+12.21/5 0.51s} 79. ... Qh8+ {-128.51/17 0.20s} 80. Kc7 {+12.02/5 0.50s} 80. ... Qg7+ {-128.51/16 0.20s} 81. Qd7 {+13.17/5 0.54s} 81. ... Qc3+ {-128.51/17 0.20s} 82. Kb6 {+16.91/5 0.50s} 82. ... Qe3+ {-128.51/18 0.20s} 83. Kb7 {+15.93/5 0.50s} 83. ... Qb3+ {-128.51/18 0.20s} 84. Kc7 {+12.30/5 0.50s} 84. ... Qc3+ {-128.51/42 0.20s} 85. Qc6 { +6.62/6 0.050s} 85. ... Qg7+ {-128.51/42 0.20s} 86. Kb6 {+10.55/3 0.024s} 86. ... Qd4+ {-128.51/16 0.20s} 87. Qc5 {+15.59/5 0.55s} 87. ... Qf6 { -128.51/17 0.20s} 88. Qd6 {+17.49/5 0.50s} 88. ... Qb2+ {-128.51/17 0.20s} 89. Kc7 {+19.47/5 0.51s} 89. ... Qc2+ {-128.51/17 0.20s} 90. Kd8 {+25.03/4 0.50s} 90. ... Qe4 {-128.51/17 0.20s} 91. e7 {+128.00/2 0.52s} 91. ... Qa8+ {-128.51/17 0.20s} 92. Kd7 {+18.34/5 0.50s} 92. ... Qb7+ {-128.51/13 0.20s} 93. Ke6 {+19.35/5 0.50s} 93. ... Qc8+ {-128.51/14 0.20s} 94. Kf7 { +17.35/5 0.50s} 94. ... Qf5+ {-128.51/13 0.20s} 95. Qf6 {+26.58/5 0.50s} 95. ... Qh5+ {-M112/14 0.20s} 96. Kf8 {+30.26/5 0.53s} 96. ... Qc5 { -128.51/17 0.20s} 97. Kg7 {+22.48/5 0.50s} 97. ... Qc7 {-128.51/13 0.20s} 98. Kf8 {+26.23/4 0.51s} 98. ... Qc5 {-128.51/61 0.20s} 99. Kf7 {+12.66/2 0.026s} 99. ... Qh5+ {-128.51/60 0.20s} 100. Kg7 {+7.38/3 0.026s} 100. ... Qg4+ {-M70/17 0.20s} 101. Kf8 {+28.90/4 0.51s} 101. ... Qb4 {-M60/20 0.20s} 102. Kf7 {+27.49/5 0.51s} 102. ... Qb7 {-128.51/15 0.20s} 103. Qe6+ {+15.65/4 0.51s} 103. ... Kd2 {-128.51/15 0.20s} 104. Kg6 {+20.09/4 0.49s} 104. ... Qg2+ {-128.51/16 0.20s} 105. Kh7 {+26.29/5 0.50s} 105. ... Qb7 { -128.51/15 0.20s} 106. Kh6 {+21.40/4 0.52s} 106. ... Qh1+ {-128.51/14 0.20s} 107. Kg7 {+19.46/5 0.50s} 107. ... Qg1+ {-128.51/14 0.20s} 108. Qg6 {+15.46/5 0.50s} 108. ... Qd4+ {-128.51/16 0.20s} 109. Qf6 {+18.71/5 0.48s } 109. ... Qg4+ {-128.51/17 0.20s} 110. Kf8 {+21.29/4 0.47s} 110. ... Qb4 {-128.51/17 0.20s} 111. Qe5 {+19.28/5 0.51s} 111. ... Kc2 {-128.51/18 0.20s} 112. Kf7 {+14.70/4 0.52s} 112. ... Qc4+ {-128.51/17 0.20s} 113. Qe6 {+16.22/5 0.50s} 113. ... Qf4+ {-128.51/15 0.20s} 114. Qf6 {+20.87/5 0.50s } 114. ... Qc7 {-128.51/18 0.20s} 115. Kf8 {+27.60/4 0.51s} 115. ... Qc5 { -128.51/19 0.20s} 116. Kg7 {+25.79/4 0.51s} 116. ... Qc7 {-128.51/19 0.20s } 117. Kh6 {+16.18/4 0.53s} 117. ... Qh2+ {-128.51/14 0.20s} 118. Kg7 { +14.99/5 0.47s} 118. ... Qc7 {-128.51/54 0.20s} 119. Kf7 {+7.68/6 0.15s} 119. ... Kd3 {-128.51/16 0.20s} 120. Kf8 {+18.85/4 0.51s} 120. ... Qc5 { -128.51/15 0.20s} 121. Kg7 {+13.07/5 0.51s} 121. ... Qc7 {-128.51/16 0.20s } 122. Kg6 {+8.85/4 0.53s} 122. ... Qg3+ {-M82/15 0.20s} 123. Kf7 { +10.67/5 0.51s} 123. ... Qc7 {-128.51/67 0.20s} 124. Qa6+ {+5.65/6 0.51s} 124. ... Kd4 {-128.51/15 0.20s} 125. Qa1+ {+5.98/5 0.54s} 125. ... Ke3 { -128.51/16 0.20s} 126. Qa3+ {+4.64/5 0.55s} 126. ... Ke2 {-128.51/17 0.20s } 127. Qa6+ {+3.75/5 0.51s} 127. ... Kf2 {-128.51/15 0.20s} 128. Qf6+ { +3.52/5 0.53s} 128. ... Ke1 {-M76/19 0.20s} 129. Kf8 {+4.14/5 0.51s} 129. ... Qc5 {-M74/21 0.20s} 130. Kf7 {+3.84/5 0.50s} 130. ... Qc7 { -128.51/51 0.20s} 131. Qd4 {+1.24/5 0.037s} 131. ... Qb7 {-M60/21 0.20s} 132. Qe5+ {+2.01/5 0.53s} 132. ... Kd1 {-128.51/15 0.20s} 133. Qa1+ { +1.71/5 0.50s} 133. ... Ke2 {-128.51/16 0.20s} 134. Qd4 {+1.25/5 0.031s} 134. ... Qf3+ {-128.51/27 0.20s} 135. Qf6 Qh5+ {-128.51/127 0.001s, Draw by 3-fold repetition} 1/2-1/2 [Event "?"] [Site "?"] [Date "2018.10.02"] [Round "163"] [White "lc0_v18_Syzygy"] [Black "SF8_Syzygy"] [Result "1/2-1/2"] [BlackTimeControl "0.2/move"] [GameDuration "00:00:07"] [GameEndTime "2018-10-02T11:23:43.490 GTB Daylight Time"] [GameStartTime "2018-10-02T11:23:36.381 GTB Daylight Time"] [PlyCount "35"] [SetUp "1"] [WhiteTimeControl "0.5/move"] [FEN "b7/8/6B1/3K4/8/8/2P5/Bk6 w - - 0 1"] 1. Kd4 Bc6 {-128.51/24 0.20s} 2. Kc5 Ba4 {-128.51/26 0.20s} 3. c4+ { +128.00/1 0.052s} 3. ... Kxa1 {-128.51/32 0.20s} 4. Kb4 Bc6 {-128.51/36 0.20s} 5. Ka3 Bh1 {-128.51/38 0.20s} 6. c5 {+128.00/2 0.42s} 6. ... Bg2 { -128.51/38 0.20s} 7. Bf7 {+5.56/5 0.47s} 7. ... Bc6 {-128.51/34 0.20s} 8. Kb3 {+6.24/5 0.49s} 8. ... Kb1 {-128.51/37 0.20s} 9. Kc3 {+5.66/6 0.47s} 9. ... Kc1 {-128.51/34 0.20s} 10. Bh5 {+5.79/6 0.50s} 10. ... Kb1 { -128.51/26 0.20s} 11. Bf7 {+5.32/5 0.039s} 11. ... Kc1 {-128.51/55 0.20s} 12. Bc4 {+4.86/5 0.031s} 12. ... Kd1 {-128.51/31 0.20s} 13. Ba6 {+5.76/2 0.018s} 13. ... Bg2 {-128.51/32 0.20s} 14. Bd3 {+6.96/5 0.50s} 14. ... Bc6 {-128.51/34 0.20s} 15. Ba6 {+5.30/4 0.021s} 15. ... Bg2 {-128.51/62 0.20s} 16. Bb5 {+4.25/4 0.024s} 16. ... Bf3 {-128.51/30 0.20s} 17. Bd3 {+5.92/5 0.52s} 17. ... Bc6 {-128.51/95 0.20s} 18. Ba6 {0.00/2 0.025s, Draw by 3-fold repetition} 1/2-1/2 [/pgn]

Hi, thanks a lot for testing! The dev team found another bug in the tablebase implementation, where only repeats from the current position but not from all positions since the last zeroing move were considered. There is now a fix for this: https://github.com/LeelaChessZero/lc0/pull/412, with compiled binary https://ci.appveyor.com/api/buildjobs/5 ... %2Flc0.exe. If you could run your testing suite with the new build again this would be great Thanks again!