Strong Leela nets competition

[corres]

I made a competition among strong Leela nets.
Parameters
Used Leela: v.0.22.0
GPUs: 2 x RTX 2060 OC
Threads: 4 X 4GHz
Backend: Multiplexing
BackendOptions: backend=cudnn-fp16,(gpu=0),(gpu=1)
NNCacheSize: 10000000
MaxPrefetchBatch: 64
Others are Default
Time Control: 1 min + 2 sec / move
50 opening positions played with alternated colors

Results
T40B.4-160 : 42872 = 4 : 3 (93 draw) 100 games
42872 : J13B.2-188 = 9 : 6 (85 draw) 100 games
T40B.4-160 : J13B.2-188 = 9 : 3 (88 draw) 100 games

In Points
T40B.4-160 net 13 points
42872 net 12 points
J13B.2-188 net 9 points

I made test with 384x30-T40-970 "big net" with disappointing result
J13B.2-188 : 384x30-T40-970 = 17(!) : 3 (80 draw) 100 games.
Seeing the result I did not make further test with it.

[Laskos]

I made a competition among strong Leela nets.
Parameters
Used Leela: v.0.22.0
GPUs: 2 x RTX 2060 OC
Threads: 4 X 4GHz
Backend: Multiplexing
BackendOptions: backend=cudnn-fp16,(gpu=0),(gpu=1)
NNCacheSize: 10000000
MaxPrefetchBatch: 64
Others are Default
Time Control: 1 min + 2 sec / move
50 opening positions played with alternated colors

Results
T40B.4-160 : 42872 = 4 : 3 (93 draw) 100 games
42872 : J13B.2-188 = 9 : 6 (85 draw) 100 games
T40B.4-160 : J13B.2-188 = 9 : 3 (88 draw) 100 games

In Points
T40B.4-160 net 13 points
42872 net 12 points
J13B.2-188 net 9 points

I made test with 384x30-T40-970 "big net" with disappointing result
J13B.2-188 : 384x30-T40-970 = 17(!) : 3 (80 draw) 100 games.
Seeing the result I did not make further test with it.

J13B.4 is strong at LTC on strong hardware, or so it seems to me.

[corres]

...
J13B.4 is strong at LTC on strong hardware, or so it seems to me.

Where we can see the results of J13b.4-xyz?

[corres]

I made a competition among strong Leela nets.
Parameters
Used Leela: v.0.22.0
GPUs: 2 x RTX 2060 OC
Threads: 4 X 4GHz
Backend: Multiplexing
BackendOptions: backend=cudnn-fp16,(gpu=0),(gpu=1)
NNCacheSize: 10000000
MaxPrefetchBatch: 64
Others are Default
Time Control: 1 min + 2 sec / move
50 opening positions played with alternated colors

Results
T40B.4-160 : 42872 = 4 : 3 (93 draw) 100 games
42872 : J13B.2-188 = 9 : 6 (85 draw) 100 games
T40B.4-160 : J13B.2-188 = 9 : 3 (88 draw) 100 games

In Points
T40B.4-160 net 13 points
42872 net 12 points
J13B.2-188 net 9 points

I made test with 384x30-T40-970 "big net" with disappointing result
J13B.2-188 : 384x30-T40-970 = 17(!) : 3 (80 draw) 100 games.
Seeing the result I did not make further test with it.

You can download the .zip file
wikisend.com
File ID: 621504
Password: leela

[shrapnel]

Playing big NNs against small NNs is one thing. Playing those NNs against Stockfish is a whole new Ball Game .
You "MAY" get different results if you play them against Stockfish.
"MAY" because your GPUs are weak and will be unable to fully utilize the powere of large NNs.
So, they may again lose on your Setup.
That's the Problem with testing of NNs.
One has to take into account BOTH the NN being used AND the GPU... TWO Factors involved.

[Damir]

in TCEC Leela lost 2 games already…. it's play is crap, and seems no improvement has been made.. It is loser engine whose play only depends on strong GPU's...

[corres]

Playing big NNs against small NNs is one thing. Playing those NNs against Stockfish is a whole new Ball Game .
You "MAY" get different results if you play them against Stockfish.
"MAY" because your GPUs are weak and will be unable to fully utilize the powere of large NNs.
So, they may again lose on your Setup.
That's the Problem with testing of NNs.
One has to take into account BOTH the NN being used AND the GPU... TWO Factors involved.

Sometimes you also should make some test, do you not think that?
I have no Twin RTX 2080 Ti only a twin RTX 2060 OC.
I put my openings on net and you should play two matches: a match (100 games) between the last Stockfish dev and your favorite big net as well as a match (100 games) between the last Stockfish dev and T40B.4-160 net with the same parameters. It takes about a day test, if you use 1 min + 2 sec / move TC (shorter time is bad!). If the big net will get better result against Stockfish than T40B.4-160 will, you are right.
It would be the exact evidence.
Any other is only ''bla-di-bla".

[Nay Lin Tun]

in TCEC Leela lost 2 games already…. it's play is crap, and seems no improvement has been made.. It is loser engine whose play only depends on strong GPU's...

I guess you are joking! Did you check what kind of custom positions they are testing for? Most of them are like that, I did not check all of them though!
Leela will probably play at 3000 rating! But not surprising!
NN are not trained for such custom positions that you wont see 1 in billion games.

[Damir]

My response was to Shrapnel,,, since he keeps bashing all other engines and praise Leela... to show him that Lc0 is patzer engine too….

[Nordlandia]

Try this position!



Lyudmil Tsvetkov claimed black wins. But Lc0 did not manage to win against SF Dev and SF 10.

[Event "DESKTOP-N1IDKRV, Rapid 30m+30s"] [Site "Expert-Desktop"] [Date "2019.11.11"] [Round "1"] [White "Stockfish 10 64 BMI2"] [Black "Lc0 v0.22.0"] [Result "1/2-1/2"] [Annotator "0.00;-0.72"] [SetUp "1"] [FEN "3n1b2/p1n1k2p/1p6/5b2/7P/1P1PP1P1/P1P1K3/R3R3 w - - 0 1"] [PlyCount "142"] [TimeControl "1800+30"] 1. Rf1 {0.00/38 134 Both last book move} Bg4+ {-0.72/14 29} 2. Kd2 {-0.09/36 22 } Bg7 {-0.70/15 75 (Nf7)} 3. d4 {-0.10/36 77} Ne8 {-0.72/15 0 (Nf7)} 4. Rae1 { 0.09/38 18 (Kd3)} Bh5 {-0.76/15 75 (Nd6)} 5. c3 {0.00/35 39} Nf6 {-0.73/16 66} 6. e4 {-0.09/39 0} Bg6 {-0.73/15 3} 7. a4 {-0.16/40 313 (Kc1)} h5 {-0.65/13 118 (Ne6)} 8. Kc1 {-0.19/41 176} Kd7 {-0.84/17 22 (Bh6+)} 9. Re2 {0.00/45 43} Ng4 {-0.89/14 53} 10. Kd2 {0.00/46 0 (Kb2)} Nf7 {-1.16/15 184} 11. Kc2 { -0.19/48 0} Nd6 {-1.16/14 16} 12. Kb2 {-0.72/43 190} Ke7 {-1.18/16 1} 13. Rfe1 {-0.55/41 28} Kd8 {-1.18/15 23 (Bh6)} 14. Ka2 {-0.80/41 84} Bf8 {-1.18/17 1 (Bh6)} 15. Ka1 {-0.75/45 104 (Rf1)} Be7 {-1.20/12 245 (Nb7)} 16. Ka2 {-1.09/41 56} Kc8 {-1.18/12 110 (Bf8)} 17. Kb2 {-1.07/44 61} Kd8 {-1.17/12 59} 18. Ka1 { -1.05/42 0 (Ka2)} Kd7 {-1.17/11 76} 19. Kb2 {-1.06/42 0 (Ka2)} Ke8 {-1.16/11 61 (Kd8)} 20. Ka1 {-0.98/43 55} Bf8 {-1.16/12 74 (Nb7)} 21. Kb2 {-0.98/45 133 (Ka2)} Be7 {-1.16/12 32 (Kd7)} 22. Ka1 {-0.97/45 23} Kd8 {-1.15/12 84 (Bf8)} 23. Ka2 {-0.87/42 19} Kd7 {-1.15/12 77} 24. Ka1 {-0.83/46 0 (Kb2)} Kc8 { -1.16/11 63 (Bf8)} 25. Kb2 {-0.83/44 29} Kd7 {-1.15/12 36} 26. Ka1 {-0.77/45 0 (Ka2)} a6 {-1.14/12 84 (Bf8)} 27. Kb2 {-0.32/36 24 (Ka2)} Bf8 {-1.13/12 47 (Bd8)} 28. Rf1 {-1.19/41 458 (Ka1)} Bh6 {-1.12/13 49} 29. Rfe1 {-1.27/37 0} Kd8 {-1.11/12 63 (Kc6)} 30. Ka2 {-0.80/39 78} Kc8 {-1.10/13 57 (Kc7)} 31. Kb2 { -1.57/38 114} Bf8 {-1.10/14 48 (Kc7)} 32. Ka1 {-1.18/38 41 (Rf1)} Kd7 {-1.11/ 12 110 (Kc7)} 33. Ka2 {-1.39/40 68} Kc8 {-1.10/13 71 (Bh6)} 34. Kb2 {-1.73/41 99 (Rf1)} Kd8 {-1.10/12 73 (Kd7)} 35. Rf1 {-1.64/43 57 (Ka2)} Bh6 {-1.11/12 40 (Ke7)} 36. Rfe1 {-1.50/43 39} Ke8 {-1.09/12 71 (Kc7)} 37. Ka2 {-1.34/42 94} Kd8 {-1.08/13 66 (Kd7)} 38. Kb2 {-1.62/40 57} Kd7 {-1.08/13 75 (Kc7)} 39. Ka3 { -1.05/38 37 (Ka2)} Bf8 {-1.09/12 34} 40. Ka2 {-1.00/37 1 (Kb2)} Be7 {-1.08/12 92 (Ke7)} 41. Kb2 {-1.20/39 67} Kd8 {-1.09/13 21} 42. Ka2 {-0.94/38 2} Bf8 { -1.08/12 73 (Kc8)} 43. Kb2 {-1.20/40 42 (Ka1)} Ke8 {-1.09/13 37 (Kc7)} 44. Ka1 {-1.05/41 45 (Ka2)} Bh6 {-1.08/12 89} 45. Ka2 {-1.01/41 4} Kd7 {-1.06/13 60 (Ke7)} 46. Kb2 {-1.06/42 72 (Ka3)} Kd8 {-1.06/12 54 (Kc6)} 47. Ka3 {-0.88/35 18 (Ka2)} Bf8 {-1.06/12 72} 48. Kb2 {-0.85/43 0} Be7 {-1.07/12 49 (Ke8)} 49. Ka2 {-0.87/38 97} Ke8 {-1.08/16 0 (Kc7)} 50. Kb2 {-1.26/41 96} Bf8 {-1.08/14 34 (Bh7)} 51. Ka2 {-1.08/37 27} Kd7 {-1.08/14 84} 52. Kb2 {-0.82/42 0 (Ka1)} Bh6 {-1.08/12 48 (Bh7)} 53. Ka2 {-1.31/38 60} Kc8 {-1.06/13 59 (Kc7)} 54. Kb2 { -1.31/37 45} Kb7 {-1.06/13 56 (Kb8)} 55. Ka3 {-0.50/34 16 (Ka2)} Kc7 {-1.06/12 34} 56. Kb2 {-0.71/36 7 (Ka2)} Kc8 {-1.06/12 47 (b5)} 57. Ka2 {-0.90/37 35} Kc7 {-1.05/13 44 (Kb7)} 58. Kb2 {-0.73/40 41 (Ka3)} Kc8 {-1.05/12 42 (Bh7)} 59. Ka2 {-0.69/38 21} a5 {-1.05/12 14 (Kb7)} 60. Ka3 {0.00/46 18} Kd7 {-1.01/11 68 (Kc7)} 61. Kb2 {0.00/49 24 (Ka2)} Ke8 {-0.99/10 36 (Kc6)} 62. Ka1 {0.00/50 27 (Ka3)} Bf8 {-0.99/11 39 (Kd7)} 63. Ka2 {0.00/47 22} Kd8 {-0.98/11 31 (Ke7)} 64. Kb2 {0.00/48 27 (Ka1)} Bh6 {-0.95/11 35 (Ke7)} 65. Ka3 {-0.22/48 121 (Ka2)} Nc8 {-0.94/10 36 (Kc7)} 66. Ka2 {-0.47/39 66} Nd6 {-0.93/12 29} 67. Ka3 {-0.53/41 2 } Bh7 {-0.92/11 36 (Be8)} 68. Ka2 {0.00/45 22 (Kb2)} Kd7 {-0.90/11 30} 69. Ka3 {-0.39/38 35} Bf8 {-0.90/12 22 (Ke7)} 70. Kb2 {-0.77/37 56} Kd8 {-0.88/12 30 (Bg7)} 71. Ka1 {-0.59/34 17 (Ka2)} Ke8 {-0.87/11 36 (Bg6) adjud. Draw accepted} 1/2-1/2

Also he claimed that white wins here too.



5+5 Blitz | Lc0 (2080 Super) vs 7-core SF Dev

[Event "?"] [Site "i7-5960X 4.5GHz"] [Date "2019.11.11"] [Round "?"] [White "lc0"] [Black "stockfish_x64_bmi2"] [Result "1-0"] [FEN "rk1r4/1b1p2pp/p1pPpp2/Pp2P2q/1P1P1P2/3BBN2/5RPP/R5K1 w - - 0 24"] [GameDuration "00:15:59"] [GameEndTime "2019-11-11T08:45:58.797 Vest-Europa (normaltid)"] [GameStartTime "2019-11-11T08:29:59.274 Vest-Europa (normaltid)"] [PlyCount "76"] [SetUp "1"] [Termination "adjudication"] [TimeControl "300+5"] 1. Re1 {+0.87/13 8.6s} Rf8 {+1.33/34 28s} 2. h3 {+0.97/13 16s} Re8 {+1.43/34 77s} 3. Bc2 {+1.26/11 13s} f5 {+0.78/35 31s} 4. Rc1 {+1.32/13 8.8s} Qf7 {+0.84/32 14s} 5. Ne1 {+1.56/12 14s} h6 {+0.84/30 6.0s} 6. Nd3 {+1.38/13 28s} Rg8 {+0.28/35 30s} 7. Nc5 {+1.83/10 9.0s} g5 {+1.22/27 6.3s} 8. Kh2 {+1.79/14 23s} g4 {0.00/37 16s} 9. hxg4 {+2.06/13 6.7s} Rxg4 {0.00/39 5.0s} 10. Bd1 {+2.08/14 8.0s} Rg8 {0.00/36 5.3s} 11. Rf3 {+2.13/12 13s} Qg6 {-0.49/40 42s} 12. Rc2 {+2.19/13 12s} Qh5+ {0.00/41 5.5s} 13. Kg1 {+2.18/13 8.2s} Rg7 {-1.39/43 52s} 14. Rd2 {+2.24/12 13s} Bc8 {-1.58/35 5.2s} 15. Bf2 {+2.29/12 12s} Qg4 {-1.38/30 3.5s} 16. Be1 {+2.33/12 12s} Ka7 {-2.78/36 31s} 17. Rg3 {+2.50/16 21s} Qxg3 {-2.54/36 4.0s} 18. Bxg3 {+2.46/17 8.4s} Rxg3 {-3.73/39 18s} 19. Bh5 {+2.51/17 4.6s} Rg7 {-4.15/33 9.5s} 20. Be8 {+2.57/17 10.0s} h5 {-4.39/32 5.2s} 21. Rd3 {+2.62/16 9.7s} h4 {-4.79/31 4.5s} 22. Rh3 {+2.65/15 11s} Rg4 {-4.81/30 3.6s} 23. g3 {+2.75/13 27s} Rg8 {-5.06/33 7.0s} 24. Bxd7 {+2.94/13 22s} Bxd7 {-5.10/30 3.3s} 25. Nxd7 {+2.95/14 6.1s} Rg6 {-6.84/31 6.7s} 26. Kf2 {+2.96/13 13s} Rd8 {-6.87/33 5.0s} 27. Nc5 {+3.01/13 13s} Rdg8 {-7.33/31 5.0s} 28. d7 {+3.07/13 7.1s} R6g7 {-6.07/28 2.5s} 29. d5 {+3.18/12 14s} cxd5 {-6.25/30 4.3s} 30. Rxh4 {+3.89/10 16s} Re7 {-5.85/31 4.2s} 31. Rh6 {+4.71/8 16s} Rd8 {-6.06/30 4.8s} 32. Ke3 {+4.71/9 15s} Ka8 {-8.37/30 9.1s} 33. Kd4 {+5.25/8 17s} Ka7 {-9.14/27 4.9s} 34. Nxe6 {+6.50/8 5.4s} Rdxd7 {-9.54/29 5.1s} 35. Nc5 {+7.25/8 4.9s} Rd8 {-9.91/27 5.0s} 36. Rxa6+ {+6.51/7 13s} Kb8 {-10.23/1 0s} 37. Rb6+ {+7.60/7 7.1s} Kc8 {-10.10/26 5.5s} 38. Rxb5 {+7.54/7 7.0s} Rh8 {-10.92/23 9.4s, White wins by adjudication} 1-0