20x256 Leela seemed quite strong

[Nay Lin Tun]

This leela seemed strong, 2 games played , 2 games draw vs stockfish is amazing.

Please test this leela.
https://drive.google.com/uc?id=11ueg-md ... t=download
This Leela, 20x256, seemed quite strong.
I played 3+2 mins blitz cuDNN 21st May (3rd version) Leela vs full strength latest vestion 4 cores i5 3.0GHZ stockfish on my 1060, for two games and both games draw. Opening book, perfect chess, 2017, 8 moves.


I will test more games later.

Nay

P.s average speed of Leela is around 2-2.5 Kn/s and average speed of stockfish is around 5 Mn/s.
https://lichess.org/UQytNWdL

https://lichess.org/IlhlNsul

[Jhoravi]

You mean it got stronger without additional training? How come?

[Albert Silver]

You mean it got stronger without additional training? How come?

Larger net, no oversampling, removal of a few million buggy games.

[duncan]

This leela seemed strong, 2 games played , 2 games draw vs stockfish is amazing.

Please test this leela.
https://drive.google.com/uc?id=11ueg-md ... t=download
This Leela, 20x256, seemed quite strong.
I played 3+2 mins blitz cuDNN 21st May (3rd version) Leela vs full strength latest vestion 4 cores i5 3.0GHZ stockfish on my 1060, for two games and both games draw. Opening book, perfect chess, 2017, 8 moves.


I will test more games later.

Nay

P.s average speed of Leela is around 2-2.5 Kn/s and average speed of stockfish is around 5 Mn/s.
https://lichess.org/UQytNWdL

https://lichess.org/IlhlNsul

would lczero being the same as stockfish mean it is stronger than alphazero ?

[Laskos]

This leela seemed strong, 2 games played , 2 games draw vs stockfish is amazing.

Please test this leela.
https://drive.google.com/uc?id=11ueg-md ... t=download
This Leela, 20x256, seemed quite strong.
I played 3+2 mins blitz cuDNN 21st May (3rd version) Leela vs full strength latest vestion 4 cores i5 3.0GHZ stockfish on my 1060, for two games and both games draw. Opening book, perfect chess, 2017, 8 moves.


I will test more games later.

Nay

P.s average speed of Leela is around 2-2.5 Kn/s and average speed of stockfish is around 5 Mn/s.
https://lichess.org/UQytNWdL

https://lichess.org/IlhlNsul

Well, it doesn't seem significantly stronger than 192x15 NN338 in my gauntlet, maybe weaker. It also performs a bit worse at 10s/position in both tactical and positional test suites. Probably due to 2x slower speed.

Gauntlet (will continue to 60 games each Lc0) with the latest cuDNN Lc0 on GTX 1060 GPU, latest 256x20 net, and NN338 192x15 net, against Houdini 1.5a on one 3.8 GHz core of i7. Time control: 1s/move. Openings: balanced short 3-mover from GM games.

Code: Select all

Games Completed = 50 of 120 (Avg game length = 116.917 sec)
Settings = Gauntlet/64MB/1000ms per move/M 9000cp for 30 moves, D 150 moves/EPD:C:\LittleBlitzer\3moves_GM_04.epd(817)
Time = 6207 sec elapsed, 8689 sec remaining
 1.  Houdini 1.5a             	28.5/50	23-16-11  	(L: m=16 t=0 i=0 a=0)	(D: r=9 i=1 f=0 s=0 a=1)	(tpm=1001.3 d=19.47 nps=3639708)
 
 2.  Lc0 NN338                	12.0/25	10-11-4  	(L: m=11 t=0 i=0 a=0)	(D: r=4 i=0 f=0 s=0 a=0)	(tpm=753.0 d=3.27 nps=5003)
 3.  Lc0 NN  256x20            	 9.5/25	6-12-7  	(L: m=12 t=0 i=0 a=0)	(D: r=5 i=1 f=0 s=0 a=1)	(tpm=777.8 d=2.07 nps=2505)

I will report later on the final result. Keep in mind that Lc0 scales better than standard AB engines, so it is stronger at LTC.

[Albert Silver]

This leela seemed strong, 2 games played , 2 games draw vs stockfish is amazing.

Please test this leela.
https://drive.google.com/uc?id=11ueg-md ... t=download
This Leela, 20x256, seemed quite strong.
I played 3+2 mins blitz cuDNN 21st May (3rd version) Leela vs full strength latest vestion 4 cores i5 3.0GHZ stockfish on my 1060, for two games and both games draw. Opening book, perfect chess, 2017, 8 moves.


I will test more games later.

Nay

P.s average speed of Leela is around 2-2.5 Kn/s and average speed of stockfish is around 5 Mn/s.
https://lichess.org/UQytNWdL

https://lichess.org/IlhlNsul

Well, it doesn't seem significantly stronger than 192x15 NN338 in my gauntlet, maybe weaker. It also performs a bit worse at 10s/position in both tactical and positional test suites. Probably due to 2x slower speed.

Gauntlet (will continue to 60 games each Lc0) with the latest cuDNN Lc0 on GTX 1060 GPU, latest 256x20 net, and NN338 192x15 net, against Houdini 1.5a on one 3.8 GHz core of i7. Time control: 1s/move. Openings: balanced short 3-mover from GM games.

Code: Select all

Games Completed = 50 of 120 (Avg game length = 116.917 sec)
Settings = Gauntlet/64MB/1000ms per move/M 9000cp for 30 moves, D 150 moves/EPD:C:\LittleBlitzer\3moves_GM_04.epd(817)
Time = 6207 sec elapsed, 8689 sec remaining
 1.  Houdini 1.5a             	28.5/50	23-16-11  	(L: m=16 t=0 i=0 a=0)	(D: r=9 i=1 f=0 s=0 a=1)	(tpm=1001.3 d=19.47 nps=3639708)
 
 2.  Lc0 NN338                	12.0/25	10-11-4  	(L: m=11 t=0 i=0 a=0)	(D: r=4 i=0 f=0 s=0 a=0)	(tpm=753.0 d=3.27 nps=5003)
 3.  Lc0 NN  256x20            	 9.5/25	6-12-7  	(L: m=12 t=0 i=0 a=0)	(D: r=5 i=1 f=0 s=0 a=1)	(tpm=777.8 d=2.07 nps=2505)

I will report later on the final result. Keep in mind that Lc0 scales better than standard AB engines, so it is stronger at LTC.

The author pointed out this did not include the buggy game removal yet, though it is a test he plans to make.

[Dann Corbit]

This leela seemed strong, 2 games played , 2 games draw vs stockfish is amazing.

Please test this leela.
https://drive.google.com/uc?id=11ueg-md ... t=download
This Leela, 20x256, seemed quite strong.
I played 3+2 mins blitz cuDNN 21st May (3rd version) Leela vs full strength latest vestion 4 cores i5 3.0GHZ stockfish on my 1060, for two games and both games draw. Opening book, perfect chess, 2017, 8 moves.


I will test more games later.

Nay

P.s average speed of Leela is around 2-2.5 Kn/s and average speed of stockfish is around 5 Mn/s.
https://lichess.org/UQytNWdL

https://lichess.org/IlhlNsul

Two games is a rather small sample.
After two thousand, we can make a better conclusion.

[shrapnel]

You mean it got stronger without additional training? How come?

Larger net, no oversampling, removal of a few million buggy games.

Really, 20x256 is looking really good !
lc0 CUDA just beat Komodo 11.2.2 in a 10'+10" Game.
Slower but ZERO Blunders.
https://lichess.org/g2ebFdCr

[Nay Lin Tun]

Sure, if Leela got 50% scores vs stockfish after playing 2000 games, then there is reasonable confidence that Leela will be within 20-30 elo . Well at least, I wont do that , it is so boring for me. In comparision with other results from other testers, the general conclusion is that Leela on my 1060GTX card is within 300-500 elo range vs 4 cores stockfish , (results may vary depending on the hardware and time control).  

I tested multiple blitz games of leela vs stockfish before, previous versions of leela never survive as black.This version amazed me cos, it was the first version of leela that could survive as black.

[shrapnel]

(results may vary depending on the hardware and time control).  

No 'may vary' about it. It will DEFINITELY vary depending on the Hardware and TC used.