leela is official(?) better than sf9

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Laskos
Posts: 9321
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: leela is official(?) better than sf9

Post by Laskos » Sat Sep 15, 2018 11:54 am

Javier Ros wrote:
Sat Sep 15, 2018 9:28 am
Laskos wrote:
Wed Sep 12, 2018 7:36 pm
Javier Ros wrote:
Wed Sep 12, 2018 6:38 pm
Laskos wrote:
Wed Sep 12, 2018 6:02 pm

I don't understand the discrepancy. Are you sure your SF9 uses 6 threads and not 1? This result would indicate 1 thread is used, but maybe I am doing something very wrong. OTOH, CCCC result, which I expected to come close to my result, because the CPU effective speed-up is about 8 compared to my 4 i7 cores, and GPU speed-up is again about 8 compared to to my GTX 1060, shows, like in my case, that Lc0 is the level of Fire 7.1. I use adjudication only for very long games, above 100 moves with evals in [-20cp, 20cp] range (usually very drawish long endgames). I use no win adjudication.
http://talkchess.com/forum3/download/file.php?id=97

I am sure Stockfish uses 6 threads of the 8 threads (4 cores), you can see the 74% in the task manager for Stockfish 9 and 0% for lc0 due to Ponder Off and also the NPS counter is 8.7 million.

I use adjudication by Arena at -9.00

I have read in this link that the net 11120 is better than the rest of 11xxx and I am testing it right now. At this moment Stockfish 6.5-lc017 11120 6.5

https://groups.google.com/forum/#!topic ... IUjoNZAxVw

Version NN ELO Perf W D L Fire Komodo SF Score Games % Fire % Komodo % SF % Total
0.16 600 3428 56 186 118 61 48.5 39.5 149 360 50.83 40.42 32.92 41.39
0.16 643 3428 56 186 118 58.5 52 38.5 149 360 48.75 43.33 32.08 41.39
0.16 809 3428 56 188 116 56 51.5 42.5 150 360 46.67 42.92 35.42 41.67
0.16 695 3435 69 169 122 66 45.5 42 153.5 360 55 37.92 35 42.64
0.16 776 3435 61 187 112 59.5 54 41 154.5 360 49.58 45 34.17 42.92
0.16 928 3442 67 179 114 61.5 54.5 40.5 156.5 360 51.25 45.42 33.75 43.47
0.16 840 3456 64 198 98 67 53.5 42.5 163 360 55.83 44.58 35.42 45.28
0.17 1066 3463 77 184 99 70.5 51 47.5 169 360 58.75 42.5 39.58 46.94
0.17 1120 3470 77 189 94 68.5 54 49 171.5 360 57.08 45 40.83 47.64
0.17 1186 3449 68 183 109 59 57 43.5 159.5 360 49.17 47.5 36.25 44.31
In this Arena output, the third row is the nodes searched, and the fourth is the speed? If yes, then why after 1 second you have 36,000 nodes searched with 2,600 nodes per second? Second thing, my NPS both in Arena (3.51) and from command line at 1 second are about 2,000 and increasing slowly to 4,000 in similar position to 30s. My NPS are increasing pretty slowly, is it the case with your NPS too? Maybe I have some issues with drivers or CUDA/CUDNN software? I refreshed them one month ago or so.
I have done another experiment with lc0 ver. 0.17 11261 on another computer with GTX 1070, i7 7700H 2.8 GHz and Cutechess Gui, trying to observe the NPS and total amount of nodes of lc0 as you said.
As you can see in the following videos

https://youtu.be/nJdAT9w4LVI

lc0 with black, in the first move the node counter starts from 0, in the second move starts from 8683, in the third move from 13790, in the 4th from 5923, in the 5th from 19525 ...

https://youtu.be/xmPkY3mdVKY
another example with the same behaviour

I think this is due to the cache of lc0 that, in my case, has the default values of the installation.
Perhaps you have something wrong with the installation of lc0 or the CUDA drivers, I have installed them only once time and never updated them.
The same thing, you play repeatedly the same games with the same cache. How is that search can start at depths 9,10,11 in the early phase of the game using just 50ms? No way. Here is a table for time used per move (seconds), NPS and depths achieved in at least 100 gameplay positions of opening - early middlegame (moves 5-19):

Code: Select all

time   NPS   depth
===================
 0.1  1050   4.69
 0.3  1800   5.49
 1.0  2200   6.58
 3.0  2600   7.85
10.0  2900   8.77
30.0  3000  10.83
Can you re-start engines in Arena (there is an option, I think) after each played game? You will get rid of that behavior if it is a cache issue. Your behavior is undesirable, as you can hardly measure correctly the strength of Lc0.

User avatar
AdminX
Posts: 5115
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: leela is official(?) better than sf9

Post by AdminX » Sat Sep 15, 2018 12:03 pm

Laskos wrote:
Sat Sep 15, 2018 11:54 am
Javier Ros wrote:
Sat Sep 15, 2018 9:28 am
Laskos wrote:
Wed Sep 12, 2018 7:36 pm
Javier Ros wrote:
Wed Sep 12, 2018 6:38 pm
Laskos wrote:
Wed Sep 12, 2018 6:02 pm

I don't understand the discrepancy. Are you sure your SF9 uses 6 threads and not 1? This result would indicate 1 thread is used, but maybe I am doing something very wrong. OTOH, CCCC result, which I expected to come close to my result, because the CPU effective speed-up is about 8 compared to my 4 i7 cores, and GPU speed-up is again about 8 compared to to my GTX 1060, shows, like in my case, that Lc0 is the level of Fire 7.1. I use adjudication only for very long games, above 100 moves with evals in [-20cp, 20cp] range (usually very drawish long endgames). I use no win adjudication.
http://talkchess.com/forum3/download/file.php?id=97

I am sure Stockfish uses 6 threads of the 8 threads (4 cores), you can see the 74% in the task manager for Stockfish 9 and 0% for lc0 due to Ponder Off and also the NPS counter is 8.7 million.

I use adjudication by Arena at -9.00

I have read in this link that the net 11120 is better than the rest of 11xxx and I am testing it right now. At this moment Stockfish 6.5-lc017 11120 6.5

https://groups.google.com/forum/#!topic ... IUjoNZAxVw

Version NN ELO Perf W D L Fire Komodo SF Score Games % Fire % Komodo % SF % Total
0.16 600 3428 56 186 118 61 48.5 39.5 149 360 50.83 40.42 32.92 41.39
0.16 643 3428 56 186 118 58.5 52 38.5 149 360 48.75 43.33 32.08 41.39
0.16 809 3428 56 188 116 56 51.5 42.5 150 360 46.67 42.92 35.42 41.67
0.16 695 3435 69 169 122 66 45.5 42 153.5 360 55 37.92 35 42.64
0.16 776 3435 61 187 112 59.5 54 41 154.5 360 49.58 45 34.17 42.92
0.16 928 3442 67 179 114 61.5 54.5 40.5 156.5 360 51.25 45.42 33.75 43.47
0.16 840 3456 64 198 98 67 53.5 42.5 163 360 55.83 44.58 35.42 45.28
0.17 1066 3463 77 184 99 70.5 51 47.5 169 360 58.75 42.5 39.58 46.94
0.17 1120 3470 77 189 94 68.5 54 49 171.5 360 57.08 45 40.83 47.64
0.17 1186 3449 68 183 109 59 57 43.5 159.5 360 49.17 47.5 36.25 44.31
In this Arena output, the third row is the nodes searched, and the fourth is the speed? If yes, then why after 1 second you have 36,000 nodes searched with 2,600 nodes per second? Second thing, my NPS both in Arena (3.51) and from command line at 1 second are about 2,000 and increasing slowly to 4,000 in similar position to 30s. My NPS are increasing pretty slowly, is it the case with your NPS too? Maybe I have some issues with drivers or CUDA/CUDNN software? I refreshed them one month ago or so.
I have done another experiment with lc0 ver. 0.17 11261 on another computer with GTX 1070, i7 7700H 2.8 GHz and Cutechess Gui, trying to observe the NPS and total amount of nodes of lc0 as you said.
As you can see in the following videos

https://youtu.be/nJdAT9w4LVI

lc0 with black, in the first move the node counter starts from 0, in the second move starts from 8683, in the third move from 13790, in the 4th from 5923, in the 5th from 19525 ...

https://youtu.be/xmPkY3mdVKY
another example with the same behaviour

I think this is due to the cache of lc0 that, in my case, has the default values of the installation.
Perhaps you have something wrong with the installation of lc0 or the CUDA drivers, I have installed them only once time and never updated them.
The same thing, you play repeatedly the same games with the same cache. How is that search can start at depths 9,10,11 in the early phase of the game using just 50ms? No way. Here is a table for time used per move (seconds), NPS and depths achieved in at least 100 gameplay positions of opening - early middlegame (moves 5-19):

Code: Select all

time   NPS   depth
===================
 0.1  1050   4.69
 0.3  1800   5.49
 1.0  2200   6.58
 3.0  2600   7.85
10.0  2900   8.77
30.0  3000  10.83
Can you re-start engines in Arena (there is an option, I think) after each played game? You will get rid of that behavior if it is a cache issue. Your behavior is undesirable, as you can hardly measure correctly the strength of Lc0.
Attachments
9-15-2018 8-02-38 AM.png
9-15-2018 8-02-38 AM.png (19.5 KiB) Viewed 1620 times
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: leela is official(?) better than sf9

Post by Javier Ros » Sat Sep 15, 2018 1:29 pm

Laskos wrote:
Sat Sep 15, 2018 11:54 am
Javier Ros wrote:
Sat Sep 15, 2018 9:28 am
Laskos wrote:
Wed Sep 12, 2018 7:36 pm
Javier Ros wrote:
Wed Sep 12, 2018 6:38 pm
Laskos wrote:
Wed Sep 12, 2018 6:02 pm

I don't understand the discrepancy. Are you sure your SF9 uses 6 threads and not 1? This result would indicate 1 thread is used, but maybe I am doing something very wrong. OTOH, CCCC result, which I expected to come close to my result, because the CPU effective speed-up is about 8 compared to my 4 i7 cores, and GPU speed-up is again about 8 compared to to my GTX 1060, shows, like in my case, that Lc0 is the level of Fire 7.1. I use adjudication only for very long games, above 100 moves with evals in [-20cp, 20cp] range (usually very drawish long endgames). I use no win adjudication.
http://talkchess.com/forum3/download/file.php?id=97

I am sure Stockfish uses 6 threads of the 8 threads (4 cores), you can see the 74% in the task manager for Stockfish 9 and 0% for lc0 due to Ponder Off and also the NPS counter is 8.7 million.

I use adjudication by Arena at -9.00

I have read in this link that the net 11120 is better than the rest of 11xxx and I am testing it right now. At this moment Stockfish 6.5-lc017 11120 6.5

https://groups.google.com/forum/#!topic ... IUjoNZAxVw

Version NN ELO Perf W D L Fire Komodo SF Score Games % Fire % Komodo % SF % Total
0.16 600 3428 56 186 118 61 48.5 39.5 149 360 50.83 40.42 32.92 41.39
0.16 643 3428 56 186 118 58.5 52 38.5 149 360 48.75 43.33 32.08 41.39
0.16 809 3428 56 188 116 56 51.5 42.5 150 360 46.67 42.92 35.42 41.67
0.16 695 3435 69 169 122 66 45.5 42 153.5 360 55 37.92 35 42.64
0.16 776 3435 61 187 112 59.5 54 41 154.5 360 49.58 45 34.17 42.92
0.16 928 3442 67 179 114 61.5 54.5 40.5 156.5 360 51.25 45.42 33.75 43.47
0.16 840 3456 64 198 98 67 53.5 42.5 163 360 55.83 44.58 35.42 45.28
0.17 1066 3463 77 184 99 70.5 51 47.5 169 360 58.75 42.5 39.58 46.94
0.17 1120 3470 77 189 94 68.5 54 49 171.5 360 57.08 45 40.83 47.64
0.17 1186 3449 68 183 109 59 57 43.5 159.5 360 49.17 47.5 36.25 44.31
In this Arena output, the third row is the nodes searched, and the fourth is the speed? If yes, then why after 1 second you have 36,000 nodes searched with 2,600 nodes per second? Second thing, my NPS both in Arena (3.51) and from command line at 1 second are about 2,000 and increasing slowly to 4,000 in similar position to 30s. My NPS are increasing pretty slowly, is it the case with your NPS too? Maybe I have some issues with drivers or CUDA/CUDNN software? I refreshed them one month ago or so.
I have done another experiment with lc0 ver. 0.17 11261 on another computer with GTX 1070, i7 7700H 2.8 GHz and Cutechess Gui, trying to observe the NPS and total amount of nodes of lc0 as you said.
As you can see in the following videos

https://youtu.be/nJdAT9w4LVI

lc0 with black, in the first move the node counter starts from 0, in the second move starts from 8683, in the third move from 13790, in the 4th from 5923, in the 5th from 19525 ...

https://youtu.be/xmPkY3mdVKY
another example with the same behaviour

I think this is due to the cache of lc0 that, in my case, has the default values of the installation.
Perhaps you have something wrong with the installation of lc0 or the CUDA drivers, I have installed them only once time and never updated them.
The same thing, you play repeatedly the same games with the same cache. How is that search can start at depths 9,10,11 in the early phase of the game using just 50ms? No way. Here is a table for time used per move (seconds), NPS and depths achieved in at least 100 gameplay positions of opening - early middlegame (moves 5-19):

Code: Select all

time   NPS   depth
===================
 0.1  1050   4.69
 0.3  1800   5.49
 1.0  2200   6.58
 3.0  2600   7.85
10.0  2900   8.77
30.0  3000  10.83
Can you re-start engines in Arena (there is an option, I think) after each played game? You will get rid of that behavior if it is a cache issue. Your behavior is undesirable, as you can hardly measure correctly the strength of Lc0.
I didn't repeat any game.
I have done several videos again to show you from the starting of Arena or Cutechess that the phenomenon is present again.

https://youtu.be/NbB3P1Ex56Q

https://youtu.be/ap-ZaaiLYdQ

Also trying the Restarting engine Option in Arena.

https://youtu.be/ufx_4t5OqBk

My behavior is that of a scientist who wants to clarify what happens.
I don't know what is the difference between your lc0 installation and mine but these facts are real.
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: leela is official(?) better than sf9

Post by Javier Ros » Sat Sep 15, 2018 1:56 pm

These are my lc0 settings as they appear in Cutechess, please compare with yours.
lc0Settings.JPG
lc0Settings.JPG (78.39 KiB) Viewed 1572 times
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

jp
Posts: 702
Joined: Mon Apr 23, 2018 5:54 am

Re: leela is official(?) better than sf9

Post by jp » Sat Sep 15, 2018 4:09 pm

Does cutechess GUi or cli by default clear the cache between games?

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: leela is official(?) better than sf9

Post by Javier Ros » Sat Sep 15, 2018 9:48 pm

Laskos wrote:
Sat Sep 15, 2018 11:54 am
Javier Ros wrote:
Sat Sep 15, 2018 9:28 am
Laskos wrote:
Wed Sep 12, 2018 7:36 pm
Javier Ros wrote:
Wed Sep 12, 2018 6:38 pm
Laskos wrote:
Wed Sep 12, 2018 6:02 pm

I don't understand the discrepancy. Are you sure your SF9 uses 6 threads and not 1? This result would indicate 1 thread is used, but maybe I am doing something very wrong. OTOH, CCCC result, which I expected to come close to my result, because the CPU effective speed-up is about 8 compared to my 4 i7 cores, and GPU speed-up is again about 8 compared to to my GTX 1060, shows, like in my case, that Lc0 is the level of Fire 7.1. I use adjudication only for very long games, above 100 moves with evals in [-20cp, 20cp] range (usually very drawish long endgames). I use no win adjudication.
http://talkchess.com/forum3/download/file.php?id=97

I am sure Stockfish uses 6 threads of the 8 threads (4 cores), you can see the 74% in the task manager for Stockfish 9 and 0% for lc0 due to Ponder Off and also the NPS counter is 8.7 million.

I use adjudication by Arena at -9.00

I have read in this link that the net 11120 is better than the rest of 11xxx and I am testing it right now. At this moment Stockfish 6.5-lc017 11120 6.5

https://groups.google.com/forum/#!topic ... IUjoNZAxVw

Version NN ELO Perf W D L Fire Komodo SF Score Games % Fire % Komodo % SF % Total
0.16 600 3428 56 186 118 61 48.5 39.5 149 360 50.83 40.42 32.92 41.39
0.16 643 3428 56 186 118 58.5 52 38.5 149 360 48.75 43.33 32.08 41.39
0.16 809 3428 56 188 116 56 51.5 42.5 150 360 46.67 42.92 35.42 41.67
0.16 695 3435 69 169 122 66 45.5 42 153.5 360 55 37.92 35 42.64
0.16 776 3435 61 187 112 59.5 54 41 154.5 360 49.58 45 34.17 42.92
0.16 928 3442 67 179 114 61.5 54.5 40.5 156.5 360 51.25 45.42 33.75 43.47
0.16 840 3456 64 198 98 67 53.5 42.5 163 360 55.83 44.58 35.42 45.28
0.17 1066 3463 77 184 99 70.5 51 47.5 169 360 58.75 42.5 39.58 46.94
0.17 1120 3470 77 189 94 68.5 54 49 171.5 360 57.08 45 40.83 47.64
0.17 1186 3449 68 183 109 59 57 43.5 159.5 360 49.17 47.5 36.25 44.31
In this Arena output, the third row is the nodes searched, and the fourth is the speed? If yes, then why after 1 second you have 36,000 nodes searched with 2,600 nodes per second? Second thing, my NPS both in Arena (3.51) and from command line at 1 second are about 2,000 and increasing slowly to 4,000 in similar position to 30s. My NPS are increasing pretty slowly, is it the case with your NPS too? Maybe I have some issues with drivers or CUDA/CUDNN software? I refreshed them one month ago or so.
I have done another experiment with lc0 ver. 0.17 11261 on another computer with GTX 1070, i7 7700H 2.8 GHz and Cutechess Gui, trying to observe the NPS and total amount of nodes of lc0 as you said.
As you can see in the following videos

https://youtu.be/nJdAT9w4LVI

lc0 with black, in the first move the node counter starts from 0, in the second move starts from 8683, in the third move from 13790, in the 4th from 5923, in the 5th from 19525 ...

https://youtu.be/xmPkY3mdVKY
another example with the same behaviour

I think this is due to the cache of lc0 that, in my case, has the default values of the installation.
Perhaps you have something wrong with the installation of lc0 or the CUDA drivers, I have installed them only once time and never updated them.
The same thing, you play repeatedly the same games with the same cache. How is that search can start at depths 9,10,11 in the early phase of the game using just 50ms? No way. Here is a table for time used per move (seconds), NPS and depths achieved in at least 100 gameplay positions of opening - early middlegame (moves 5-19):

Code: Select all

time   NPS   depth
===================
 0.1  1050   4.69
 0.3  1800   5.49
 1.0  2200   6.58
 3.0  2600   7.85
10.0  2900   8.77
30.0  3000  10.83
Can you re-start engines in Arena (there is an option, I think) after each played game? You will get rid of that behavior if it is a cache issue. Your behavior is undesirable, as you can hardly measure correctly the strength of Lc0.
The final proof that lc0 uses the cache properly and the total amount of nodes can start from a value greater than zero from one move to another, is the following experiment with random election of the starting position from o-deville.pgn with 11586 games from Frank Quisinsky

in Arena
https://youtu.be/_CdKn-yZKIs

in Cutechess
https://youtu.be/FkMSb-5Wfj0

One can check that at the beginning of the game the cache is empty and the amount of nodes starts from zero, but as the game progresses the amonunt of nodes can start from values greater than zero depending on the content of the cache.

If the amount of nodes start always from zero it indicates that the cache of lc0 doesn't work well.
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: leela is official(?) better than sf9

Post by Javier Ros » Sat Sep 15, 2018 9:51 pm

jp wrote:
Sat Sep 15, 2018 4:09 pm
Does cutechess GUi or cli by default clear the cache between games?
In cutechess cli you can specify if you want reset the engine or not, see

https://github.com/cutechess/cutechess/ ... c/help.txt


Engine options:

conf=NAME Use an engine with the name NAME from Cute Chess'
engines.json configuration file.
name=NAME Set the name to NAME
cmd=COMMAND Set the command to COMMAND
dir=DIR Set the working directory to DIR
arg=ARG Pass ARG to the engine as a command line argument
initstr=TEXT Send TEXT to the engine's standard input at startup.
TEXT may contain multiple lines seprated by '\n'.
stderr=FILE Redirect standard error output to FILE
restart=MODE Set the restart mode to MODE which can be:
'auto': the engine decides whether to restart (default)
'on': the engine is always restarted between games
'off': the engine is never restarted between games
Setting this option does not prevent engines from being
restarted between rounds in a tournament featuring more
than two engines.
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: leela is official(?) better than sf9

Post by Javier Ros » Sun Sep 16, 2018 9:22 pm

It is very instructive to examine the debug output of the engine lc0.

I am giving here an example calculated in my computer but in the following link appears another similar debug output

https://gist.github.com/zz4032/7af71a69 ... a3d27cb0bb

The following output has been obtained with Cutechess doing a tournament between lc0 11261 and Fire 7 with time control 5 minutes and 3 sec starting from the initial position. (GPU GTX 1070 and CPU i7 7700H)

At the first line I highlight time 38 milliseconds, nodes 4 and hashfull 0. For being at the start of the game the nodes have started from 0 (the 4 nodes correspond to the 38 millisecs) and the hash table is empty.

<lc011261(0): info depth 1 seldepth 2 time 38 nodes 4 score cp 16 hashfull 0 nps 105 tbhits 0 pv g1f3 g8f6
<lc011261(0): info depth 1 seldepth 2 time 53 nodes 9 score cp 21 hashfull 0 nps 169 tbhits 0 pv e2e4 c7c5
<lc011261(0): info depth 2 seldepth 3 time 69 nodes 27 score cp -1 hashfull 0 nps 391 tbhits 0 pv e2e3 g8f6 g1f3
<lc011261(0): info depth 2 seldepth 3 time 85 nodes 50 score cp 7 hashfull 0 nps 588 tbhits 0 pv c2c4 c7c5 e2e3
<lc011261(0): info depth 2 seldepth 4 time 101 nodes 77 score cp 9 hashfull 0 nps 762 tbhits 0 pv c2c4 c7c5 e2e3 b8a6
<lc011261(0): info depth 3 seldepth 4 time 131 nodes 157 score cp 12 hashfull 0 nps 1198 tbhits 0 pv c2c4 c7c5 g1f3 g8f6
<lc011261(0): info depth 3 seldepth 5 time 147 nodes 185 score cp 13 hashfull 0 nps 1258 tbhits 0 pv c2c4 c7c5 g1f3 g8f6
<lc011261(0): info depth 3 seldepth 5 time 191 nodes 279 score cp 6 hashfull 1 nps 1460 tbhits 0 pv d2d4 g8f6 c2c4 e7e6 g1f3 b8a6
<lc011261(0): info depth 3 seldepth 6 time 248 nodes 466 score cp 10 hashfull 2 nps 1879 tbhits 0 pv d2d4 g8f6 g1f3 e7e6 c2c4 d7d5
<lc011261(0): info depth 3 seldepth 6 time 287 nodes 584 score cp 13 hashfull 2 nps 2034 tbhits 0 pv c2c4 c7c5 e2e3 g8f6 g1f3
<lc011261(0): info depth 4 seldepth 7 time 320 nodes 680 score cp 13 hashfull 3 nps 2125 tbhits 0 pv d2d4 d7d5 c2c4 e7e6 g1f3 g8f6
<lc011261(0): info depth 4 seldepth 7 time 360 nodes 796 score cp 19 hashfull 3 nps 2211 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 a1b1
<lc011261(0): info depth 4 seldepth 8 time 400 nodes 924 score cp 19 hashfull 4 nps 2310 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 d2d4
<lc011261(0): info depth 4 seldepth 9 time 503 nodes 1281 score cp 23 hashfull 6 nps 2546 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 d2d4
<lc011261(0): info depth 4 seldepth 10 time 590 nodes 1571 score cp 26 hashfull 7 nps 2662 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 d2d4 a8b8
<lc011261(0): info depth 5 seldepth 10 time 976 nodes 3026 score cp 25 hashfull 13 nps 3100 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 d2d4 c5d4
<lc011261(0): info depth 5 seldepth 11 time 1431 nodes 4819 score cp 22 hashfull 20 nps 3367 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 d2d4 c5d4
<lc011261(0): info depth 5 seldepth 12 time 1879 nodes 6611 score cp 19 hashfull 27 nps 3518 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 d2d4 c5d4
<lc011261(0): info depth 5 seldepth 13 time 2506 nodes 9171 score cp 17 hashfull 37 nps 3659 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g8f6 d2d4 c5d4 a1b1
<lc011261(0): info depth 5 seldepth 14 time 3289 nodes 12499 score cp 14 hashfull 50 nps 3800 tbhits 0 pv e2e4 c7c5 g1f3 d7d6 b1c3 g8f6 d2d4 c5d4 f3d4 g7g6
<lc011261(0): info depth 5 seldepth 15 time 4103 nodes 16083 score cp 15 hashfull 63 nps 3919 tbhits 0 pv e2e4 c7c5 g1f3 d7d6 b1c3 g8f6 d2d4 c5d4 f3d4 g7g6 f1b5
<lc011261(0): info depth 5 seldepth 16 time 4345 nodes 17107 score cp 16 hashfull 67 nps 3937 tbhits 0 pv e2e4 c7c5 g1f3 d7d6 b1c3 g8f6 d2d4 c5d4 f3d4 g7g6 f1b5 c8d7
<lc011261(0): info depth 6 seldepth 16 time 5403 nodes 21433 score cp 19 hashfull 83 nps 3966 tbhits 0 pv e2e4 e7e6 d2d4 d7d5 b1c3 g8f6 e4e5 f6d7 f2f4 c7c5 g1f3 b8c6 c1e3
<lc011261(0): info depth 6 seldepth 17 time 5592 nodes 22202 score cp 19 hashfull 86 nps 3970 tbhits 0 pv e2e4 e7e6 d2d4 d7d5 b1c3 g8f6 e4e5 f6d7 f2f4 c7c5 g1f3 b8c6 c1e3
<lc011261(0): info depth 6 seldepth 18 time 6905 nodes 27839 score cp 20 hashfull 108 nps 4031 tbhits 0 pv e2e4 c7c5 g1f3 b8c6 b1c3 g7g6 d2d4 c5d4 f3d4 f8g7 c1e3 g8f6 f1e2
<lc011261(0): info depth 6 seldepth 19 time 8312 nodes 33987 score cp 20 hashfull 130 nps 4088 tbhits 0 pv e2e4 c7c5 g1f3 d7d6 b1c3 g8f6 d2d4 c5d4 f3d4 g7g6 f1b5 c8d7 b5d7
<lc011261(0): info depth 6 seldepth 19 time 12624 nodes 52956 score cp 18 hashfull 199 nps 4194 tbhits 0 pv e2e4 c7c5 g1f3 d7d6 b1c3 g8f6 d2d4 c5d4 f3d4 g7g6 f1b5 c8d7 b5d7
<lc011261(0): bestmove e2e4 ponder c7c5

lc0 finished the calculation at depth ply 6 and selective depth 19, with hasfull 199 which means 19.9 % of the hash tables filled and played 1.e4


>Fire_7(1): position startpos moves e2e4
>Fire_7(1): isready
<Fire_7(1): readyok
>Fire_7(1): go wtime 290376 btime 300000 winc 3000 binc 3000
<Fire_7(1): info time 114 nodes 320890 nps 2814824 tbhits 0 depth 8 multipv 1 score cp -32 pv e7e5 g1f3 b8c6 b1c3 f8c5 f1b5 c6d4 b5a4
...
<Fire_7(1): info time 9402 nodes 27172080 nps 2890031 tbhits 0 depth 16 multipv 1 score cp -12 pv e7e6 d2d4 d7d5 b1c3 g8f6 c1g5 f8e7 e4e5 f6d7 h2h4 h7h6 g5e3 c7c5 d1g4 e8f8 g1f3
<Fire_7(1): bestmove e7e6 ponder d2d4
>lc011261(0): position startpos moves e2e4 e7e6
>lc011261(0): isready
<lc011261(0): readyok
>lc011261(0): go wtime 290376 btime 293599 winc 3000 binc 3000

After the moves 1.e4 e6, lc0 starts from time 0, nodes 8859 and hashfull 200. The amount of nodes doesn´t start from 0 because lc0 uses the information stored in the hash table and continues from that point.
lc0 started the calculation at depth ply 4 and selective depth 4, less than depth ply 6/19 this happened because the move e6 of Fire was not guessed by lc0, though e6 was analized at depth 6 seldepth 16 and depth 6 seldepth 17.


<lc011261(0): info depth 4 seldepth 4 time 0 nodes 8859 score cp 18 hashfull 200 nps 0 tbhits 0 pv d2d4 d7d5 b1c3 g8f6 e4e5 f6d7 f2f4 c7c5 g1f3 b8c6 c1e3 d8b6 c3a4
<lc011261(0): info depth 6 seldepth 8 time 0 nodes 8860 score cp 18 hashfull 200 nps 0 tbhits 0 pv d2d4 d7d5 b1c3 g8f6 e4e5 f6d7 f2f4 c7c5 g1f3 b8c6 c1e3 d8b6 c3a4
<lc011261(0): info depth 6 seldepth 13 time 71 nodes 9116 score cp 18 hashfull 201 nps 3633 tbhits 0 pv d2d4 d7d5 b1c3 g8f6 e4e5 f6d7 f2f4 c7c5 g1f3 b8c6 c1e3 d8b6 c3a4
<lc011261(0): info depth 6 seldepth 14 time 434 nodes 10549 score cp 21 hashfull 207 nps 3896 tbhits 0 pv d2d4 d7d5 b1c3 g8f6 e4e5 f6d7 c3e2 c7c5 c2c3 b8c6 g1f3 f7f6 e2f4 d8e7
<lc011261(0): info depth 6 seldepth 15 time 2639 nodes 20350 score cp 19 hashfull 242 nps 4354 tbhits 0 pv d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1
<lc011261(0): info depth 7 seldepth 15 time 4942 nodes 30339 score cp 21 hashfull 279 nps 4346 tbhits 0 pv d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1
<lc011261(0): info depth 7 seldepth 16 time 6980 nodes 39314 score cp 20 hashfull 311 nps 4363 tbhits 0 pv d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1
<lc011261(0): info depth 7 seldepth 16 time 11999 nodes 62871 score cp 19 hashfull 390 nps 4501 tbhits 0 pv d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4
<lc011261(0): info depth 7 seldepth 17 time 14464 nodes 74413 score cp 18 hashfull 429 nps 4532 tbhits 0 pv d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4
<lc011261(0): info depth 7 seldepth 17 time 17552 nodes 88264 score cp 18 hashfull 478 nps 4524 tbhits 0 pv d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): bestmove d2d4 ponder d7d5

lc0 finished the calculation at depth ply 7 and selective depth 17

>Fire_7(1): position startpos moves e2e4 e7e6 d2d4
>Fire_7(1): isready
<Fire_7(1): readyok
>Fire_7(1): go wtime 275824 btime 293599 winc 3000 binc 3000
<Fire_7(1): info time 2 nodes 4356 nps 2178000 tbhits 0 depth 8 multipv 1 score cp -12 pv d7d5 b1c3 g8f6 c1g5 f8e7 e4e5 f6d7 h2h4
...
<Fire_7(1): info time 13302 nodes 39339735 nps 2957430 tbhits 0 depth 17 multipv 1 score cp -20 pv d7d5 b1d2 g8f6 f1d3 c7c5 e4e5 f6d7 c2c3 b8c6 g1e2 f8e7 d2f3 b7b6 e1g1 c8b7 f1e1 h7h6
<Fire_7(1): bestmove d7d5 ponder b1d2
>lc011261(0): position startpos moves e2e4 e7e6 d2d4 d7d5

After 1.e4 e6 2.d4 d5, lc0 at time 66 millisecs, starts with nodes 27821 and hashfull 480. The amount of nodes doesn´t start from 0 again.
lc0 started the calculation at depth ply 8 and selective depth 14 continuing from the depth 7/17 reached in the previous move probably due to that the move d5 of Fire was guessed by lc0. During the computation the depth varied from 7 to 8 several times

>lc011261(0): isready
<lc011261(0): readyok
>lc011261(0): go wtime 275824 btime 283299 winc 3000 binc 3000
<lc011261(0): info depth 8 seldepth 14 time 66 nodes 27821 score cp 20 hashfull 480 nps 3878 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 7 seldepth 14 time 188 nodes 28333 score cp 20 hashfull 482 nps 4085 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 7 seldepth 15 time 773 nodes 30893 score cp 20 hashfull 491 nps 4305 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 8 seldepth 15 time 997 nodes 31919 score cp 20 hashfull 495 nps 4367 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 7 seldepth 15 time 1062 nodes 32175 score cp 20 hashfull 496 nps 4340 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 8 seldepth 15 time 1182 nodes 32687 score cp 19 hashfull 498 nps 4333 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 7 seldepth 15 time 1310 nodes 33199 score cp 19 hashfull 500 nps 4300 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 8 seldepth 15 time 1378 nodes 33457 score cp 19 hashfull 501 nps 4275 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 7 seldepth 15 time 1436 nodes 33713 score cp 19 hashfull 502 nps 4281 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 c7c5 e1g1 c5d4 f3d4 b8c6 c4d5
<lc011261(0): info depth 8 seldepth 15 time 1486 nodes 33969 score cp 19 hashfull 503 nps 4309 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 b4c3 b2c3 d5c4 e2c4 f8e8 c1e3
<lc011261(0): info depth 8 seldepth 16 time 2761 nodes 39354 score cp 19 hashfull 523 nps 4269 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 f8b4 b1c3 e8g8 f1e2 b4c3 b2c3 d5c4 e2c4 f8e8 c1e3
<lc011261(0): info depth 8 seldepth 16 time 7779 nodes 61575 score cp 18 hashfull 604 nps 4372 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 c8e6 b1c3 d5c4 f1e2 b8c6 e1g1 f8e7 d1a4 e8g8
<lc011261(0): info depth 8 seldepth 17 time 12176 nodes 80180 score cp 17 hashfull 674 nps 4321 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 c8e6 b1c3 d5c4 f3g5 e6d5 c3d5 d8d5 f1e2 f8b4 e1f1
<lc011261(0): info depth 8 seldepth 17 time 16601 nodes 98634 score cp 17 hashfull 745 nps 4281 tbhits 0 pv e4d5 e6d5 g1f3 g8f6 c2c4 c8e6 b1c3 d5c4 d1a4 c7c6 f1c4 b7b5 c3b5 c6b5 c4b5
<lc011261(0): bestmove e4d5 ponder e6d5
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: leela is official(?) better than sf9

Post by Javier Ros » Tue Sep 18, 2018 9:50 pm

Final result with 30 positions Noomen Sharp Gambits

Lc01711261 - Stockfish_9_x64_bmi2 : 41,5/120 9-46-65 (=0===00==000=010==01000000=1=0=======0=0=======0=0=0=0=000=1==10=0===0=1=0=0=====1==0001=01====0===0=00=0==0=0=0===000=0) 35%
Stockfish_9_x64_bmi2 - Lc01711261 : 78,5/120 46-9-65 (=1===11==111=101==10111111=0=1=======1=1=======1=1=1=1=111=0==01=1===1=0=1=1=====0==1110=10====1===1=11=1==1=1=1===111=1) 65%

This score for Stockfish at tactical positions is better than the obtained with more quiet positions, like Nunn´s ones 58% and 12 positions from AplhaZero-Stockfish test 60%.


Games 47-81


Last edited by Javier Ros on Tue Sep 18, 2018 9:53 pm, edited 1 time in total.
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: leela is official(?) better than sf9

Post by Javier Ros » Tue Sep 18, 2018 9:51 pm

Games 82-120

The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Post Reply