Stockfish 12 dominating CCRL

[mwyoung]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

[RogerC]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)
Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

Yes at Very long TC LC0 is quite good, how about 32 thread at Blitz TC (2 min+1 s) ?

[mwyoung]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)
Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

Yes at Very long TC LC0 is quite good, how about 32 thread at Blitz TC (2 min+1 s) ?

And lets be clear, 15m+15s is not very long time controls. That is only relative to the described testing below.

Lc0 gets crushed, as my testing has shown. And again my warning about fast only testing, and 1 tier testing at 1 core! At blitz, and ultra bullet fast only time controls! With Stockfish NNUE. Stockfish NNUE is the ultimate sprinter....and I give much credit, and thanks. It is a awesome chess engine.

[Uri Blass]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

The question is if it is the fault of stockfish.

Maybe it is simply impossible to win at long time control against top engines.
I do not see stockfish losing in your results.

Scales badly means for me something like the following:
Stockfish get 50% at 5 seconds per move against 10 seconds per move of LC0 but get less than 50% at 50 seconds per move against 100 seconds per move of LC0

Do you have some evidence for this(maybe with different time controls but the idea is clear)?

[mwyoung]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

The question is if it is the fault of stockfish.

Maybe it is simply impossible to win at long time control against top engines.
I do not see stockfish losing in your results.

Scales badly means for me something like the following:
Stockfish get 50% at 5 seconds per move against 10 seconds per move of LC0 but get less than 50% at 50 seconds per move against 100 seconds per move of LC0

Do you have some evidence for this(maybe with different time controls but the idea is clear)?

This is a possible, and if true means the death of chess engine development. Because if chess is dead at 15m+15s on a 32 thread system. Logically you know what the means in a very short time...

The formula is simple....

Computer speed is equal to time!

[Laskos]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

The question is if it is the fault of stockfish.

Maybe it is simply impossible to win at long time control against top engines.
I do not see stockfish losing in your results.

Scales badly means for me something like the following:
Stockfish get 50% at 5 seconds per move against 10 seconds per move of LC0 but get less than 50% at 50 seconds per move against 100 seconds per move of LC0

Do you have some evidence for this(maybe with different time controls but the idea is clear)?

This is a possible, and if true means the death of chess engine development. Because if chess is dead at 15m+15s on a 32 thread system. Logically you know what the means in a very short time...

Luckily TCEC and other LTC tournaments don't use your openings.

[mwyoung]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

The question is if it is the fault of stockfish.

Maybe it is simply impossible to win at long time control against top engines.
I do not see stockfish losing in your results.

Scales badly means for me something like the following:
Stockfish get 50% at 5 seconds per move against 10 seconds per move of LC0 but get less than 50% at 50 seconds per move against 100 seconds per move of LC0

Do you have some evidence for this(maybe with different time controls but the idea is clear)?

This is a possible, and if true means the death of chess engine development. Because if chess is dead at 15m+15s on a 32 thread system. Logically you know what the means in a very short time...

Luckily TCEC and other LTC tournaments don't use your openings.

Yes, but most testing sites do. Moron!

And if chess is dead at 6 moves. With the chess engines playing most of the opening. We have a much bigger problem!

And it is not my opening book! But the best testing book I have found. Sedatchess Perfect book 2019. Yell at him for the openings.

https://sites.google.com/site/computers ... t2019books

[abgursu]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

The question is if it is the fault of stockfish.

Maybe it is simply impossible to win at long time control against top engines.
I do not see stockfish losing in your results.

Scales badly means for me something like the following:
Stockfish get 50% at 5 seconds per move against 10 seconds per move of LC0 but get less than 50% at 50 seconds per move against 100 seconds per move of LC0

Do you have some evidence for this(maybe with different time controls but the idea is clear)?

This is a possible, and if true means the death of chess engine development. Because if chess is dead at 15m+15s on a 32 thread system. Logically you know what the means in a very short time...

Luckily TCEC and other LTC tournaments don't use your openings.

Yes, but most testing sites do. Moron!

And if chess is dead at 6 moves. With the chess engines playing most of the opening. We have a much bigger problem!

And it is not my opening book! But the best testing book I have found. Sedatchess Perfect book 2019. Yell at him for the openings.

https://sites.google.com/site/computers ... t2019books

What about Drawkillers?

[mwyoung]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

The question is if it is the fault of stockfish.

Maybe it is simply impossible to win at long time control against top engines.
I do not see stockfish losing in your results.

Scales badly means for me something like the following:
Stockfish get 50% at 5 seconds per move against 10 seconds per move of LC0 but get less than 50% at 50 seconds per move against 100 seconds per move of LC0

Do you have some evidence for this(maybe with different time controls but the idea is clear)?

This is a possible, and if true means the death of chess engine development. Because if chess is dead at 15m+15s on a 32 thread system. Logically you know what the means in a very short time...

Luckily TCEC and other LTC tournaments don't use your openings.

Yes, but most testing sites do. Moron!

And if chess is dead at 6 moves. With the chess engines playing most of the opening. We have a much bigger problem!

And it is not my opening book! But the best testing book I have found. Sedatchess Perfect book 2019. Yell at him for the openings.

https://sites.google.com/site/computers ... t2019books

What about Drawkillers?

Yes, but Drawkillers are A.K.A inferior openings. Remember 1 win for white, and 1 win for white with the same opening with 2 different engines with equal strength is the same as 1/2 and 1/2. But I guess some like TCEC have a motivation for wins. I do not.... I will not force a engine to play an inferior opening for wins, for the sake of wins....

Remember the book is only to 6 moves....and the engine plays the rest of the opening.

[Raphexon]

Thanks, so it seems RTX2060 is a good match for 1 fast thread. Still, it's close, RTX2080 is not even 2 times faster. CCRL seems to show 50 Elo advantage of SF 12 over Lc0 1541 2080, which I guess are either $$$ or an artifact of the rating list with many much weaker opponents.

SF12 has not actually played any of the RTX2080 engines head-to-head (different tester has the RTX) so we need to take care with that initial rating. I think things could change a little once those games happen.

As i said, Stockfish 12 is dominating CCRL , but CEGT also !
http://www.cegt.net/40_40%20Rating%20Li ... liste.html
Here are the results of the matchs between SF12 and opponents : SF12 (1 CPU) vs LCZero 0.25.1 Cuda (LS15.0) : 100 games , 11 wins, 86 draws, 3 losses.
http://www.cegt.net/40_40%20Rating%20Li ... ons/1.html

SF12 on 8 cores will crush LC0... LC0 is the past, and is finished because it needs more and more Graphical power (2000+ Cores) to compute and evaluate half dozen kn/s. On the other hand SF do an intelligent mix of classical and NNUE evaluations on just a single CPU Core. And finally, NNUE nets are just at the very beginning and early of there trainings (2 month old). See you in 12 months !

Stockfish 12 is the best engine, but still scales badly. The best engine in most TC, but crushing! Still not so with Lc0....

Be careful!

This is just the latest test.

Live Stockfish 12 vs Lc0 26.2 (NN-J92-115), (32 Threads), (TC=15m+15s), (40 Games)

Hardware 2950x, RTX 2080 Ti

Stockfish 12 Default NN
Lc0 26.2 (NN-J92-115)
Ponder off.
TC=15m+15s
32 threads.
4 Gb hash.
6 man TB, and the top ten 7 man TB.
Opening book 6 moves.
Default settings.

Code: Select all

Result:
--------------------------------------------------------------------------
  #  name          games    wins   draws  losses   score    los%  elo+/-
  1. Stockfish 12     40       2      38       0    21.0    92.1    17.4
  2. Lc0 v0.26.2      40       0      38       2    19.0     7.9   -17.4

Cross table:
--------------------------------------------------------------------------
  #  name             score   games                                        1                                        2
  1. Stockfish 12      21.0      40                                        x =========1==========1===================
  2. Lc0 v0.26.2       19.0      40 =========0==========0===================                                        x

Tech:
--------------------------------------------------------------------------

Tech (average nodes, depths, time/m per move, others per game), counted for computing moves only, ignored moves with zero nodes:
  #  name            nodes/m         NPS  depth/m   time/m    moves     time   #fails
  1. Stockfish 12    715650K    25949227     59.1     27.6     58.8   1622.3         
  2. Lc0 v0.26.2        847K       29221     11.6     29.0     58.8   1704.3        1
     all ---         349926K    12669925     35.3     28.3     58.8   1663.3        1

The question is if it is the fault of stockfish.

Maybe it is simply impossible to win at long time control against top engines.
I do not see stockfish losing in your results.

Scales badly means for me something like the following:
Stockfish get 50% at 5 seconds per move against 10 seconds per move of LC0 but get less than 50% at 50 seconds per move against 100 seconds per move of LC0

Do you have some evidence for this(maybe with different time controls but the idea is clear)?

This is a possible, and if true means the death of chess engine development. Because if chess is dead at 15m+15s on a 32 thread system. Logically you know what the means in a very short time...

Luckily TCEC and other LTC tournaments don't use your openings.

Yes, but most testing sites do. Moron!

And if chess is dead at 6 moves. With the chess engines playing most of the opening. We have a much bigger problem!

And it is not my opening book! But the best testing book I have found. Sedatchess Perfect book 2019. Yell at him for the openings.

https://sites.google.com/site/computers ... t2019books

What about Drawkillers?

Yes, but Drawkillers are A.K.A inferior openings. Remember 1 win for white, and 1 win for white with the same opening with 2 different engines with equal strength is the same as 1/2 and 1/2. But I guess some like TCEC have a motivation for wins. I do not.... I will not force a engine to play an inferior opening for wins, for the sake of wins....

Remember the book is only to 6 moves....and the engine plays the rest of the opening.

Have you ever seen the drawkiller opening set?

They're not guaranteed wins, they're extremely complex that can lead to both black and white wins.
When I did my 5000 game test white barely had an advantage over black.

The moves to get into that position are extremely dubious, like extremely and barely resemble chess.
But the end position is balanced.

I actually did a 5000 game match between SF12 and SF11 at 60+0.6
Results were +2862 -356 =1782 for SF12
Elo 191.4 +/- 8.0

White vs black score was +1711 -1507 =1782 (52% score for white)
So the drawkiller opening set is actually more balanced than regular opening positions.


Your assumption of the drawkiller set is wrong. They don't lead to forced wins.
They're a combination of dubious moves that lead the engines into a spaghetti position that's somehow complex yet balanced.