Stockfish 10 Strikes Back

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Javier Ros
Posts: 185
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Stockfish 10 Strikes Back

Post by Javier Ros » Wed Dec 19, 2018 7:16 pm

The appearance of Alpha Zero a year ago made the world of computer chess tremble with its victory over Stockfish 8. Then Leela Chess Zero appeared and gave us the possibility to have NN chess programs in our personal computers.
During this time the Stockfish team has continued working well and has developed Stockfish 10 that significantly improves previous versions.

This improvement makes a possible confrontation Alpha Zero -Stockfish 10 more interesting than ever.

In the thread

http://talkchess.com/forum3/viewtopic.php?f=2&t=68524

I analyzed some games where Lc0 played better than Stockfish in the position 50 of Silver Test Suite, an interesting King's Indian position where Black's attack is powerful.

Now Stockfish 10 wins with Black and White pieces.

Stockfish 10 played on six logical cores of i7 4790S, Lc0 0.19.1.1 on RTX 2070 Games at 25 minutes plus 10 sec.



Stockfish 10 played on six logical cores of i7 980X, Lc0 0.19.1.1 on 2x GTX 1060 3Gb Games at 5 minutes plus 3 sec.




The good play of Stockfish 10 in the previous games is confirmed by the following matches against different versions of Lc0 using the 50 positions of the Silver Test Suite.

Stockfish 10 played on six logical cores of i7 4790S 3.2 Ghz, Lc0 0.19.1.1 on RTX 2070 Games at 1 minute plus 1 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_bmi2 - Lc01911_11261 65.5 - 34.5 +38/=55/-7 65.50%
Stockfish_10_x64_bmi2 - Lc0191rc2_31956 65.0 - 35.0 +41/=48/-11 65.00%
Stockfish_10_x64_bmi2 - Lc01911_32007 71.0 - 29.0 +46/=50/-4 71.00%

Pgn of the games
https://sites.google.com/site/unoallavo ... ects=0&d=1

Stockfish 10 played on six logical cores of i7 4790S 3.2 Ghz, Lc0 0.19.1.1 on RTX 2070 Games at 5 minute plus 3 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_ +29/=64/-7 61.00% 61.0/100
Lc0191rc2_31956 +7/=64/-29 39.00% 39.0/100

Pgn of the games

https://sites.google.com/site/unoallavo ... ects=0&d=1

Stockfish 10 played on six logical cores of i7 980X, Lc0 0.19.1.1 on 2x GTX 1060 3Gb Games at 5 minutes plus 3 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_popcnt +52 +21/=73/-6 57.50% 57.5/100
Lc01911_11261x2 -52 +6/=73/-21 42.50% 42.5/100

Pgn of the games
https://sites.google.com/site/unoallavo ... ects=0&d=1
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish 10 Strikes Back

Post by Laskos » Wed Dec 19, 2018 7:27 pm

Javier Ros wrote:
Wed Dec 19, 2018 7:16 pm
The appearance of Alpha Zero a year ago made the world of computer chess tremble with its victory over Stockfish 8. Then Leela Chess Zero appeared and gave us the possibility to have NN chess programs in our personal computers.
During this time the Stockfish team has continued working well and has developed Stockfish 10 that significantly improves previous versions.

This improvement makes a possible confrontation Alpha Zero -Stockfish 10 more interesting than ever.

In the thread

http://talkchess.com/forum3/viewtopic.php?f=2&t=68524

I analyzed some games where Lc0 played better than Stockfish in the position 50 of Silver Test Suite, an interesting King's Indian position where Black's attack is powerful.

Now Stockfish 10 wins with Black and White pieces.

Stockfish 10 played on six logical cores of i7 4790S, Lc0 0.19.1.1 on RTX 2070 Games at 25 minutes plus 10 sec.



Stockfish 10 played on six logical cores of i7 980X, Lc0 0.19.1.1 on 2x GTX 1060 3Gb Games at 5 minutes plus 3 sec.




The good play of Stockfish 10 in the previous games is confirmed by the following matches against different versions of Lc0 using the 50 positions of the Silver Test Suite.

Stockfish 10 played on six logical cores of i7 4790S 3.2 Ghz, Lc0 0.19.1.1 on RTX 2070 Games at 1 minute plus 1 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_bmi2 - Lc01911_11261 65.5 - 34.5 +38/=55/-7 65.50%
Stockfish_10_x64_bmi2 - Lc0191rc2_31956 65.0 - 35.0 +41/=48/-11 65.00%
Stockfish_10_x64_bmi2 - Lc01911_32007 71.0 - 29.0 +46/=50/-4 71.00%

Pgn of the games
https://sites.google.com/site/unoallavo ... ects=0&d=1

Stockfish 10 played on six logical cores of i7 4790S 3.2 Ghz, Lc0 0.19.1.1 on RTX 2070 Games at 5 minute plus 3 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_ +29/=64/-7 61.00% 61.0/100
Lc0191rc2_31956 +7/=64/-29 39.00% 39.0/100

Pgn of the games

https://sites.google.com/site/unoallavo ... ects=0&d=1

Stockfish 10 played on six logical cores of i7 980X, Lc0 0.19.1.1 on 2x GTX 1060 3Gb Games at 5 minutes plus 3 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_popcnt +52 +21/=73/-6 57.50% 57.5/100
Lc01911_11261x2 -52 +6/=73/-21 42.50% 42.5/100

Pgn of the games
https://sites.google.com/site/unoallavo ... ects=0&d=1
Thanks, Javier. Haven't checked Silver suite, but the results reflect what I get from "not favoring Lc0/A0" opening test suite. Both results with RTX 2070 and GTX 1060. By the way, at short TC test30 is weaker than test10 nets, not sure at longer TC, never checked.

You got RTX 2070 too? :)

jp
Posts: 1438
Joined: Mon Apr 23, 2018 5:54 am

Re: Stockfish 10 Strikes Back

Post by jp » Wed Dec 19, 2018 7:42 pm

Javier Ros wrote:
Wed Dec 19, 2018 7:16 pm
the following matches against different versions of Lc0 using the 50 positions of the Silver Test Suite.

Stockfish 10 played on six logical cores of i7 4790S 3.2 Ghz, Lc0 0.19.1.1 on RTX 2070 Games at 1 minute plus 1 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_bmi2 - Lc01911_11261 65.5 - 34.5 +38/=55/-7 65.50%
Stockfish_10_x64_bmi2 - Lc0191rc2_31956 65.0 - 35.0 +41/=48/-11 65.00%
Stockfish_10_x64_bmi2 - Lc01911_32007 71.0 - 29.0 +46/=50/-4 71.00%

Stockfish 10 played on six logical cores of i7 4790S 3.2 Ghz, Lc0 0.19.1.1 on RTX 2070 Games at 5 minute plus 3 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_ +29/=64/-7 61.00% 61.0/100
Lc0191rc2_31956 +7/=64/-29 39.00% 39.0/100

Stockfish 10 played on six logical cores of i7 980X, Lc0 0.19.1.1 on 2x GTX 1060 3Gb Games at 5 minutes plus 3 sec. Lc0 used 6-men syzygy tablebases, SF10 not.

Stockfish_10_x64_popcnt +52 +21/=73/-6 57.50% 57.5/100
Lc01911_11261x2 -52 +6/=73/-21 42.50% 42.5/100
Javier, what sort of nps ratios do these give?

Javier Ros
Posts: 185
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: Stockfish 10 Strikes Back

Post by Javier Ros » Wed Dec 19, 2018 7:55 pm

jp wrote:
Wed Dec 19, 2018 7:42 pm
Javier, what sort of nps ratios do these give?
At initial position GTX 2x 1060 about 6000-7000 nps

RTX 2070 about 20.000-30.000
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

yanquis1972
Posts: 1766
Joined: Tue Jun 02, 2009 10:14 pm

Re: Stockfish 10 Strikes Back

Post by yanquis1972 » Wed Dec 19, 2018 8:01 pm

nice test, the other thing that stands out to me is test30s results vs test10. it's really remarkably close in strength if it's LR can be taken at more or less face value (only one drop so far).

SF has reigned supreme in KIDs among AB engines for years but to completely flip the results would seem to be quite an accomplishment. did you notice any particular theme in the games?

Javier Ros
Posts: 185
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: Stockfish 10 Strikes Back

Post by Javier Ros » Wed Dec 19, 2018 8:03 pm

Laskos wrote:
Wed Dec 19, 2018 7:27 pm

You got RTX 2070 too? :)

I have read your recommendation on the best value for money for GPU and I could not resist. :D


The acceleration achieved with fp16 is impressive!
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Javier Ros
Posts: 185
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: Stockfish 10 Strikes Back

Post by Javier Ros » Wed Dec 19, 2018 8:37 pm

yanquis1972 wrote:
Wed Dec 19, 2018 8:01 pm
nice test, the other thing that stands out to me is test30s results vs test10. it's really remarkably close in strength if it's LR can be taken at more or less face value (only one drop so far).

SF has reigned supreme in KIDs among AB engines for years but to completely flip the results would seem to be quite an accomplishment. did you notice any particular theme in the games?
In the KID games, Stockfish 10 played more actively on Queen side with White with Nb5 as a typical move.

These KID games were played expressly to see the behavior of Stockfish 10, but I had not checked the corresponding KID games of the previous matches. For this particular position, it seems that the fastest time controls benefit Lc0 while SF10 improves with more computing time.
The 11261 version was better than 30xxx. Here are the results:

SF10-Lc0 31956 +0 =2 -0 1-1 on RTX2070 5 min+3sec



SF10-Lc0 11261 +1 =0 -1 50% 1-1 on 2xGTX1060 5 min+3sec




SF10-Lc0 11261 +1 =0 -1 50% 1-1 on RTX2070 1 min+1sec
SF10-Lc0 31956 +2 =0 -0 100% 2-0 on RTX2070 1 min+1sec
SF10-Lc0 32007 +2 =0 -0 100% 2-0 on RTX2070 1 min+1sec

The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

Post Reply