StockFiNN Release and Scaling

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Gejsi Marku
Posts: 20
Joined: Wed Mar 11, 2020 3:02 pm
Full name: Gejsi Marku

Re: StockFiNN Release and Scaling

Post by Gejsi Marku » Tue Jul 07, 2020 4:35 pm

Just finished the test of 100 games at TC 3+3 between StockfiNN vs Stockfish Dev on 8 cores.
StockfiNN won the match with a W-L-D rate of 14-6-80

Here are some notable victories of StockfiNN



Game 56
SF vs SFNN 0-1




Game 94
SF vs SFNN 0-1


Joerg Oster
Posts: 767
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: StockFiNN Release and Scaling

Post by Joerg Oster » Tue Jul 07, 2020 5:03 pm

Rom77 wrote:
Tue Jul 07, 2020 3:51 pm
Joerg Oster wrote:
Tue Jul 07, 2020 8:38 am
Try one of these https://www.file-upload.net/download-14 ... UE.7z.html, Ed.

Based on current master https://github.com/nodchip/Stockfish, these are static compiles without AVX2 support, but with SSE41 instead,
popcount (modern) and bmi2 compiles included.
This is the fastest "modern" compilation I've ever run!

Joerg, can you make a "modern" compilation for a regular 256x2 network?
Find them here https://github.com/joergoster/Stockfish ... ses/tag/v1, please.
Source is here https://github.com/nodchip/Stockfish
Jörg Oster

Rom77
Posts: 39
Joined: Wed Oct 24, 2018 5:37 am
Full name: Roman Zhukov

Re: StockFiNN Release and Scaling

Post by Rom77 » Tue Jul 07, 2020 5:20 pm

Joerg Oster wrote:
Tue Jul 07, 2020 5:03 pm
Find them here https://github.com/joergoster/Stockfish ... ses/tag/v1, please.
Source is here https://github.com/nodchip/Stockfish
Thanks!

dkappe
Posts: 468
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: StockFiNN Release and Scaling

Post by dkappe » Tue Jul 07, 2020 10:32 pm

lkaufman wrote:
Tue Jul 07, 2020 6:26 am
Komodo rules!
Can we expect a net trained with Komodo 14 evals and pv? I’d do it myself, but all of my compute resources are busy training Bad Gyal 9.

lkaufman
Posts: 4305
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: StockFiNN Release and Scaling

Post by lkaufman » Tue Jul 07, 2020 11:48 pm

Gejsi Marku wrote:
Tue Jul 07, 2020 4:35 pm
Just finished the test of 100 games at TC 3+3 between StockfiNN vs Stockfish Dev on 8 cores.
StockfiNN won the match with a W-L-D rate of 14-6-80

Here are some notable victories of StockfiNN



Game 56
SF vs SFNN 0-1




Game 94
SF vs SFNN 0-1

At 10' + 6" on one thread I got a 52 to 48 victory (+14 elo) for SFnn2 vs latest (July 6) SF dev, a bit less than your result which suggests that it does even better with eight threads than with one (subject to margin of error).
Komodo rules!

Gejsi Marku
Posts: 20
Joined: Wed Mar 11, 2020 3:02 pm
Full name: Gejsi Marku

Re: StockFiNN Release and Scaling

Post by Gejsi Marku » Wed Jul 08, 2020 2:33 am

Honestly I thought it would do worse in 8 threads compared to your single core test as I as usual expected SF to scale much better. So I'm really surprised it went the other way around. Might test at higher times next time with double the cores to see if that holds true for even higher cores.

lkaufman
Posts: 4305
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: StockFiNN Release and Scaling

Post by lkaufman » Wed Jul 08, 2020 4:05 am

dkappe wrote:
Tue Jul 07, 2020 10:32 pm
lkaufman wrote:
Tue Jul 07, 2020 6:26 am
Komodo rules!
Can we expect a net trained with Komodo 14 evals and pv? I’d do it myself, but all of my compute resources are busy training Bad Gyal 9.
Well, we have plenty of compute resources and are very interested in this, but we lack expertise with NNs so we are dependent on chess.com personnel with other obligations, so I don't know how fast it will happen. Once we have a net that can work inside Komodo, I think Mark and I will be good at modifying Komodo to take better advantage of such a net. If you would be interested in working with us on this, using our hardware for training, please email or message me.
Komodo rules!

jjoshua2
Posts: 94
Joined: Sat Mar 10, 2018 5:16 am

Re: StockFiNN Release and Scaling

Post by jjoshua2 » Wed Jul 08, 2020 10:24 pm

jjoshua2 wrote:
Sat Jul 04, 2020 10:41 pm

Code: Select all

15s+0.15s
Score of StockFiNN vs BrainFish_200420: 66 - 155 - 99 [0.361]
Elo difference: -99.2 +/- 32.4, LOS: 0.0 %, DrawRatio: 30.9 %

30s+0.3s
Score of StockFiNN vs BrainFish_200420: 72 - 127 - 121 [0.414]
Elo difference: -60.3 +/- 30.2, LOS: 0.0 %, DrawRatio: 37.8 %

60s+0.6s 
Score of StockFiNN vs BrainFish_200420: 74 - 103 - 143 [0.455]
Elo difference: -31.6 +/- 28.4, LOS: 1.5 %, DrawRatio: 44.7 %

120+1.2s 
Score of StockFiNN vs BrainFish_200420: 79 - 75 - 166 [0.506]
Elo difference: 4.3 +/- 26.4, LOS: 62.6 %, DrawRatio: 51.9 %
Same conditions as before but updated to most recent SF-dev.

Code: Select all

Score of StockFiNN2 vs stockfish_20070321_x64_modern: 72 - 109 - 139 [0.442]
Elo difference: -40.4 +/- 28.7, LOS: 0.3 %, DrawRatio: 43.4 %

Same but contempt=0
Score of StockFiNN2 vs stockfish_20070321_x64_modern: 79 - 115 - 126 [0.444]
Elo difference: -39.3 +/- 29.7, LOS: 0.5 %, DrawRatio: 39.4 %

60s+0.6 16 threads
Score of StockFiNN2 vs stockfish_20070321_x64_modern: 63 - 77 - 180 [0.478]
Elo difference: -15.2 +/- 25.2, LOS: 11.8 %, DrawRatio: 56.3 %

8x time. 1 thread
Score of StockFiNN2 vs stockfish_20070321_x64_modern: 73 - 68 - 179 [0.508]
Elo difference: 5.4 +/- 25.3, LOS: 66.3 %, DrawRatio: 55.9 %
In the past 3 months SF has only improved about 10 elo. StockfiNN scales better than SF-dev both in cores and time, but very much more so with time, and a lot of that seems to be from more increment, partly because time management is bad and endgame blunders on low increment. (Preview in progress results with 10min+10s)

Code: Select all

Score of StockFiNN2 vs stockfish_20070321_x64_modern: 63 - 38 - 134 [0.553]
Elo difference: 37.1 +/- 29.1, LOS: 99.4 %, DrawRatio: 57.0 %

lkaufman
Posts: 4305
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: StockFiNN Release and Scaling

Post by lkaufman » Thu Jul 09, 2020 6:53 pm

jjoshua2 wrote:
Wed Jul 08, 2020 10:24 pm
jjoshua2 wrote:
Sat Jul 04, 2020 10:41 pm

Code: Select all

15s+0.15s
Score of StockFiNN vs BrainFish_200420: 66 - 155 - 99 [0.361]
Elo difference: -99.2 +/- 32.4, LOS: 0.0 %, DrawRatio: 30.9 %

30s+0.3s
Score of StockFiNN vs BrainFish_200420: 72 - 127 - 121 [0.414]
Elo difference: -60.3 +/- 30.2, LOS: 0.0 %, DrawRatio: 37.8 %

60s+0.6s 
Score of StockFiNN vs BrainFish_200420: 74 - 103 - 143 [0.455]
Elo difference: -31.6 +/- 28.4, LOS: 1.5 %, DrawRatio: 44.7 %

120+1.2s 
Score of StockFiNN vs BrainFish_200420: 79 - 75 - 166 [0.506]
Elo difference: 4.3 +/- 26.4, LOS: 62.6 %, DrawRatio: 51.9 %
Same conditions as before but updated to most recent SF-dev.

Code: Select all

Score of StockFiNN2 vs stockfish_20070321_x64_modern: 72 - 109 - 139 [0.442]
Elo difference: -40.4 +/- 28.7, LOS: 0.3 %, DrawRatio: 43.4 %

Same but contempt=0
Score of StockFiNN2 vs stockfish_20070321_x64_modern: 79 - 115 - 126 [0.444]
Elo difference: -39.3 +/- 29.7, LOS: 0.5 %, DrawRatio: 39.4 %

60s+0.6 16 threads
Score of StockFiNN2 vs stockfish_20070321_x64_modern: 63 - 77 - 180 [0.478]
Elo difference: -15.2 +/- 25.2, LOS: 11.8 %, DrawRatio: 56.3 %

8x time. 1 thread
Score of StockFiNN2 vs stockfish_20070321_x64_modern: 73 - 68 - 179 [0.508]
Elo difference: 5.4 +/- 25.3, LOS: 66.3 %, DrawRatio: 55.9 %
In the past 3 months SF has only improved about 10 elo. StockfiNN scales better than SF-dev both in cores and time, but very much more so with time, and a lot of that seems to be from more increment, partly because time management is bad and endgame blunders on low increment.
(Preview in progress results with 10min+10s)

Code: Select all

Score of StockFiNN2 vs stockfish_20070321_x64_modern: 63 - 38 - 134 [0.553]
Elo difference: 37.1 +/- 29.1, LOS: 99.4 %, DrawRatio: 57.0 %
I'm running same test, 1 thread 10 min + 10s, SFNN2 vs SF July 6 (3 days newer than your test), on 5 Ghz I7, Hert 250 lowdraw book. So far, I have 24 wins, 9 losses, 95 draws, which is +41 elo. So confirming your result.
Komodo rules!

User avatar
Rebel
Posts: 5378
Joined: Thu Aug 18, 2011 10:04 am

Re: StockFiNN Release and Scaling

Post by Rebel » Fri Jul 10, 2020 10:44 am

Can't share the joy, ran 200 games (40/240 six seconds per move) against SF11 and got 36 - 50 - 114 = 46.5%

Makes me wonder if I am using the correct StockFINN version.

Bench
Nodes searched : 3.857.236
Nodes / Second : 501.330
90% of coding is debugging, the other 10% is writing bugs.

Post Reply