SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jjoshua2
Posts: 94
Joined: Sat Mar 10, 2018 5:16 am

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by jjoshua2 » Sun Jul 12, 2020 7:43 pm

It looks like SPCC uses a skylake CPU so that build should be good. Sadly I do not have an intel machine to test and compile with, but it should be just a little bit faster than the AVX2 builds I would think. It looks like SPCC also uses Hertz book which I did some tests with and found it was similar to the other book I was testing and not the TCEC S18 book. I believe Lkaufman posted all results with same book, so it would seem only reason for difference left is the binary or speed? (Since Larry tested K14 just now and it showed some performance as our prior results with SF-dev.)
I think the sample sizes are too big now for it to be pure bad/good luck. The project is still young so maybe some other issue exists...

lkaufman
Posts: 4289
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by lkaufman » Sun Jul 12, 2020 8:01 pm

jjoshua2 wrote:
Sun Jul 12, 2020 7:43 pm
It looks like SPCC uses a skylake CPU so that build should be good. Sadly I do not have an intel machine to test and compile with, but it should be just a little bit faster than the AVX2 builds I would think. It looks like SPCC also uses Hertz book which I did some tests with and found it was similar to the other book I was testing and not the TCEC S18 book. I believe Lkaufman posted all results with same book, so it would seem only reason for difference left is the binary or speed? (Since Larry tested K14 just now and it showed some performance as our prior results with SF-dev.)
I think the sample sizes are too big now for it to be pure bad/good luck. The project is still young so maybe some other issue exists...

After 162 games vs. SF 11 same conditions your net is still +35 elo, so all my results are pretty self-consistent in assessing the elo as slightly higher than latest SF dev on one thread when running at 60% nps of SF11.
Komodo rules!

pohl4711
Posts: 1302
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by pohl4711 » Mon Jul 13, 2020 10:12 am

jjoshua2 wrote:
Sun Jul 12, 2020 7:43 pm
It looks like SPCC uses a skylake CPU so that build should be good.
Of course. I measured the binary-speed and that build was the fastest on my machine. So, I used it for the testrun.

pohl4711
Posts: 1302
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by pohl4711 » Mon Jul 13, 2020 10:14 am

jjoshua2 wrote:
Sun Jul 12, 2020 7:43 pm
It looks like SPCC also uses Hertz book which I did some tests with
1) it is called HERT not Hertz (Human and Engine Relevant Theory)
2) Of course, I use it. Thomas Zipproth developed the 500 positions set for me and my testings and the 250 lowdraw set was filtered by me.
3) It is not a book, it is only a 500 (and 250 low draw) positions openings set (pgn). Way too small for making a book out of it.

Raphexon
Posts: 270
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by Raphexon » Wed Jul 15, 2020 6:24 pm

lkaufman wrote:
Sun Jul 12, 2020 8:01 pm
jjoshua2 wrote:
Sun Jul 12, 2020 7:43 pm
It looks like SPCC uses a skylake CPU so that build should be good. Sadly I do not have an intel machine to test and compile with, but it should be just a little bit faster than the AVX2 builds I would think. It looks like SPCC also uses Hertz book which I did some tests with and found it was similar to the other book I was testing and not the TCEC S18 book. I believe Lkaufman posted all results with same book, so it would seem only reason for difference left is the binary or speed? (Since Larry tested K14 just now and it showed some performance as our prior results with SF-dev.)
I think the sample sizes are too big now for it to be pure bad/good luck. The project is still young so maybe some other issue exists...

After 162 games vs. SF 11 same conditions your net is still +35 elo, so all my results are pretty self-consistent in assessing the elo as slightly higher than latest SF dev on one thread when running at 60% nps of SF11.
It seems like both Jjoshua's and Nodchip's binaries were bugged in the past.
Recent binaries no longer need non-default slowmover and are just mind-blowingly strong.
They just work without needing to adjust slowmover and they work well.

https://workupload.com/file/ggEUrvNVgmH

or if you're more trusting of Github and net:
https://github.com/nodchip/Stockfish/re ... 2020-07-15
https://drive.google.com/file/d/1EI9Kbt ... sp=sharing

Raphexon
Posts: 270
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by Raphexon » Wed Jul 15, 2020 6:29 pm

pohl4711 wrote:
Mon Jul 13, 2020 10:12 am
jjoshua2 wrote:
Sun Jul 12, 2020 7:43 pm
It looks like SPCC uses a skylake CPU so that build should be good.
Of course. I measured the binary-speed and that build was the fastest on my machine. So, I used it for the testrun.
Would you be willing to do a quick test with the binaries from the test above?
The earlier ones were legit bugged, the new ones work perfectly with default slowmover.

Doesn't have to be thorough, just a quick sanity test that everyone is getting these results now.

pohl4711
Posts: 1302
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by pohl4711 » Fri Jul 17, 2020 5:49 am

Raphexon wrote:
Wed Jul 15, 2020 6:29 pm
pohl4711 wrote:
Mon Jul 13, 2020 10:12 am
jjoshua2 wrote:
Sun Jul 12, 2020 7:43 pm
It looks like SPCC uses a skylake CPU so that build should be good.
Of course. I measured the binary-speed and that build was the fastest on my machine. So, I used it for the testrun.
Would you be willing to do a quick test with the binaries from the test above?
The earlier ones were legit bugged, the new ones work perfectly with default slowmover.

Doesn't have to be thorough, just a quick sanity test that everyone is getting these results now.
I started a new 5000 games testrun with latest bmi2-binary (200715) from
https://github.com/nodchip/Stockfish/releases
and with the gk200627 net, which is the strongest net so far in my pre-tests.

Will take 5-6 days. After 400 games played, the score is really impressive.
Stay tuned.

mehmet123
Posts: 134
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: SPCC: Testruns of StockFiNN 0.2 and Lc0 J90-40 finished

Post by mehmet123 » Fri Jul 17, 2020 4:42 pm

pohl4711 wrote:
Fri Jul 17, 2020 5:49 am
Raphexon wrote:
Wed Jul 15, 2020 6:29 pm
pohl4711 wrote:
Mon Jul 13, 2020 10:12 am
jjoshua2 wrote:
Sun Jul 12, 2020 7:43 pm
It looks like SPCC uses a skylake CPU so that build should be good.
Of course. I measured the binary-speed and that build was the fastest on my machine. So, I used it for the testrun.
Would you be willing to do a quick test with the binaries from the test above?
The earlier ones were legit bugged, the new ones work perfectly with default slowmover.

Doesn't have to be thorough, just a quick sanity test that everyone is getting these results now.
I started a new 5000 games testrun with latest bmi2-binary (200715) from
https://github.com/nodchip/Stockfish/releases
and with the gk200627 net, which is the strongest net so far in my pre-tests.

Will take 5-6 days. After 400 games played, the score is really impressive.
Stay tuned.
To wait 5-6 days is really hard. I'm looking forward to the test results.
I have ben testing Stockfish NNUE Gekehenker chess engine and my test results are much better than my guess.

Post Reply