1 core can be better than 6 cores (Stockfish)

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Jouni
Posts: 3546
Joined: Wed Mar 08, 2006 8:15 pm
Full name: Jouni Uski

1 core can be better than 6 cores (Stockfish)

Post by Jouni »

I ran Arasan testsuite 2024 with Stockfish dev 12.1. With 15 sec limit 6 cores got 174 correct (2 tries same result). Funnily 1 cores got 175!
Jouni
Hai
Posts: 645
Joined: Sun Aug 04, 2013 1:19 pm

Re: 1 core can be better than 6 cores (Stockfish)

Post by Hai »

That's your fault because you are using the Arasan testsuite 2024.

Try the same test with this testsuite: https://www.mediafire.com/file/cypaz2t0 ... 2.pgn/file
Ciekce
Posts: 174
Joined: Sun Oct 30, 2022 5:26 pm
Full name: Conor Anstey

Re: 1 core can be better than 6 cores (Stockfish)

Post by Ciekce »

That's your fault because you are using a testsuite.
User avatar
Ras
Posts: 2680
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: 1 core can be better than 6 cores (Stockfish)

Post by Ras »

So there's one test position where the six cores don't get the expected result. Could be interesting to have a look at that. It's even possible that this position is actually ambiguous.
Rasmus Althoff
https://www.ct800.net
User avatar
Ajedrecista
Posts: 2058
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: 1 core can be better than 6 cores (Stockfish).

Post by Ajedrecista »

Hello Rasmus:
Ras wrote: Sat Jan 18, 2025 11:32 am So there's one test position where the six cores don't get the expected result. Could be interesting to have a look at that. It's even possible that this position is actually ambiguous.
Most likely IMHO, but not necessarily since the used test has got more than 175 positions (200 in fact). I mean, it could be that 1-core and 6-core solved the same s positions (149 =< s =< 174, but not necessarily s = 174 only), this is why I would say at least —and more likely— one position. If engine 1 solves s1 positions and engine 2 solves s2 positions in a test suite with n total positions (0 =< s1 =< n and 0 =< s2 =< n), I think that the number of common solved positions s must be max(0, s1 + s2 - n) =< s =< min(s1s2). Hence, max(0, 175 + 174 - 200) =< s =< min(175, 174); 149 =< s =< 174. Any corrections are welcome.

Otherwise, I agree with you that it would be interesting to have a look into the discrepant position/s and its/their ambiguity or soundness, all of this without trying to discredit Jon's continous effort of collecting, testing and choosing positions for his longly reviewed test suite.

Regards from Spain.

Ajedrecista.
Jouni
Posts: 3546
Joined: Wed Mar 08, 2006 8:15 pm
Full name: Jouni Uski

Re: 1 core can be better than 6 cores (Stockfish)

Post by Jouni »

In third try 6 cores solved 171. In second try 1 cores solved 177,when 2 positions were solved in just 15 seconds! OK this shows test suite is buggy and/or too easy.
Jouni
Hai
Posts: 645
Joined: Sun Aug 04, 2013 1:19 pm

Re: 1 core can be better than 6 cores (Stockfish)

Post by Hai »

Jouni wrote: Sat Jan 18, 2025 4:07 pm In third try 6 cores solved 171. In second try 1 cores solved 177,when 2 positions were solved in just 15 seconds! OK this shows test suite is buggy and/or too easy.
Hai wrote: Fri Jan 17, 2025 11:13 am That's your fault because you are using the Arasan testsuite 2024.

Try the same test with this testsuite: https://www.mediafire.com/file/cypaz2t0 ... 2.pgn/file
Werewolf
Posts: 1965
Joined: Thu Sep 18, 2008 10:24 pm

Re: 1 core can be better than 6 cores (Stockfish)

Post by Werewolf »

Jouni wrote: Fri Jan 17, 2025 9:18 am I ran Arasan testsuite 2024 with Stockfish dev 12.1. With 15 sec limit 6 cores got 174 correct (2 tries same result). Funnily 1 cores got 175!
The more I've been thinking about this, the more I think this shouldn't happen. Even accepting there is virtually no link between Elo and tactical testsuite performance, you should see a better score on more cores.

Is there a pgn of Arasan to download?
Paloma
Posts: 1193
Joined: Thu Dec 25, 2008 9:07 pm
Full name: Herbert L

Re: 1 core can be better than 6 cores (Stockfish)

Post by Paloma »

Not a pgn, but an epd here :https://arasanchess.org/testsuite.shtml

But you can create a new cbh database, and then chose
in Fritz GUI attach epd
Werewolf
Posts: 1965
Joined: Thu Sep 18, 2008 10:24 pm

Re: 1 core can be better than 6 cores (Stockfish)

Post by Werewolf »

Paloma wrote: Wed Jan 29, 2025 12:15 am Not a pgn, but an epd here :https://arasanchess.org/testsuite.shtml

But you can create a new cbh database, and then chose
in Fritz GUI attach epd
Thanks - but how do you actually do this? I have no import option or attach option. I'm wondering if this was in an earlier version of Fritz/Chessbase. I may well be being blind!