I'm disappointed with Stockfish dev.

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Uri Blass
Posts: 10790
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: I'm disappointed with Stockfish dev.

Post by Uri Blass »

AlexChess wrote: Wed Mar 15, 2023 5:58 pm
Sopel wrote: Wed Mar 15, 2023 5:25 pm
DrEinstein wrote: Wed Mar 15, 2023 12:46 pm
Sopel wrote: Tue Mar 14, 2023 11:37 am
AlexChess wrote: Tue Mar 14, 2023 10:10 am
syzygy wrote: Mon Mar 13, 2023 11:30 pm
CornfedForever wrote: Mon Mar 13, 2023 10:59 pm
syzygy wrote: Mon Mar 13, 2023 9:04 pm [
(Of course the point remains that absolute certainty does not exist. But let's wait for Cornfed to enlighten us on how to make the mathematically impossible possible.)
Dude, nothing is 100% certain. Just stop it with the straw-man.
Hey, you were the wishcraft guy... Did you lose it?
What about to simply re-insert a much improved Contempt option to avoid that a ThreadRipper 128 threads boosted SF hitting 75 MN/s often draws against Raspberry-P3 SF calculating only 79 kN/s? :wink:
sounds like an easy change, I'm sure you can find one of the 30 or so stockfish clone developers competent enough to implement it in their engine
You are one of these 30 developers (FT), and certainly competent enough, so go ahead!
Or do you think I should do it instead and make 'optimism' somehow adjustable in the uci-options?
I'm definitely not competent enough. Sorry.
Image

I've re-added it to ProteusSF since months and it helps to avoid draws, I was suggesting this for Stockfish-dev itself. :) They are the edge of programmers (Sopel included, I can't wait his next NNUEs) and they should use every mhz of top hardwares, adding a feature to force a win even taking some risks. It's the best engine obvioulsly and draw means perfect chess knowledge, but isn't normal that a 10.000$ ThreadRipper doesn't score 95% against a 60$ Raspberry-P3. PS ...Right now I'm studying Berserk 11.1 contempt performance :)
It is clearly normal.
If the engine is strong enough then a 60$ Raspberry-P3 may be unbeatable in chess and score 50% against stronger hardware when you do not use biased book.

I also think that the 60$ Raspberry-P3 may use negative contempt to increase chances to get a draw against the better hardware because I see no reason to play with parameters in order to win with the stronger hardware and not to play with parameters in order to draw with the weaker hardware.
jefk
Posts: 941
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: I'm disappointed with Stockfish dev.

Post by jefk »

yes Uri B, chess -fundamentally- is a draw, that's obvious by now.
But there can be sharp/unbalanced positions, where a deeper
calculations pays off; and then your Raspberry may not hold the draw.

And then i doubt whether a higher contempt for SF is sufficient
to find sharper/more unbalanced positions; there's something in the
SF eval which may be a bit superficial (even with the Nnue nets)
and maybe the search is too narrow (an old problem, where you can
let SF calculate to 40+ply even in unbalanced -but rather equal- positions
and still get 0.00 evals all the time); but it won't be easy to change
(improve) this, SF has been clearly optimized for rating and if you
change on small aspect, the overall performance may suffer.
An example how it can be done in a different way is Rebel16, they
maybe be on to something (clearly different evals) and obviously
Rebel16 isn't just like SF with a high contempt. But then Rebel16
doesn't have such a high rating (yet?) as SF, doesn't matter;
it may well be possible that such efforts (Rebel16 or otherwise eg.
Shashchess(!)) could lead to a better engine than SF, my 2 cnts.
syzygy
Posts: 5693
Joined: Tue Feb 28, 2012 11:56 pm

Re: I'm disappointed with Stockfish dev.

Post by syzygy »

DrEinstein wrote: Wed Mar 15, 2023 10:31 pm
syzygy wrote: Wed Mar 15, 2023 9:25 pm
DrEinstein wrote: Wed Mar 15, 2023 6:47 pm Now I'm completely confused. The SFdevs first deleted contempt, apparently because it doesn't work well (or at all?) with NNUEs, then they introduced optimism:

https://github.com/official-stockfish/S ... d32bb351c3

which seems pretty easy to be tweaked with an uci-parameter, let's say from -50 to 50, and now the two posts from Sopel and Alexchess... :?: :idea:
If it improves play against itself, why make it a parameter?
I don't want to make optimism a parameter, I only was thinking, obviously too loud, about adjusting snicolet's optimism somehow and very carefully by an uci-parameter, in such a way, that SF's playing strength against weaker opponents is improved. I assume that this should be possible.
That is a big assumption for which there is no evidence.
syzygy
Posts: 5693
Joined: Tue Feb 28, 2012 11:56 pm

Re: I'm disappointed with Stockfish dev.

Post by syzygy »

jefk wrote: Thu Mar 16, 2023 11:43 amAn example how it can be done in a different way is Rebel16, they maybe be on to something (clearly different evals) and obviously Rebel16 isn't just like SF with a high contempt. But then Rebel16 doesn't have such a high rating (yet?) as SF, doesn't matter; it may well be possible that such efforts (Rebel16 or otherwise eg. Shashchess(!)) could lead to a better engine than SF, my 2 cnts.
When Rebel16 (or some other engine) does better than ThreadRipper SF in a long match against RaspBerry-Pi SF, then we can talk.
jefk
Posts: 941
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: I'm disappointed with Stockfish dev.

Post by jefk »

Rebel16 (or some other engine) does better than ThreadRipper SF in a long match against RaspBerry-Pi SF, then we can talk.
depends on what you call 'better'...
(hint, i was talking about unbalanced positions, SfNnue -without book-
does a good job in finding solid positions (with a tiny edge for White which
will evaporate) but does not seem to have any clue about
sharpness or how to find unbalanced positions.

Note for human chess, this is extremely important, example the Slav Marshall gambit,
yes Black can draw (if he can memorize eg the entire Chinese database) but
-once this variation is chosen- has to play almost always one correct move,
otherwise he's lost. Yet the eval shows 0.0. A bit silly, i think.
DrEinstein
Posts: 75
Joined: Wed Sep 15, 2021 8:50 pm
Full name: Albert Einstein

Re: I'm disappointed with Stockfish dev.

Post by DrEinstein »

syzygy wrote: Thu Mar 16, 2023 9:09 pm
DrEinstein wrote: Wed Mar 15, 2023 10:31 pm
syzygy wrote: Wed Mar 15, 2023 9:25 pm
DrEinstein wrote: Wed Mar 15, 2023 6:47 pm Now I'm completely confused. The SFdevs first deleted contempt, apparently because it doesn't work well (or at all?) with NNUEs, then they introduced optimism:

https://github.com/official-stockfish/S ... d32bb351c3

which seems pretty easy to be tweaked with an uci-parameter, let's say from -50 to 50, and now the two posts from Sopel and Alexchess... :?: :idea:
If it improves play against itself, why make it a parameter?
I don't want to make optimism a parameter, I only was thinking, obviously too loud, about adjusting snicolet's optimism somehow and very carefully by an uci-parameter, in such a way, that SF's playing strength against weaker opponents is improved. I assume that this should be possible.
That is a big assumption for which there is no evidence.
Big assumption... LOL

Since I've actually already left this thread I've asked ChatGPT for a better example of a really BIG assumption made by Einstein in 1905 or earlier. I've asked in German:

Sure, no problem! Here's your original question in German and my answer, translated into English:

Question: "What was Einstein's assumption regarding the speed of light for his Special Theory of Relativity?"

Answer: "Einstein's assumption for his Special Theory of Relativity was that the speed of light in a vacuum is the same for all observers, regardless of their relative motion. This means that the speed of light, denoted by 'c', is a constant and has the same value of approximately 299,792,458 meters per second for all observers, regardless of their motion. This is known as the postulate of the constancy of the speed of light and is a fundamental principle of the theory of relativity."
syzygy
Posts: 5693
Joined: Tue Feb 28, 2012 11:56 pm

Re: I'm disappointed with Stockfish dev.

Post by syzygy »

jefk wrote: Fri Mar 17, 2023 3:55 pm
Rebel16 (or some other engine) does better than ThreadRipper SF in a long match against RaspBerry-Pi SF, then we can talk.
depends on what you call 'better'...
It is pretty clear what i mean by better if I suggest running the two against equally "weak" opponents.

The original "complaint" was about ThreadRipper SF drawing games against weak RaspBerry-Pi SF.
If Rebel16 indeed does this "better", then Rebel16 would pass the test that I suggest.

Of course I do not expect Rebel16 to do better (which in no way is meant as criticism of Rebel16).
(hint, i was talking about unbalanced positions, SfNnue -without book-
does a good job in finding solid positions (with a tiny edge for White which
will evaporate) but does not seem to have any clue about
sharpness or how to find unbalanced positions.

Note for human chess, this is extremely important, example the Slav Marshall gambit,
yes Black can draw (if he can memorize eg the entire Chinese database) but
-once this variation is chosen- has to play almost always one correct move,
otherwise he's lost. Yet the eval shows 0.0. A bit silly, i think.
OK, then let's SF ThreadRipper play a long match against a human, let Rebel16 play a long match against the same human under the same conditions, and see which engine "does it better".
User avatar
yurikvelo
Posts: 710
Joined: Sat Dec 06, 2014 1:53 pm

Re: I'm disappointed with Stockfish dev.

Post by yurikvelo »

AlexChess wrote: Wed Mar 15, 2023 5:58 pm 10.000$ ThreadRipper doesn't score 95% against a 60$ Raspberry-P3
how to achieve 95% if you mandate playing from startpos?
it is either 1:0 or 0.5-0.5?

I want to re-test your claim using Time handicap on my CPU, which time control should I use?
User avatar
AlexChess
Posts: 1562
Joined: Sat Feb 06, 2021 8:06 am
Full name: Alex Morales

Re: I'm disappointed with Stockfish dev.

Post by AlexChess »

yurikvelo wrote: Fri Mar 17, 2023 8:49 pm
AlexChess wrote: Wed Mar 15, 2023 5:58 pm 10.000$ ThreadRipper doesn't score 95% against a 60$ Raspberry-P3
how to achieve 95% if you mandate playing from startpos?
it is either 1:0 or 0.5-0.5?

I want to re-test your claim using Time handicap on my CPU, which time control should I use?
5 0 like on Playchess. Raspberry P-3 79 kN/s running Stockfish 15.1 has 2900 ELO, top engines boosted by Cluster AMD up to 1536 threads (!) , ThreadRipper 3990wx 128 threads 75 MN/s, Xeon, Ryzen 9 and Core i9 CPUs are slighy above 3000 ELO... Not normal.
Chess engines and dedicated chess computers fan since 1981 :D macOS Sequoia 16GB-512GB, Windows 11 & Ubuntu ARM64.
ProteusSF Dev Forum
Uri Blass
Posts: 10790
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: I'm disappointed with Stockfish dev.

Post by Uri Blass »

yurikvelo wrote: Fri Mar 17, 2023 8:49 pm
AlexChess wrote: Wed Mar 15, 2023 5:58 pm 10.000$ ThreadRipper doesn't score 95% against a 60$ Raspberry-P3
how to achieve 95% if you mandate playing from startpos?
it is either 1:0 or 0.5-0.5?

I want to re-test your claim using Time handicap on my CPU, which time control should I use?
Stockfish with many cores is not deterministic so in theory 95% is possible from the opening position even if the sides use no book.