I'm disappointed with Stockfish dev.

[Ajedrecista]

Hello again:

No I was speaking about:

Author: Michael Chaly
Date: Fri Feb 24 20:14:44 2023 +0100
Timestamp: 1677266084

Search tuning at very long time control

This patch is a result of tuning session of approximately 100k games at 120+1.2.
Biggest changes are in extensions, stat bonus and depth reduction for nodes without a tt move.

Failed STC:
https://tests.stockfishchess.org/tests/ ... 625bcd7938
LLR: -2.94 (-2.94,2.94) <0.00,2.00>
Total: 13872 W: 3535 L: 3769 D: 6568 Elo -5.86
Ptnml(0-2): 56, 1621, 3800, 1419, 40

Close to neutral at LTC:
https://tests.stockfishchess.org/tests/ ... 625bcd7b8a
Elo: 0.80 +-1.2 (95%) LOS: 90.0%
Total: 60000 W: 16213 L: 16074 D: 27713 Elo +0.80
Ptnml(0-2): 24, 5718, 18379, 5853, 26
nElo: 1.82 +-2.8 (95%) PairsRatio: 1.02

Passed 180+1.8 VLTC:
https://tests.stockfishchess.org/tests/ ... 625bcdb33e
LLR: 2.95 (-2.94,2.94) <0.50,2.50>
Total: 15864 W: 4449 L: 4202 D: 7213 Elo +5.41
Ptnml(0-2): 1, 1301, 5083, 1544, 3

Passed 60+0.6 8 threads SMP VLTC:
https://tests.stockfishchess.org/tests/ ... 625bcdbdb3
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 6288 W: 1821 L: 1604 D: 2863 Elo +11.99
Ptnml(0-2): 0, 402, 2123, 619, 0

closes https://github.com/official-stockfish/S ... /pull/4406

Good point, I missed that one. Anyway, improving a super optimized engine must be a nightmare and very, very difficult unless a major step is found, as NNUE in recent times. If not, please be ready for continuous, yet snail-speed-going-uphill gains.



Regards from Spain.

Ajedrecista.

[RubiChess]

Yesterday 24.2. there was a stunning patch Elo +11.99! But RT test was about -1 ELO . Slow progress nowadays.

You have even copied the commit message of this patch here
but you obviously failed to understand it.

[syzygy]

To call a new net 'just another patch' because the engine and net work together is apparently your way of looking at it and if so I can't argue with that 'Borg' concept in the semantics. To me a 'patch' is sometihng you see tried in the development versions - like a LMR capture bonus 'change', or decreasing reduction for 'killer moves' or lowering the SEE pruning in qsearch....engine specific. While a new net is...well, a new net for it to use along with those actual changes to the engine like I just mentioned.

Objective testing is not affected by unfounded psychological biases.

[Uri Blass]

Yesterday 24.2. there was a stunning patch Elo +11.99! But RT test was about -1 ELO . Slow progress nowadays.

There is no 11.99 elo improvement in a single patch for the following reasons.

1)If you want to get a good estimate for the elo improvement then you need fixed number of games and not SPRT test.
2)You measure elo improvement by a normal book and not by a biased book.

It is possible that the same patch give 1 elo improvement with a biased book but lose 1 elo with a biased book.

At time control of 60+0.6 These are the estimates:
Latest version is 32.42 elo better than stockfish15 with a biased book and 11.69 elo better with a normal book
Earlier version was 31.91 elo better than stockfish15 with a biased book and 12.58 elo better with a normal book.

Not enough games to know but It is possible that at time control of 60+0.6 the patches for stockfish were productive with a biased book and counterproductive with a non-biased book.

The stockfish team usually optimize for a biased book so it is logical that some patches that improve stockfish when the evaluation is near +1 but are counterproductive when the evaluation is near draw may pass.

[Magnum]

I'm disappointed with Stockfish dev!

Donate Frontier.
https://www.top500.org/
It should speed up the Stockfish development a little bit.
Probably 500 elo after 1 day.

[Eduard]

Yesterday 24.2. there was a stunning patch Elo +11.99! But RT test was about -1 ELO . Slow progress nowadays.

There is no 11.99 elo improvement in a single patch for the following reasons.

1)If you want to get a good estimate for the elo improvement then you need fixed number of games and not SPRT test.
2)You measure elo improvement by a normal book and not by a biased book.

It is possible that the same patch give 1 elo improvement with a biased book but lose 1 elo with a biased book.

At time control of 60+0.6 These are the estimates:
Latest version is 32.42 elo better than stockfish15 with a biased book and 11.69 elo better with a normal book
Earlier version was 31.91 elo better than stockfish15 with a biased book and 12.58 elo better with a normal book.

Not enough games to know but It is possible that at time control of 60+0.6 the patches for stockfish were productive with a biased book and counterproductive with a non-biased book.

The stockfish team usually optimize for a biased book so it is logical that some patches that improve stockfish when the evaluation is near +1 but are counterproductive when the evaluation is near draw may pass.

A total of 73 parameters were changed here. Known parameters that are constantly changing. Let's see when one of these parameters will be changed again? It won't take too long.

As written here, the tests were carried out with a separate short book, and everything in super bullet mode. What should I do if I play at least 5min games (plus ponder ON) and play against strong other players who play with extremely strong tournament books? The fact is that Stockfish is being made faster and faster in the search. It's about getting to such depths of search as quickly as possible, which of course gives you an advantage in the bullet. However, such engines overlook a lot that does not play a role in the bullet. For me these changes are completely useless.

[Jouni]

Updated original NCM page has no progress in 2023.

[CornfedForever]

A total of 73 parameters were changed here. Known parameters that are constantly changing. Let's see when one of these parameters will be changed again? It won't take too long.

And they wonder why I question how they can know which changes actually resulted in a positive change and which result in a negative change.

[Eduard]

I wish the Stockfish team a lot of fun tuning. I myself have a lot of fun with Stockfish. Since October 2022 I have looked at every patch, documented it - graphically and as text. I could see everything with one click and think about what could be useful for my purpose (to build a nice analysis engine). Most were worthless. If you exceed the limit of Bullet, then there is nothing left that should lead to an improvement. The opposite was even the case. There were only a few patches that were really interesting in the last few months. I analyze a lot, study openings, etc...

I have decided not to implement any more Stockfish patches as long as they are only based on parameter changes. My engine is now as successful as I had imagined. Perfect for analysis. If there are any meaningful changes, I'll be happy to test them. Until then, I'd rather drink tea and enjoy server games and nice analyses. Yesterday on PlayChess a friend played with my engine at the tournament. Unfortunately he only has a slow PC with about 2700 kns. My laptop in energy safe mode and at only 6 watts (usually 30 watts) is just as fast. The engine calculates very slowly. This can't go well, I thought to myself. To my surprise I saw today that he went 17 rounds undefeated and even tied on points with the tournament champion. I couldn't believe it, I'm playing with 20000 kns myself, and that's not very much these days either.

If you increase the game level to at least 5 min, Stockfish dev is no longer the best choice. Corchess alone is significantly better. Some parts of my engine are based on Corchess (Some codes are even from Corchess October 2022, I haven't changed them).

[CornfedForever]

If you increase the game level to at least 5 min, Stockfish dev is no longer the best choice. Corchess alone is significantly better. Some parts of my engine are based on Corchess (Some codes are even from Corchess October 2022, I haven't changed them).

Is it Corcess or Charisma which you use for analysis (not playing so much)?