New Stockfish 15,1 evaluations

[Rowen]

Any thoughts on the new evaluations? Why the change? is it due to Lc0 data being used in the Nets? On a practical level what do the evaluations represent, a +1 evaluation indicates a 50% chance of winning but what about other evaluations? Is this a step forward?

[CornfedForever]

Any thoughts on the new evaluations? Why the change? is it due to Lc0 data being used in the Nets? On a practical level what do the evaluations represent, a +1 evaluation indicates a 50% chance of winning but what about other evaluations? Is this a step forward?

Better for analysis...in my opinion.

[Uri Blass]

Any thoughts on the new evaluations? Why the change? is it due to Lc0 data being used in the Nets? On a practical level what do the evaluations represent, a +1 evaluation indicates a 50% chance of winning but what about other evaluations? Is this a step forward?

The only way to know what the evaluation represent is testing.
The evaluation may mean something different at different time control.

If somebody play a lot of games of stockfish against itself from random chess positions from chess games at 1 second per move then we can calculate based on statistics what +1 evaluation means at 1 second per move with some hardware) and what +1.3 means at 1 second per move with the same hardware).

Not sure if it means the same at 10 seconds per move and it may be interesting to know the difference but somebody need to devote a lot of computer time for it if we want to know.

I do not know based on what time control and what hardware +1 means expected result of 0.75(I guess it is not exactly about probability to win because it may be something like 50.1% for a win 49.8% for a draw and 0.1% for a loss).

[Plutie]

...
I do not know based on what time control and what hardware +1 means expected result of 0.75(I guess it is not exactly about probability to win because it may be something like 50.1% for a win 49.8% for a draw and 0.1% for a loss).

+1.00 is equal to a 50% win chance at move 32, fitted to fishtest LTC data (60s+0.6s @ 1.328m nps)

[CornfedForever]

...
I do not know based on what time control and what hardware +1 means expected result of 0.75(I guess it is not exactly about probability to win because it may be something like 50.1% for a win 49.8% for a draw and 0.1% for a loss).

+1.00 is equal to a 50% win chance at move 32, fitted to fishtest LTC data (60s+0.6s @ 1.328m nps)

Right, you beat me to that.
I (may be wrong and apologize if so) think I remember (can't find the post) Larry K saying a similar thing had been done to Dragon. He may well have been referring to something else though.

[CornfedForever]

And it seems they just had an update on that today:

Author: Joost VandeVondele
Date: Thu Feb 2 17:58:05 2023 +0100
Timestamp: 1675357085

Update WLD model

update the WLD model with about 400M positions extracted from recent LTC games after the net updates.
This ensures that the 50% win rate is again at 1.0 eval.

closes https://github.com/official-stockfish/S ... /pull/4373

[dkappe]

I (may be wrong and apologize if so) think I remember (can't find the post) Larry K saying a similar thing had been done to Dragon.

That is correct.

[Milton]

...
I do not know based on what time control and what hardware +1 means expected result of 0.75(I guess it is not exactly about probability to win because it may be something like 50.1% for a win 49.8% for a draw and 0.1% for a loss).

+1.00 is equal to a 50% win chance at move 32, fitted to fishtest LTC data (60s+0.6s @ 1.328m nps)

Right, you beat me to that.
I (may be wrong and apologize if so) think I remember (can't find the post) Larry K saying a similar thing had been done to Dragon. He may well have been referring to something else though.

So if an evaluation of "1" means a 50% chance of a win, would this be equivalent to an evaluation of "0" (i.e. no advantage to either side) under the previous scheme?

[Plutie]

...
I do not know based on what time control and what hardware +1 means expected result of 0.75(I guess it is not exactly about probability to win because it may be something like 50.1% for a win 49.8% for a draw and 0.1% for a loss).

+1.00 is equal to a 50% win chance at move 32, fitted to fishtest LTC data (60s+0.6s @ 1.328m nps)

Right, you beat me to that.
I (may be wrong and apologize if so) think I remember (can't find the post) Larry K saying a similar thing had been done to Dragon. He may well have been referring to something else though.

So if an evaluation of "1" means a 50% chance of a win, would this be equivalent to an evaluation of "0" (i.e. no advantage to either side) under the previous scheme?

no, 0.00 keeps the same meaning of 100% draw chance. the difference is that now, an old eval of, let's say 1.7, is now equal to a current eval of 1.00 (probably not exactly this, I forget the exact scaling right now)

[lkaufman]

...
I do not know based on what time control and what hardware +1 means expected result of 0.75(I guess it is not exactly about probability to win because it may be something like 50.1% for a win 49.8% for a draw and 0.1% for a loss).

+1.00 is equal to a 50% win chance at move 32, fitted to fishtest LTC data (60s+0.6s @ 1.328m nps)

Right, you beat me to that.
I (may be wrong and apologize if so) think I remember (can't find the post) Larry K saying a similar thing had been done to Dragon. He may well have been referring to something else though.

We scaled Dragon 3.2 by a factor of 0.68, which makes opening evals for positions near the win,draw line about the same as sf15.1, I.e. about 1. Didn’t check middlegame evals (like move32) but probably also close to Sf on average for such positions.