Are draws hard to predict?

[jp]

What do Leela etc. do re. W/D/L vs. W/L?

treat as one continuous value, I think, last time I looked. value is not treated as categorical, the trained network output is a value between 0 and 1, representing win probablity.

I've been wanting to check what that means. Does 0.7 mean win prob. 70% and combined draw & loss prob. 30% (not distinguishing between D & L), or does it mean something else (e.g. expected score, Pr(win)+ (1/2).Pr(draw))?

[chrisw]

What do Leela etc. do re. W/D/L vs. W/L?

treat as one continuous value, I think, last time I looked. value is not treated as categorical, the trained network output is a value between 0 and 1, representing win probablity.

I've been wanting to check what that means. Does 0.7 mean win prob. 70% and combined draw & loss prob. 30% (not distinguishing between D & L), or does it mean something else (e.g. expected score, Pr(win)+ (1/2).Pr(draw))?

it means the same as a typical AB program score, they are convertible between centipawns and win chances via formula. okay, so you ask what is “win chance”, good question, nobody programming cares much, the idea is just to maximise the magic number.

for AB the number comes from adding up some chessic features. The fact, for example, that you will win 100% of your games against any entity, however strong, if you are +5 for KRK or +9 for KQK, tells you that this AB score is not really a “win chance”.
Likewise NN, might say 90% or something, when you know its 100% against any entity.

The NN win prob, is actually the result of hitting the network during training with either a 1 or a 0 or a -1, in similar positions, say 10,000 times, when it got hit maybe 6000 times with 1, 3000 times with 0 and 1000 times with -1. Okay I cheat a little, but this going to give 0.9 or whatever. And this means? errrrm, doesn’t matter, what counts is that this 0.9 is greater than 0.87 someplace else, so the algorithm “knows what to do”.

Ultimately, the problem rests with those endusers, encouraged by those who really ought to know better, who spread the narrative that these scores are exact and accurate and correct and meaningful and all you need is mega silicon bucks plus Stockfish or whatever. Then you have god and can prove anything.

So to answer your question. The value is a magic number whose task is to provide direction by comparing it with other magic numbers. Avoid trying to treat it as anything absolute, cos it isn’t.

[jp]

it means the same as a typical AB program score, they are convertible between centipawns and win chances via formula. okay, so you ask what is “win chance”, good question, nobody programming cares much, the idea is just to maximise the magic number.

for AB the number comes from adding up some chessic features.

Ultimately, the problem rests with those endusers, encouraged by those who really ought to know better, who spread the narrative that these scores are exact and accurate and correct and meaningful and all you need is mega silicon bucks plus Stockfish or whatever. Then you have god and can prove anything.

So to answer your question. The value is a magic number whose task is to provide direction by comparing it with other magic numbers. Avoid trying to treat it as anything absolute, cos it isn’t.

Yes. On converting between cP & win chance: this is a question re. Lc0 evaluation. In cP, Lc0 always overestimates. Some people claim this is just because of the conversion formula used. It's another thing I wanted to look into. I don't know what formula is used or if it's "standard". Those people would then claim Lc0's eval. is really okay, even if it looks crazy.

[chrisw]

it means the same as a typical AB program score, they are convertible between centipawns and win chances via formula. okay, so you ask what is “win chance”, good question, nobody programming cares much, the idea is just to maximise the magic number.

for AB the number comes from adding up some chessic features.

Ultimately, the problem rests with those endusers, encouraged by those who really ought to know better, who spread the narrative that these scores are exact and accurate and correct and meaningful and all you need is mega silicon bucks plus Stockfish or whatever. Then you have god and can prove anything.

So to answer your question. The value is a magic number whose task is to provide direction by comparing it with other magic numbers. Avoid trying to treat it as anything absolute, cos it isn’t.

Yes. On converting between cP & win chance: this is a question re. Lc0 evaluation. In cP, Lc0 always overestimates. Some people claim this is just because of the conversion formula used. It's another thing I wanted to look into. I don't know what formula is used or if it's "standard". Those people would then claim Lc0's eval. is really okay, even if it looks crazy.

the conversion is empirical, I don’t remember who first did the statistic equivalences, it may have been Vas. It’s unlikely any rigorous connection between the two has ever been established. LC0 evaluation is optimistic usually, because, er, dunno. Not really thought about it.

[cdani]

...LC0 evaluation is optimistic usually, because, er, dunno. Not really thought about it.

I guess it's the result of that the positions it reaches tend to work against himself in selfplay.

[jp]

...LC0 evaluation is optimistic usually, because, er, dunno. Not really thought about it.

I guess it's the result of that the positions it reaches tend to work against himself in selfplay.

Another thing is Kai found the Lc0 eval is asymmetric:
http://talkchess.com/forum3/viewtopic.php?f=2&t=68072

Now looking at that thread again I see Alexander replied.

Interesting! Our Q to centipawns formula is symmetric (namely, 290.680623072 * tan(1.548090806 * Q)), where Q is between -1 and 1).
The reason for that bias is probably Lc0 being overly optimistic, due to blunders, not knowing how to win won endgames etc.

[chrisw]

...LC0 evaluation is optimistic usually, because, er, dunno. Not really thought about it.

I guess it's the result of that the positions it reaches tend to work against himself in selfplay.

Another thing is Kai found the Lc0 eval is asymmetric:
http://talkchess.com/forum3/viewtopic.php?f=2&t=68072

Now looking at that thread again I see Alexander replied.

Interesting! Our Q to centipawns formula is symmetric (namely, 290.680623072 * tan(1.548090806 * Q)), where Q is between -1 and 1).
The reason for that bias is probably Lc0 being overly optimistic, due to blunders, not knowing how to win won endgames etc.

It would be a thing if the words asymmetric, optimistic and unbalanced were defined and consistently used. Optimistic tends to be used okay, but the others not.

[jp]

True. But his graphs at least show what he means by "asymmetric".

[chrisw]

True. But his graphs at least show what he means by "asymmetric".

Maybe, but it’s not the “traditional” use of the term in comp chess. Where it means the evaluation and/or search are skewed depending on whether the engine is on move or not. eg, the evaulation and/or search are different depending on odd or even ply. I didn’t look for a while, but I don’t think LC0 has any odd/even ply differences.

Laskos graphs show either optimism or else an inability to convert a higher earlier score into a later win. Or both.