WDL contempt of Lc0 is inversely proportional to (drawelo)^2.

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
Ajedrecista
Posts: 1985
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

WDL contempt of Lc0 is inversely proportional to (drawelo)^2.

Post by Ajedrecista »

Hello:

Disclaimer: I open this thread just as a curiosity of how simple some things can be. If I am wrong in my next reasoning, please answer.

------------

I reached by chance a Lichess blog about sharpness of positions:

Evaluating Sharpness using LC0's WDL

And I saw the WDL contempt of Lc0 according to that article:

Code: Select all

contempt = {2/[ln(1/W - 1) + ln(1/L - 1)]}^2
Where W and L are win and lose probabilities. I rewrote 'log' as 'ln' in view of the sharpness values given in the graphs.

Then, I remembered the early days of Fishtest around 2013, where SPRT bounds were expressed in Bayeselo units instead of Elo. IIRC:

Code: Select all

bayeselo = 200*log10[(W/L)*(1 - W)/(1 - L)]
 drawelo = 200*log10{[(1 - W)/W]*[(1 - L)/L]}
There is a quite evident relation to the trained eye in the logarithm thing between the contempt and drawelo:

Code: Select all

drawelo/200 = log10{[(1 - W)/W]*[(1 - L)/L]}
drawelo/200 = log10[(1 - W)/W] + log10[(1 - L)/L]
drawelo/200 = log10(1/W - 1) + log10(1/L - 1)
drawelo/200 = [ln(1/W - 1) + ln(1/L - 1)]/ln(10)
drawelo*{[ln(10)]/200} = [ln(1/W - 1) + ln(1/L - 1)]/ln(10)
Voilà! The relation could not be simpler:

Code: Select all

contempt = (2/{drawelo*[ln(10)]/200})^2 = {[400/ln(10)]/drawelo}^2 = (C/drawelo)^2
The WDL contempt of Lc0 is inversely proportional to (drawelo)^2, which makes sense: more drawish, less sharpness; more double-edged, more sharpness.

Curiously, the constant C = 400/ln(10) is not that strange. While working with error bars in the past, I came across 1600/ln(10) several times (at least here, here, here, here and here).

------------

Please note that contempt and drawelo values can be unbound, for example with extreme values of W and L such as W = L = 0 (drawelo → infinity; contempt → 0) in KB vs K and KN vs K endgames; or W = 1 and L = 0 (or W = 0 and L = 1) (drawelo → 0; contempt → infinity), for example in a position where the only possible move/s lead to checkmate, like in the following position, unlikely to be seen in a real game:

[d]6qk/8/5r1K/7P/7Q/8/8/8 w - - 0 1

Qxf6+ (only), Qg7+ (only); Qxg7# (only)

An even funnier position is this one: beware with which queen you capture... win or lose!

[d]3Q2qk/8/4qq1K/7P/7Q/8/8/8 w - - 0 1

a) White wins (perfect play):
Qhxf6+, Qxf6+; Qxf6+, Qg7+; Qxg7#

b) Black wins (white blunders):
Qdxf6+??, Qg7#

c) White wins (back-to-back blunders):
Qdxf6+??, Qxf6+??; Qxf6+, Qg7+; Qxg7#

Regards from Spain.

Ajedrecista.
BeyondCritics
Posts: 406
Joined: Sat May 05, 2012 2:48 pm
Full name: Oliver Roese

Re: WDL contempt of Lc0 is inversely proportional to (drawelo)^2.

Post by BeyondCritics »

Interesting, but i don't really understand where the expressions for drawelo and bayeselo come from:

Code: Select all

bayeselo = 200*log10[(W/L)*(1 - W)/(1 - L)]
 drawelo = 200*log10{[(1 - W)/W]*[(1 - L)/L]}
Can you point out briefly or refer to some source?
User avatar
Ajedrecista
Posts: 1985
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: WDL contempt of Lc0 is inversely proportional to (drawelo)^2.

Post by Ajedrecista »

Hello Oliver:
BeyondCritics wrote: Sat Apr 27, 2024 7:56 am Interesting, but i don't really understand where the expressions for drawelo and bayeselo come from:

Code: Select all

bayeselo = 200*log10[(W/L)*(1 - W)/(1 - L)]
 drawelo = 200*log10{[(1 - W)/W]*[(1 - L)/L]}
Can you point out briefly or refer to some source?
Spoiler: the answer is at the end of my post. I am going to describe the process of how I found a source for your question. I hope you find it useful:

My first findings:

I read it long time ago, around 2013 or so, when SF Testing Network started. It was in fishcooking Google Group, but its content is blocked now.

I had to search in my old files because I ported to Fortran 95 a SPRT simulator in Python by Michel Van den Bergh. I found there a bayeselo to Elo factor scale that I still had not forgotten, which was:

Code: Select all

x = 10^(-drawelo/400)
Elo = bayeselo*{4*x/[(1+x)^2]}
I also found a thing that I did not remember, a comment saying that drawelo was estimated from a sample of games (number of wins, draws and loses):

Code: Select all

drawelo = 200*log10[(1/L - 1)*(1/W - 1)]
Which can be rewritten in the form of the original post, which is what I actually remembered, because my quotes of bayeselo and drawelo were from memory. They were correct... match point saved!

I usually copied URLs as comments in the source codes of my little tools, but it was not the case in this exact simulator. Fortunately, I wrote other SPRT simulator, a 2-step simulator combining STC + LTC (short and long time controls). There are two URLs as comments in the first and second lines of the source code!

http://hardy.uhasselt.be/Toga/sprta.py (This archived copy works).
http://chess-sprt-calc.azurewebsites.net/ (This archived copy works).

You can find the bayeselo to Elo factor scale at the beginning of the first link.

The answer:

Last by not least, I found an old post at TalkChess that pointed to a source code of cutechess. You will find the desired formulas between lines 68 and 71 of the link below:

https://github.com/cutechess/cutechess/ ... pp#L68-L71

I picked them long ago from fishcooking Google Group, as I said before. I do not remember who posted them. I had to look for alternate sources since fishcooking is blocked.

Edit: I found other calculator in my old files, with an URL that slightly changed over time (this one, not working anymore). The current one, with the desired formulas highlighted, is:

https://github.com/official-stockfish/f ... py#L86-L87

This is from the official SF GitHub.

If you want a detailed mathematical proof: sorry, I can not provide it.

Regards from Spain.

Ajedrecista.
BeyondCritics
Posts: 406
Joined: Sat May 05, 2012 2:48 pm
Full name: Oliver Roese

Re: WDL contempt of Lc0 is inversely proportional to (drawelo)^2.

Post by BeyondCritics »

Many thanks for the useful references.
In the end they led me to the website of the bayeselo tool from Remi Coulom [1]. In the section "Bayesian approach" we have the definition of a probability function for game outcome in terms of the parameters eloBlack, eloWhite, eloAdvantage and eloDraw. Matching his parameter names with theirs we can get:

Code: Select all

bayesElo := eloWhite - eloBlack + eloAdvantage
drawElo := eloDraw
With the shorter names

Code: Select all

b := bayesElo
e := drawElo
f(x) := 1 / (1  + 10 ^ (-x / 400))
we get the probability function:

Code: Select all

L(b,e) = f(-b - e)
W(b,e) = f( b - e)
D(b,e) = 1 - W - L
The idea is to solve for e (and b in the process). First invert f:

Code: Select all

10^(-x/400) = (1-f) / f
x = -400 * log10((1-f)/f)
Applying this inverse gives the two equations:

Code: Select all

-b -e = -400 *  log10((1-L)/L)
 b -e = -400 *  log10((1-W)/W)
Adding and subtracting gives

Code: Select all

e = 200*log10((1-L)/L) +  log10((1-W)/W)) =  200*(log10((1-L)*(1-W)/(L*W)) =  200*(log10(1+D/(W*L))
b = 200*( log10((1-L)/L) -  log10((1-W)/W)) =  200*(log10((1-L)*W/((1-W)*L))
and matches the equations in [2].
Hence the mystery from [3] is solved, they inverted the model [1] for the parameter eloDraw and finally transformed it with the monotone function

Code: Select all

m(x) = (400/eloDraw)^2
But this latter step gives no further information and just makes the expression more complicated! Why not simplify instead with the strict monotone transforms:

Code: Select all

 200*(log10(1+D/(W*L)) -> log10(1+D/(W*L) ->  1+D/(W*L) ->  D/(W*L) -> (W*L)/D  
Hence the simple expression (W*L)/D gives the exact same ranking of chess positions as in [2].

[1] https://www.remi-coulom.fr/Bayesian-Elo/#theory
[2] https://github.com/cutechess/cutechess/ ... pp#L68-L71
[3] https://lichess.org/@/jk_182/blog/evalu ... l/EXZ3pRoy