Stockfish has included WDL stats in engine output

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Deberger
Posts: 49
Joined: Sat Nov 02, 2019 5:42 pm
Full name: ɹǝƃɹǝqǝᗡ ǝɔnɹꓭ

Stockfish has included WDL stats in engine output

Post by Deberger » Thu Jul 02, 2020 2:30 am

Stockfish has included WDL stats in engine output.

Code: Select all

info depth 1 seldepth 1 multipv 1 score cp 110 wdl 355 620 25 nodes 111 nps 37000 tbhits 0 time 3 pv e2e3
info depth 2 seldepth 2 multipv 1 score cp 122 wdl 392 587 21 nodes 448 nps 149333 tbhits 0 time 3 pv e2e3 b7b6
info depth 3 seldepth 3 multipv 1 score cp 116 wdl 375 602 23 nodes 772 nps 257333 tbhits 0 time 3 pv e2e3 b7b6 g1e2
info depth 4 seldepth 4 multipv 1 score cp 75 wdl 253 707 40 nodes 2080 nps 520000 tbhits 0 time 4 pv e2e3 h7h6 g1e2 e7e6
info depth 5 seldepth 5 multipv 1 score cp 79 wdl 263 699 38 nodes 3586 nps 896500 tbhits 0 time 4 pv d2d4 e7e6 c2c3 b8c6 e2e3
info depth 6 seldepth 6 multipv 1 score cp 74 wdl 250 710 40 nodes 12718 nps 1589750 tbhits 0 time 8 pv d2d4 d7d5 b1c3 b8c6 e2e4
Well I suppose it's mildly interesting, but this should not be the default setting:

Code: Select all

option name UCI_ShowWDL type check default true

nnnnnnnn
Posts: 22
Joined: Mon Nov 18, 2019 1:36 pm
Full name: Mark Thellen

Re: Stockfish has included WDL stats in engine output

Post by nnnnnnnn » Thu Jul 02, 2020 3:48 am

wdl may be just what I need which is to determine the win/loss/draw probability. I had been using heuristics based on score.

(In fact I asked exactly this question last week, http://www.talkchess.com/forum3/viewtop ... =7&t=74243 , how to determine when to stop search based on a cp 0 score, and nobody mentioned wdl, which would have solved the issue.)

When was this added to Stockfish?

User avatar
lucasart
Posts: 3109
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: Stockfish has included WDL stats in engine output

Post by lucasart » Thu Jul 02, 2020 4:25 am

Deberger wrote:
Thu Jul 02, 2020 2:30 am
Stockfish has included WDL stats in engine output.

Code: Select all

info depth 1 seldepth 1 multipv 1 score cp 110 wdl 355 620 25 nodes 111 nps 37000 tbhits 0 time 3 pv e2e3
info depth 2 seldepth 2 multipv 1 score cp 122 wdl 392 587 21 nodes 448 nps 149333 tbhits 0 time 3 pv e2e3 b7b6
info depth 3 seldepth 3 multipv 1 score cp 116 wdl 375 602 23 nodes 772 nps 257333 tbhits 0 time 3 pv e2e3 b7b6 g1e2
info depth 4 seldepth 4 multipv 1 score cp 75 wdl 253 707 40 nodes 2080 nps 520000 tbhits 0 time 4 pv e2e3 h7h6 g1e2 e7e6
info depth 5 seldepth 5 multipv 1 score cp 79 wdl 263 699 38 nodes 3586 nps 896500 tbhits 0 time 4 pv d2d4 e7e6 c2c3 b8c6 e2e3
info depth 6 seldepth 6 multipv 1 score cp 74 wdl 250 710 40 nodes 12718 nps 1589750 tbhits 0 time 8 pv d2d4 d7d5 b1c3 b8c6 e2e4
Well I suppose it's mildly interesting, but this should not be the default setting:

Code: Select all

option name UCI_ShowWDL type check default true
I am appalled by this patch... Really doubt this kind of silly gimmick would have made it when Marco was the maintainer.

An alpha beta search trying to pretend to be a MCTS, for marketing reasons.

I would expect as much from commercial closed source engines, like when Rybka was underreporting its search depth to fool naive users into thinking that it was a quantum leap in quality: we compute much less, but much better. Or course the BS was eventually exposed.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

syzygy
Posts: 4600
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish has included WDL stats in engine output

Post by syzygy » Fri Jul 03, 2020 5:21 pm

nnnnnnnn wrote:
Thu Jul 02, 2020 3:48 am
wdl may be just what I need which is to determine the win/loss/draw probability. I had been using heuristics based on score.

(In fact I asked exactly this question last week, http://www.talkchess.com/forum3/viewtop ... =7&t=74243 , how to determine when to stop search based on a cp 0 score, and nobody mentioned wdl, which would have solved the issue.)

When was this added to Stockfish?
https://github.com/official-stockfish/S ... dfdf670392
"6 days ago"

I'm also surprised by this patch. Apparently it translates the score into win/draw/loss probabilities (also taking into account search depth).

Is UCI_ShowWDL a thing nowadays? Where can I find the updated UCI spec?

I also see little reason to have this enabled by default (if this info is needed at all). It seems it could unnecessarily confuse some GUIs and I'm not convinced this is UCI compliant (perhaps I'm wrong on that). Those that use a GUI that understands this information can just enable it in their GUI.

syzygy
Posts: 4600
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish has included WDL stats in engine output

Post by syzygy » Fri Jul 03, 2020 5:28 pm

syzygy wrote:
Fri Jul 03, 2020 5:21 pm
Is UCI_ShowWDL a thing nowadays?
Apparently it is:
http://www.talkchess.com/forum3/viewtopic.php?t=72140
GCP wrote: Based on the above patch, I suggest:

option name UCI_ShowWDL type check default false

If this is toggled to enabled, the engine will include wdl info in the info updates. Making it optional prevents interfaces from breaking, even though UCI does say the interface should ignore stuff it doesn't understand. But depending on the option seems safer as UCI is widely deployed and currmove etc use the same technique. GUIs which don't know about the option (and can't show wdl anyway) won't show it, if they're correctly implemented (UCI spec says UCI_ options shouldn't be displayed to the user).
So having the option enabled by default should not break UCI GUIs, but obviously it will break some non-fully compliant GUIs for no good reason.

User avatar
hgm
Posts: 24652
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Stockfish has included WDL stats in engine output

Post by hgm » Fri Jul 03, 2020 5:49 pm

What people suggest carries no weight. The official UCI specs are maintained by Stephan Mayer-Kahlen; I doubt you will find it there.

syzygy
Posts: 4600
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish has included WDL stats in engine output

Post by syzygy » Fri Jul 03, 2020 9:36 pm

hgm wrote:
Fri Jul 03, 2020 5:49 pm
What people suggest carries no weight. The official UCI specs are maintained by Stephan Mayer-Kahlen; I doubt you will find it there.
Well, if a number of engines and GUIs implement this variable, who am I to say that it is not a thing. If enough people start to use the word covfefe in a more or less uniform sense, it becomes part of the language.

User avatar
hgm
Posts: 24652
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Stockfish has included WDL stats in engine output

Post by hgm » Sat Jul 04, 2020 8:41 am

You cannot say that a GUI not implementing it is not fully UCI-compliant, though. Or did you mean that UCI compliance should make a GUI resistant against whatever garbage an engine might cough up? The specs require that of the engine; I am not sure they also require it of the GUI; they are written very much from an engine POV.

nnnnnnnn
Posts: 22
Joined: Mon Nov 18, 2019 1:36 pm
Full name: Mark Thellen

Re: Stockfish has included WDL stats in engine output

Post by nnnnnnnn » Sat Jul 04, 2020 10:23 am

nnnnnnnn wrote:
Thu Jul 02, 2020 3:48 am
wdl may be just what I need which is to determine the win/loss/draw probability...

(In fact I asked exactly this question last week, http://www.talkchess.com/forum3/viewtop ... =7&t=74243 , how to determine when to stop search based on a cp 0 score, and nobody mentioned wdl, which would have solved the issue.)
Just to be clear, based on information on another thread: the current WDL implementation is not helpful to me. It accounts for neither material nor tablebase information nor perpetual check (as someone pointed out in another thread, it reports an 11% of chance of win in K v. K for example).

AndrewGrant
Posts: 588
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: Stockfish has included WDL stats in engine output

Post by AndrewGrant » Sat Jul 04, 2020 10:30 am

lucasart wrote:
Thu Jul 02, 2020 4:25 am
Deberger wrote:
Thu Jul 02, 2020 2:30 am
Stockfish has included WDL stats in engine output.

Code: Select all

info depth 1 seldepth 1 multipv 1 score cp 110 wdl 355 620 25 nodes 111 nps 37000 tbhits 0 time 3 pv e2e3
info depth 2 seldepth 2 multipv 1 score cp 122 wdl 392 587 21 nodes 448 nps 149333 tbhits 0 time 3 pv e2e3 b7b6
info depth 3 seldepth 3 multipv 1 score cp 116 wdl 375 602 23 nodes 772 nps 257333 tbhits 0 time 3 pv e2e3 b7b6 g1e2
info depth 4 seldepth 4 multipv 1 score cp 75 wdl 253 707 40 nodes 2080 nps 520000 tbhits 0 time 4 pv e2e3 h7h6 g1e2 e7e6
info depth 5 seldepth 5 multipv 1 score cp 79 wdl 263 699 38 nodes 3586 nps 896500 tbhits 0 time 4 pv d2d4 e7e6 c2c3 b8c6 e2e3
info depth 6 seldepth 6 multipv 1 score cp 74 wdl 250 710 40 nodes 12718 nps 1589750 tbhits 0 time 8 pv d2d4 d7d5 b1c3 b8c6 e2e4
Well I suppose it's mildly interesting, but this should not be the default setting:

Code: Select all

option name UCI_ShowWDL type check default true
I am appalled by this patch... Really doubt this kind of silly gimmick would have made it when Marco was the maintainer.

An alpha beta search trying to pretend to be a MCTS, for marketing reasons.

I would expect as much from commercial closed source engines, like when Rybka was underreporting its search depth to fool naive users into thinking that it was a quantum leap in quality: we compute much less, but much better. Or course the BS was eventually exposed.
Agreed. Just as we should not care to see CP scores from Leela, we should not care to see WDL scores from Stockfish. It has no basis. Its purely cosmetic, and almost condescending to the user.

Post Reply