Stockfish and Multi-PV

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Joerg Oster
Posts: 669
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Stockfish and Multi-PV

Post by Joerg Oster » Mon Jun 03, 2019 2:22 pm

Rebel wrote:
Mon Jun 03, 2019 1:29 pm
I am disappointed, I was wondering if it made sense SF could build an opening book by itself using Multi-PV. First results (analysis time 100ms) looked promising 48.3% against the ProDeo book, 49.4% (analysis time 1000ms), now trying 2000ms but I think I am wasting my time looking at the moves it sometimes produces. Was hoping for a bug that could be fixed but looking at the responses that's unlikely.
You could apply this patch https://github.com/official-stockfish/S ... /pull/2163
which should speedup the MultiPV search to some extent, but I doubt it will noticeably improve the quality of the moves.
Jörg Oster

User avatar
Rebel
Posts: 4518
Joined: Thu Aug 18, 2011 10:04 am

Re: Stockfish and Multi-PV

Post by Rebel » Mon Jun 03, 2019 7:35 pm

Joerg Oster wrote:
Mon Jun 03, 2019 2:22 pm
Rebel wrote:
Mon Jun 03, 2019 1:29 pm
I am disappointed, I was wondering if it made sense SF could build an opening book by itself using Multi-PV. First results (analysis time 100ms) looked promising 48.3% against the ProDeo book, 49.4% (analysis time 1000ms), now trying 2000ms but I think I am wasting my time looking at the moves it sometimes produces. Was hoping for a bug that could be fixed but looking at the responses that's unlikely.
You could apply this patch https://github.com/official-stockfish/S ... /pull/2163
which should speedup the MultiPV search to some extent, but I doubt it will noticeably improve the quality of the moves.
I am not very familiar with Github, where can I find the executable of your patch?
Everybody is unique, except me.

User avatar
Rebel
Posts: 4518
Joined: Thu Aug 18, 2011 10:04 am

Re: Stockfish and Multi-PV

Post by Rebel » Mon Jun 03, 2019 7:40 pm

For those interested with SOMU 1.4 - Analyze EPD+++ you can analyze an EPD collection with Multi-PV. Just try it with the example.epd in the EPD folder. The util to make a Polyglot book from the result will be ready likely tomorrow.

http://rebel13.nl/download/utilities.html
Everybody is unique, except me.

Joerg Oster
Posts: 669
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Stockfish and Multi-PV

Post by Joerg Oster » Tue Jun 04, 2019 9:10 am

Rebel wrote:
Mon Jun 03, 2019 7:35 pm
Joerg Oster wrote:
Mon Jun 03, 2019 2:22 pm
Rebel wrote:
Mon Jun 03, 2019 1:29 pm
I am disappointed, I was wondering if it made sense SF could build an opening book by itself using Multi-PV. First results (analysis time 100ms) looked promising 48.3% against the ProDeo book, 49.4% (analysis time 1000ms), now trying 2000ms but I think I am wasting my time looking at the moves it sometimes produces. Was hoping for a bug that could be fixed but looking at the responses that's unlikely.
You could apply this patch https://github.com/official-stockfish/S ... /pull/2163
which should speedup the MultiPV search to some extent, but I doubt it will noticeably improve the quality of the moves.
I am not very familiar with Github, where can I find the executable of your patch?
Sorry, Ed, this is still an open Pull Request under review and has not been merged into master.
I was sure you could do your own compiles ...
Jörg Oster

User avatar
Rebel
Posts: 4518
Joined: Thu Aug 18, 2011 10:04 am

Re: Stockfish and Multi-PV

Post by Rebel » Wed Jun 05, 2019 6:05 am

Joerg Oster wrote:
Tue Jun 04, 2019 9:10 am
Rebel wrote:
Mon Jun 03, 2019 7:35 pm
Joerg Oster wrote:
Mon Jun 03, 2019 2:22 pm
Rebel wrote:
Mon Jun 03, 2019 1:29 pm
I am disappointed, I was wondering if it made sense SF could build an opening book by itself using Multi-PV. First results (analysis time 100ms) looked promising 48.3% against the ProDeo book, 49.4% (analysis time 1000ms), now trying 2000ms but I think I am wasting my time looking at the moves it sometimes produces. Was hoping for a bug that could be fixed but looking at the responses that's unlikely.
You could apply this patch https://github.com/official-stockfish/S ... /pull/2163
which should speedup the MultiPV search to some extent, but I doubt it will noticeably improve the quality of the moves.
I am not very familiar with Github, where can I find the executable of your patch?
Sorry, Ed, this is still an open Pull Request under review and has not been merged into master.
I was sure you could do your own compiles ...
Sure, your assumption is right, I just did not expect it works that way.

You are right about the "red", increasing the analysis time to 2000ms and 5000ms thereafter only gave regressions. It's hopeless. I think I will have to switch to Lc0 for the purpose.
Everybody is unique, except me.

User avatar
MikeGL
Posts: 864
Joined: Thu Sep 01, 2011 12:49 pm

Re: Stockfish and Multi-PV

Post by MikeGL » Wed Jun 05, 2019 12:52 pm

Even Leela avoids the direct 3.d4 move and prefers 3.Bb5 instead.



Unusual game, giving up pawns for free in openning and getting unknown compensation then went on to win.


edit: Ok I looked at it again, and saw the weird compensation seems to be the black N being misplaced and can't reach an ideal outpost until the end of the game. But really, that's too deep a compensation for the gambit pawn given at move nine and realizing the compensation at move 30. .
I told my wife that a husband is like a fine wine; he gets better with age. The next day, she locked me in the cellar.

User avatar
Rebel
Posts: 4518
Joined: Thu Aug 18, 2011 10:04 am

Re: Stockfish and Multi-PV

Post by Rebel » Wed Jun 05, 2019 2:52 pm

MikeGL wrote:
Wed Jun 05, 2019 12:52 pm
Even Leela avoids the direct 3.d4 move and prefers 3.Bb5 instead.
But Leela, like most average engines, will have 3.d4 in their list with Multi-PV=3.
Everybody is unique, except me.

jp
Posts: 656
Joined: Mon Apr 23, 2018 5:54 am

Re: Stockfish and Multi-PV

Post by jp » Wed Jun 05, 2019 5:23 pm

MikeGL wrote:
Wed Jun 05, 2019 12:52 pm
Even Leela avoids the direct 3.d4 move and prefers 3.Bb5 instead.

[Event "CCC: Testing Updates II"][Date "2019.06.02"][Round "1"]
[White "Leelafish"][Black "Brainfish"]
[Result "1-0"]
[Variation "Nimzovich-Rossolimo attack (without ...d6)"]

Unusual game, giving up pawns for free in openning and getting unknown compensation then went on to win.

edit: Ok I looked at it again, and saw the weird compensation seems to be ...
Well, we don't know those were good moves just because it went on to win the game. What was the TC?

User avatar
MikeGL
Posts: 864
Joined: Thu Sep 01, 2011 12:49 pm

Re: Stockfish and Multi-PV

Post by MikeGL » Wed Jun 05, 2019 11:52 pm

jp wrote:
Wed Jun 05, 2019 5:23 pm
MikeGL wrote:
Wed Jun 05, 2019 12:52 pm
Even Leela avoids the direct 3.d4 move and prefers 3.Bb5 instead.

[Event "CCC: Testing Updates II"][Date "2019.06.02"][Round "1"]
[White "Leelafish"][Black "Brainfish"]
[Result "1-0"]
[Variation "Nimzovich-Rossolimo attack (without ...d6)"]

Unusual game, giving up pawns for free in openning and getting unknown compensation then went on to win.

edit: Ok I looked at it again, and saw the weird compensation seems to be ...
Well, we don't know those were good moves just because it went on to win the game. What was the TC?
[Event "CCC: Testing Updates II"]
[Site "?"]
[Date "2019.06.02"]
[Round "1"]
[White "Leelafish"]
[Black "Brainfish"]
[Result "1-0"]
[ECO "B30"]
[GameDuration "00:30:38"]
[GameEndTime "2019-06-02T02:48:50.956 PDT"]
[GameStartTime "2019-06-02T02:18:12.260 PDT"]
[Opening "Sicilian"]
[PlyCount "259"]
[TimeControl "600+2"]
[Variation "Nimzovich-Rossolimo attack (without ...d6)"]

so 10 mins + 2 seconds increment.
Sorry, the PGN header was cleaned up by my GUI during copy paste.
I told my wife that a husband is like a fine wine; he gets better with age. The next day, she locked me in the cellar.

User avatar
Ovyron
Posts: 2256
Joined: Tue Jul 03, 2007 2:30 am

Re: Stockfish and Multi-PV

Post by Ovyron » Fri Jun 07, 2019 3:37 am

Rebel wrote:
Mon Jun 03, 2019 1:29 pm
I think I am wasting my time looking at the moves it sometimes produces. Was hoping for a bug that could be fixed but looking at the responses that's unlikely.
There's a bug, but it's not on the engine.

This situation happened (note, this is not about Stockfish, all engines suck like this, unless they get helped by theory, like ProDeo or Shredder <12, or Eman's Experience File) because engine testers chose to play with generic books that take over the opening and play reasonable moves for the engine, and when the engine has to play it already skipped the problem entirely.

Engine developers have had no reason whatsoever to get better at the opening, and self-play has had a similar resolution. The elo chase is real, and even Komodo, an engine that has been created with "analysis first" on mind, doesn't behave better in the opening (heck, it doesn't behave better in the early middle game either.)

InfinityChess's Chess Engine Master tournament has shown that Leela isn't good at the openings either, because people are gaining much better advantage by using books, and it seems a book is good on its own and it doesn't matter if Stockfish or Leela play when out of book, and Leela doesn't have the time to find a better move than the one the book'd have played.
Make someone happy today.

Post Reply