Out of the Kai 450 positions, 99 not found by SF

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 9246
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Laskos » Tue Aug 13, 2019 9:42 pm

Uri Blass wrote:
Tue Aug 13, 2019 1:17 pm

I think that it may be interesting if somebody throw away all opening theory and replace it with stockfish analysis with a lot of computer time and see if other books can beat this book.
Laskos wrote:
Tue Aug 13, 2019 2:22 pm
Yes, I maybe will do such a thing in 2 LTC games. Lc0 with "bignet", which seems to agree by far the most with the human opening theory against SF_dev from the standard opening position, side and reversed. I am curious how Lc0 "bignet" will evaluate SF_dev opening moves and quiet midgame moves.
Look what Lc0 Bignet (which agrees most of the times with human openings and midgames) thinks of SF_dev (the last one) opening and midgame performance from the initial standard opening position in LTC 60 min + 15 seconds games side and reversed. RTX 2070 and 4 cores i7. The reverse game is not yet finished, but the opening and midgame are pretty much there, and I am going to bed. SF_dev after some time has to agree with the bad opinion of Lc0 about its play in opening and midgame.

First game: Lc0 Bignet White:
40b_W.jpg
40b_W.jpg (57.23 KiB) Viewed 341 times
SF_dev was outplayed in the opening and midgame (and SF agrees too), and drew because Lc0 made a mistake in the endgame.


Second game: Lc0 Bignet Black (unfinished).
40b_B.jpg
40b_B.jpg (55.68 KiB) Viewed 341 times
SF_dev was again outplayed in the opening and midgame (and SF agrees again), with unclear result.


I am curious. You are an experienced ICCF player, a GM if I am not wrong. You should know a lot about these things. Really, would you rely on Stockfish long back and forth analysis in the openings? Would you rely on Rybka 3? On Fritz 7? When you started to trust regular engines in the openings?

Uri Blass
Posts: 8507
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Uri Blass » Wed Aug 14, 2019 2:51 am

Laskos wrote:
Tue Aug 13, 2019 2:22 pm
Uri Blass wrote:
Tue Aug 13, 2019 1:17 pm
<snipped>
Laskos wrote:
Tue Aug 13, 2019 12:32 pm
What is so bad about openings and humans? Most of opening theory to our days is built by humans, engines' contribution is small and mostly came with "tricks", and often in messy positions. I think you just imply that we can throw away all opening theory and replace it with Stockfish analysis. That is, with the Cerebellum book. Good luck with that!
How do you measure that most of opening theory is built by humans?

It is possible that humans played many moves based on analysis by engines so the fact that they were the first to play a move does not prove that engines did not practically built it.

You can say only about old theory that it is built by humans and it is not clear about novelties in the last 20 years.
Engines started to play _reasonable_ (not good) openings starting with Rybka late incarnations some 10 years ago, due to much handcrafted knowledge introduced in it. In the early 2000s there was a joke --- "engines play excellent chess, but...let them play White, they will start with Nf3, reply with d5, and they will play...Nc3!!!". I don't think that engines until recently were more than a side-tool for OTB GM games in the openings, some sort of sanity check in preparations. And the trend is obvious: From Fritz 7 to Stockfish_dev, it is the engines playing more human-like openings, not the humans playing more engine-like openings.

Do you have examples of large chunks of opening theory built with a substantial engine contribution (not blunder check or sanity check or tactical tricks contributions, that is irrelevant to the point of positional play)?
I think that it may be interesting if somebody throw away all opening theory and replace it with stockfish analysis with a lot of computer time and see if other books can beat this book.
Yes, I maybe will do such a thing in 2 LTC games. Lc0 with "bignet", which seems to agree by far the most with the human opening theory against SF_dev from the standard opening position, side and reversed. I am curious how Lc0 "bignet" will evaluate SF_dev opening moves and quiet midgame moves.
I think that testing the level of engines even in year 2000 should not be done at short time control because people could give the computer hours for analysis at home in order to find novelties.

I wonder if there is a top engine that suggested 2.Nc3 after 1.Nf3 d5 when you gave it one hour to search and note that using engines to find novelties does not always mean choosing the top moves of chess engines.

Note that I used chess engines in correspondence games from the opening stage and did not trust opening theory when I played correspondence chess(I do not play correspondence chess today).

Uri Blass
Posts: 8507
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Uri Blass » Wed Aug 14, 2019 3:01 am

Laskos wrote:
Tue Aug 13, 2019 9:42 pm
Uri Blass wrote:
Tue Aug 13, 2019 1:17 pm

I think that it may be interesting if somebody throw away all opening theory and replace it with stockfish analysis with a lot of computer time and see if other books can beat this book.
Laskos wrote:
Tue Aug 13, 2019 2:22 pm
Yes, I maybe will do such a thing in 2 LTC games. Lc0 with "bignet", which seems to agree by far the most with the human opening theory against SF_dev from the standard opening position, side and reversed. I am curious how Lc0 "bignet" will evaluate SF_dev opening moves and quiet midgame moves.
Look what Lc0 Bignet (which agrees most of the times with human openings and midgames) thinks of SF_dev (the last one) opening and midgame performance from the initial standard opening position in LTC 60 min + 15 seconds games side and reversed. RTX 2070 and 4 cores i7. The reverse game is not yet finished, but the opening and midgame are pretty much there, and I am going to bed. SF_dev after some time has to agree with the bad opinion of Lc0 about its play in opening and midgame.

First game: Lc0 Bignet White:

40b_W.jpg

SF_dev was outplayed in the opening and midgame (and SF agrees too), and drew because Lc0 made a mistake in the endgame.


Second game: Lc0 Bignet Black (unfinished).

40b_B.jpg

SF_dev was again outplayed in the opening and midgame (and SF agrees again), with unclear result.


I am curious. You are an experienced ICCF player, a GM if I am not wrong. You should know a lot about these things. Really, would you rely on Stockfish long back and forth analysis in the openings? Would you rely on Rybka 3? On Fritz 7? When you started to trust regular engines in the openings?
I do not play today correspondence games but I believe that good moves can be verified by back and forth analysis with the right lines.
The fact that stockfish played the opening worse than leela does not mean that you cannot find stockfish's mistakes by analysis with stockfish and
note that a book based on stockfish's analysis can be based on more computer time than a 60 minutes+15 time control game.

The idea is not to trust a move that stockfish suggest but to build a tree with evaluation and scores based on stockfish.
Stockfish may use some minutes for every node in the tree but it may also not choose the top move that it suggested at every node(because the choice of the move is based on the scores that it has).

For example if stockfish at multi-pv suggest 1.e4 0.20 1.d4 0.19 1.c4 0.18 1.Nf3 0.17(scores always from white point of view) then it may later analysis 1.e4 and see 1.e4 e6 0.1 is best for black and later analyze 1.d4 and see 1.d4 Nf6 0.2 is best for black so decide 1.d4 is better(of course the tree should be bigger and I only explain the idea).

Dann Corbit
Posts: 9849
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Dann Corbit » Wed Aug 14, 2019 5:15 am

As far as how do engines improve opening theory, consider these contests by Sedat:
https://sites.google.com/site/computers ... book-cs-29
The top books are heavily computer influenced. I guess that the "noob" book is based.on.the 20 ply Stockfish data but I just guess that by the name.

Of course,many opening lines are analyzed by decades of human examination. This does debug most serious gaffes. Top correspondence players have some really good hardware to certify that their ideas are not faulty. NOw, when there are no tactical snacks within 45 plies, I suspect that NN engines give the best response (given good hardware and sufficient time).

I suspect eventually we will have hybrid algorithms that are better than anything we have now, or something even better. We also have massive piles of statistics building up.

The way I see it, we should use all the tools in the toolbox. You get better work that way.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Uri Blass
Posts: 8507
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Uri Blass » Wed Aug 14, 2019 12:24 pm

Laskos wrote:
Tue Aug 13, 2019 9:42 pm


I am curious. You are an experienced ICCF player, a GM if I am not wrong. You should know a lot about these things. Really, would you rely on Stockfish long back and forth analysis in the openings? Would you rely on Rybka 3? On Fritz 7? When you started to trust regular engines in the openings?
I relied on engines in the opening before the time of Rybka3

I practically lost only one game in the years that I played correspondence chess for ICCF rating.(I mean games that I played from the first move and there were some games when I replaced another player that I do not count)

game started in 16.10.2004 and the last move of my opponent is from 2006.
Here is the game with some comments about my mistakes




Analysis after the game today with stockfish suggest that I did 3 mistakes(I did not understand it at the time of the game).

1)16...Rc8(16...d5 gives me a better position based on analysis by stockfish today) date of the mistake 19.12.2005
2)18...Be7(18...d5 gives me equality based on analysis by stockfish today) date of the mistake 8.3.2006
3)22...g6(22...Bxb5 seems to give good chances for a draw) date of the mistake 12.6.2006

Note that I did not do a lot of analysis today but even one hour of analysis today give better results then analysis that I did in 2004 in some days

After the game I thought that I blundered by allowing the sacrifice but it seems that I was wrong.

Here are part of my comments after the game in a mail for my opponent after resigning:

I decided not to trust theory and use long search of programs that often help but not in this case.

At that time that I played 11...Qc7 I already analyzed the sacrifice 14.Ndxb5 but when I analyzed the position after 14...Kd8 for hours I saw an advantage for black and I saw no increase in score for white even after a long search(I used fruit that was the best program at that time but it seems that rybka that is the best now is not better there).

User avatar
Ovyron
Posts: 2256
Joined: Tue Jul 03, 2007 2:30 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Ovyron » Thu Aug 15, 2019 5:39 am

zullil wrote:
Tue Aug 13, 2019 10:39 am
What does "best move" or "right move" even mean here? Surely there are many moves in most of these positions that are equal with correct play---likely all leading to draws.
The "best move" is the one that makes it the hardest for the opponent to find the "correct play", which in some positions may turn out to be extremely hard, or even impossible (I've been on the receiving end.) This depends on the opponent, though at least, something that works for strong opponents works for weaker opponents.

I concur with Uri that the only way to test this is building a chess tree (... or graph, or whatever that allows you to see the statistics/score for 1.e3 and it shows all statistics of all games that include e3 somewhere, as if 1.e3 was played first, because a common flaw is needing to play the transposing moves to see right statistics/scores on the database.)

Finally, the best move depends on "obscurity" also, since the very best line that makes it the hardest for an opponent becomes easy once someone played against it and found the draw and the opponent only needs to copy-paste the moves, the "best move" of a position can change.

What I propose is starting to explore the variations that defeat a weak opponent, say, Stockfish Depth 10. There are many moves in positions that aren't strong enough to defeat Stockfish Depth 10, some others do beat it, so clearly, despite all moves leading from one drawn position to another, the ones that defeat SFD10 are clearly better. Once you have them, you can explore the ones that defeat Stockfish Depth 11, Depth 12, and so on.

Eventually you'll find a road block, some Stockfish depth that you can't defeat, so more investigation will not help. However, on positions that go into this position, there would have been deviations where the moves would have led to an early road block (say, Depth 28 is enough to hold the position) or to a later road block (you can beat Stockfish Depth 40), so this clearly can be used to rank the moves, and the top one would be the best.
Make someone happy today.

zullil
Posts: 5445
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Out of the Kai 450 positions, 99 not found by SF

Post by zullil » Thu Aug 15, 2019 11:43 am

Ovyron wrote:
Thu Aug 15, 2019 5:39 am
zullil wrote:
Tue Aug 13, 2019 10:39 am
What does "best move" or "right move" even mean here? Surely there are many moves in most of these positions that are equal with correct play---likely all leading to draws.
The "best move" is the one that makes it the hardest for the opponent to find the "correct play", which in some positions may turn out to be extremely hard, or even impossible (I've been on the receiving end.) This depends on the opponent, though at least, something that works for strong opponents works for weaker opponents.
"The" best move is the "one" that makes finding "the" correct play most difficult. Sounds awfully unworkable to me. :wink:

User avatar
Ovyron
Posts: 2256
Joined: Tue Jul 03, 2007 2:30 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Ovyron » Thu Aug 15, 2019 2:40 pm

zullil wrote:
Thu Aug 15, 2019 11:43 am
"The" best move is the "one" that makes finding "the" correct play most difficult. Sounds awfully unworkable to me. :wink:
And yet I just beat this guy yesterday and that gave me the IM norm...


1-0

So, in practice, finding the best moves is very workable :)

That's why I don't get why people would analyze chess positions for the sake of it, as best moves depend on what's the opponent's level. I have been able to calibrate, and play the moves that defeat someone, after playing several games with them, just by knowing how they play (...I haven't been able to do that by just looking at their already played games.) If with all your analysis, you're not beating someone, you're wasting your time.
Make someone happy today.

zullil
Posts: 5445
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Out of the Kai 450 positions, 99 not found by SF

Post by zullil » Thu Aug 15, 2019 2:47 pm

Ovyron wrote:
Thu Aug 15, 2019 2:40 pm
zullil wrote:
Thu Aug 15, 2019 11:43 am
"The" best move is the "one" that makes finding "the" correct play most difficult. Sounds awfully unworkable to me. :wink:
If with all your analysis, you're not beating someone, you're wasting your time.
I think we have rather different world views. :D

User avatar
Ovyron
Posts: 2256
Joined: Tue Jul 03, 2007 2:30 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Ovyron » Thu Aug 15, 2019 2:53 pm

But in your world view, it's not chess at all, as at the base, there's two colors, white and black, you have to be one of the colors to play. If you're not one of the colors, you're not playing chess, and if you're both colors, you're not playing chess. So what are you doing?
Make someone happy today.

Post Reply