Whatever is current - Amazing Leela

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: Whatever is current - Amazing Leela

Post by corres »

Ovyron wrote: Tue Jul 09, 2019 6:45 pm
Uri Blass wrote: Sat Jul 06, 2019 11:03 amI wonder what you mean by "true score".
Haha, I was the same guy that a few weeks ago was saying that there's no such thing as a "true score". But it clearly does, and private books prove it, as they aim, and succeed, at having the best possible "true score" once out of book, and if it's high enough, you get a "book win" (the game was decided before the engine had to decide what to move.)

So a true score means how likely is that you win the game at the leaf node. There's actually two metrics, how likely is the game to end in draw, and how likely are you to win if the game is decided. Had I been around (with this knowledge) back when computer chess started and it was decided to show scores in centipawns, I'd have pushed for engines to show two scores (the one for winning chances, and another for drawing chances). Without this, engines are prone to play the worse move, where drawing chances skyrocket and it doesn't matter to have more than the opponent, you should be playing the move that keeps drawing chances lower if you have the advantage, even if your chance of winning minus the chance of winning of the opponent is lower than with the other move.

I suspect Robert Houdart figured this out at some point, as after some threshold, Houdini 6 will only show scores between -0.10 and 0.10 if the drawing chances seem too high, and -0.11<>0.11 if someone has decent chances of winning.

The "true score" of a move depends on how strong is your opponent. Against a stronger opponent, you want to increase the true score of the moves that lead to a draw. Against a weaker opponent, you want to increase the "true score" of moves that lead to decided games (0% draw chance is optimal, because it ensures you'll win.) Unfortunately, in practice you have no idea who you're going to face, so the true score falls in the middle.

So that, the "true score" of a move doesn't come from winning chances, drawing chances, or evaluation of position, all that matters is the ELO performance that the move would have against a pool of opponents.

Chess is all about ranking the moves of a position from best to worse, and playing the top one. A true score of a leaf node ends being arbitrary (by itself "0.25" has no meaning, you only put it there to mean some rating performance, so that it's a better one than "0.24" scores and worse than "0.26" scores, making non-leaf nodes to aim for the best.)

For non-leaf nodes the "true score" of a position is the leaf-node's "true score" if both sides play the best performing move, backsolved to the root. What I have is the opening position with 0.03 "true score" for 1.d4 and 0.00 (or worse) for the rest. The Spanish is the best try for 1.e4 but black has nothing better than the Marshall Counter-Attack, and white can't crack it. On the Italian black can force a "true score" that is better for black in the critical variation, no matter what white plays (which to me was a shocker.)

You can recognize critical variations playing against strong opponents that play the same moves you'd have played from the other side, because it means both of you arrived at the same leaf node (for optimal play for both sides), and any deviation from this would lead to worse performance (for this line. Of course the Spanish would have had a better performance than the Italian, so the Italian isn't critical from the opening position, but if you were forced to play it it has a critical line.)

The "David Vs. Goliath" scenario is very useful for detecting leaf nodes that are very good at elo performance. Just let Depth 20 Stockfish play one side of the board, and Depth 30 Stockfish the other. If Depth 20 Stockfish wins you know this variation is very good for that side and you can give it a high "true score."

Otherwise, if, let's say, your mainline leads to a leaf node that gives you a 60 elo advantage, but the opponent is 120 elo stronger, he'd only play 60 elo stronger than you, but they'll still beat you more often than not. I still find surprising that I'm able to gain such an elo advantage as black in the Italian.

What I find so funny is that not only people are ignorant of such things, they actually don't believe you when you tell them and find what you say ridiculous! I never expected becoming the conspiracy theorist of computer chess opening theory! :mrgreen: It is so funny, but I guess it's for the best.
What you wrote down in its essence is the practice of the OT GM applied to chess engines and the circumstances of infinitychess.com. To practice this method successfully you need not only a good "private book" but a lot of analysis (and time) for what you need a powerful PC- more stronger than a 10 years old with four cores.
Robert Pope
Posts: 558
Joined: Sat Mar 25, 2006 8:27 pm

Re: Whatever is current - Amazing Leela

Post by Robert Pope »

Ovyron wrote: Tue Jul 09, 2019 6:45 pm Haha, I was the same guy that a few weeks ago was saying that there's no such thing as a "true score". But it clearly does, and private books prove it, as they aim, and succeed, at having the best possible "true score" once out of book, and if it's high enough, you get a "book win" (the game was decided before the engine had to decide what to move.)

So a true score means how likely is that you win the game at the leaf node. There's actually two metrics, how likely is the game to end in draw, and how likely are you to win if the game is decided. Had I been around (with this knowledge) back when computer chess started and it was decided to show scores in centipawns, I'd have pushed for engines to show two scores (the one for winning chances, and another for drawing chances). Without this, engines are prone to play the worse move, where drawing chances skyrocket and it doesn't matter to have more than the opponent, you should be playing the move that keeps drawing chances lower if you have the advantage, even if your chance of winning minus the chance of winning of the opponent is lower than with the other move.
That's fine, but I would just say that in no way describes a "true score". How likely you are to win the game is entirely dependent on the person playing, for all but trivial situations. So your "true score" is no more than an engine's fallible view of the position, just that you want it to return different metrics.
Dann Corbit
Posts: 12537
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Whatever is current - Amazing Leela

Post by Dann Corbit »

Robert Pope wrote: Wed Jul 10, 2019 9:36 pm
Ovyron wrote: Tue Jul 09, 2019 6:45 pm Haha, I was the same guy that a few weeks ago was saying that there's no such thing as a "true score". But it clearly does, and private books prove it, as they aim, and succeed, at having the best possible "true score" once out of book, and if it's high enough, you get a "book win" (the game was decided before the engine had to decide what to move.)

So a true score means how likely is that you win the game at the leaf node. There's actually two metrics, how likely is the game to end in draw, and how likely are you to win if the game is decided. Had I been around (with this knowledge) back when computer chess started and it was decided to show scores in centipawns, I'd have pushed for engines to show two scores (the one for winning chances, and another for drawing chances). Without this, engines are prone to play the worse move, where drawing chances skyrocket and it doesn't matter to have more than the opponent, you should be playing the move that keeps drawing chances lower if you have the advantage, even if your chance of winning minus the chance of winning of the opponent is lower than with the other move.
That's fine, but I would just say that in no way describes a "true score". How likely you are to win the game is entirely dependent on the person playing, for all but trivial situations. So your "true score" is no more than an engine's fallible view of the position, just that you want it to return different metrics.
Any score besides a proven win, loss, or draw is an estimate. If it were not an estimate, then we would *know* if it wins, loses, or draws.
Therefore, not a true score.
Unless we bend Kierkegaard's "Truth is man's approximation of thought to reality." a bit.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Whatever is current - Amazing Leela

Post by Ovyron »

corres wrote: Wed Jul 10, 2019 10:56 amTo practice this method successfully you need not only a good "private book" but a lot of analysis (and time) for what you need a powerful PC- more stronger than a 10 years old with four cores.
No, you don't :) I've been proving that back when I had a slow 1CPU 32CPU and playing corr chess at the level of top corr chess players with octals at the time. But I won't toot my own horn, I'll let good old friend PortCitySlim (Bobby C) do it for me (this was a message posted in 2009, he's speaking to someone that was predicting I'd be crunched once out of book. I was called Vytron back then. The person he's talking to was the one that donated my current computer to me :) ):
Bobby C wrote:My bet is that there is a good chance this game will end in a draw, myself and Vytron make the perfect match-up when playing a game. It is me with powerful hardware vs Vytron with incredible talent, like a David and Goliath match-up and remember how that ended? If you were talking to Vytron then you might need to rethink how much of an advantage you give hardware, because if you think for one second that just because he has only 1CPU 32bit that he will definitely lose is a huge overstatement and I would love to see you challenge him 1v1 after this game. You will learn that your octal that you spent thousands on means absolutely nothing without a skilled driver behind the wheel, I am not saying you are not skilled but I think Vytron has enough talent to hold you to a draw at least 8 out of ten games.
It turns out I'd have drawn that game, but we didn't care about the result, but about knowing the true value of the position. We opened the game so everyone could send analysis and went into a chat session to discuss our best lines. It turned out that the moves he'd had played would have led to a draw, and after we saw it, I helped him find the winning plan to show the position was won. His computer was about 20 times faster than mine and he was analyzing more positions, yet with my infamous computer (which was referenced in a Rybka version as "snail") I was providing better analysis than what he could find.

Modesty be damned, I haven't done anything but improving in the 10 years since then, and I probably have the most accurate backsolved chess tree graph in the world (at least after comparing it to Nelson Hernandez, Thomas Zipproth and Dann Corbit's efforts.)

Of course, I stand over the shoulders of giants, and if I started enumerating the people I'd have to thank for this in a new post, it'd be longer than this one, but the point is hardware is overrated, I have at IC a higher rating than people in much better hardware (you could say it's mainly thanks to a book that doesn't belong to me, I could say the book just saved me months of work, but I'd have gotten there) and the reason I don't need better hardware is that I don't need to compensate for anything :mrgreen:
Your beliefs create your reality, so be careful what you wish for.
corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: Whatever is current - Amazing Leela

Post by corres »

Ovyron wrote: Thu Jul 11, 2019 2:53 am
corres wrote: Wed Jul 10, 2019 10:56 amTo practice this method successfully you need not only a good "private book" but a lot of analysis (and time) for what you need a powerful PC- more stronger than a 10 years old with four cores.
No, you don't. I've been proving that back when I had a slow 1CPU 32CPU and playing corr chess at the level of top corr chess players with octals at the time. But I won't toot my own horn, I'll let good old friend PortCitySlim (Bobby C) do it for me (this was a message posted in 2009, he's speaking to someone that was predicting I'd be crunched once out of book. I was called Vytron back then. The person he's talking to was the one that donated my current computer to me
I did not say you copy the method from a GM or from others.
I only establish the similarity between the method used by you and used by OT GMs.
Do you have FIDE Elo?
If you do not have it how high you value himself?
You mentioned you play(ed) CC games too.
What is/was your CC Elo?
corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: Whatever is current - Amazing Leela

Post by corres »

Ovyron wrote: Thu Jul 11, 2019 2:53 am ...
Modesty be damned, I haven't done anything but improving in the 10 years since then, and I probably have the most accurate backsolved chess tree graph in the world (at least after comparing it to Nelson Hernandez, Thomas Zipproth and Dann Corbit's efforts.)
If you are right (and I can believe you) your book contains huge human effort also and not only engine results.
Why I think it?
Because during ten years the chess power and capacity of analyzing of engines enhanced drastically and you can not check continuously every variation of your book - even you have a more stronger computer than a ten years old one what you mentioned.
Ovyron wrote: Fri Jul 12, 2019 3:53 am Of course, I stand over the shoulders of giants, ...
...
There are peoples who do not like this statement and they expect everybody should invent the sealing-wax themselves.
Typically they are not an engineer or a scientific co-worker but very Experte.
todd
Posts: 56
Joined: Thu Apr 19, 2018 9:09 pm

Re: Whatever is current - Amazing Leela

Post by todd »

Ovyron,

It is understandable you do not want to post your analysis in the main lines of a popular opening.

However, how about some lines that are not played as often in engine tournaments?

Can you show a path to advantage for black after any or all of:

a) 1. e4 e5 2. Nf3 Nc6 3. Bc4 Nf6 4. Ng5 (with white intending to play the variation 4...d5 5. exd5 Na5 6. Bb5+ c6 7. dxc6 bxc6 8. Bd3 Nd5 9. Nf3 - 9. h4 is also interesting but finding an advantage for black after Nf3 would be more astounding to me)

b) 1. e4 e5 2. Nf3 Nc6 3. Bc4 Nf6 4. d4 exd4 5. e5 (black can equalize without too much trouble, but I'd be seriously impressed to find a way to be better!)

c) 1. e4 e5 2. Nf3 Nc6 3. Bc4 Bc5 4. c3 Nf6 5. d4 exd4 6. e5 (similar to b but I'd be even more surprised to find a path to more than equality here for black)
carldaman
Posts: 2283
Joined: Sat Jun 02, 2012 2:13 am

Re: Whatever is current - Amazing Leela

Post by carldaman »

I think that if you substitute 'equalize/equality/whatever' for 'bust', it will all make sense. 8-)
User avatar
mclane
Posts: 18748
Joined: Thu Mar 09, 2006 6:40 pm
Location: US of Europe, germany
Full name: Thorsten Czub

Re: Whatever is current - Amazing Leela

Post by mclane »

Book theory is unimportant.
Moves and plan in a game count.
I still think LC0 versus stockfish is boring,
Stockfish is too stupid.
Let stockfish compete versus Komodo or Houdini. They deserve it,
But LC0 is about chess.
And not about elo shit,
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....
chrisw
Posts: 4313
Joined: Tue Apr 03, 2012 4:28 pm

Re: Whatever is current - Amazing Leela

Post by chrisw »

mclane wrote: Thu Jul 11, 2019 9:11 pm Book theory is unimportant.
Moves and plan in a game count.
I still think LC0 versus stockfish is boring,
Stockfish is too stupid.
Let stockfish compete versus Komodo or Houdini. They deserve it,
But LC0 is about chess.
And not about elo shit,
Put LCZero on nodes=0 or 1 and SF at fixed depth 7 or 8 or so. The games are like from 1990 between micros, full of errors but quite fun and comprehendable. Amazing what LCZero can do with no lookahead at all.