Whatever is current - Amazing Leela

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Whatever is current - Amazing Leela

Post by Ovyron »

Uri Blass wrote: Sat Jul 06, 2019 11:03 amI wonder what you mean by "true score".
Haha, I was the same guy that a few weeks ago was saying that there's no such thing as a "true score". But it clearly does, and private books prove it, as they aim, and succeed, at having the best possible "true score" once out of book, and if it's high enough, you get a "book win" (the game was decided before the engine had to decide what to move.)

So a true score means how likely is that you win the game at the leaf node. There's actually two metrics, how likely is the game to end in draw, and how likely are you to win if the game is decided. Had I been around (with this knowledge) back when computer chess started and it was decided to show scores in centipawns, I'd have pushed for engines to show two scores (the one for winning chances, and another for drawing chances). Without this, engines are prone to play the worse move, where drawing chances skyrocket and it doesn't matter to have more than the opponent, you should be playing the move that keeps drawing chances lower if you have the advantage, even if your chance of winning minus the chance of winning of the opponent is lower than with the other move.

I suspect Robert Houdart figured this out at some point, as after some threshold, Houdini 6 will only show scores between -0.10 and 0.10 if the drawing chances seem too high, and -0.11<>0.11 if someone has decent chances of winning.

The "true score" of a move depends on how strong is your opponent. Against a stronger opponent, you want to increase the true score of the moves that lead to a draw. Against a weaker opponent, you want to increase the "true score" of moves that lead to decided games (0% draw chance is optimal, because it ensures you'll win.) Unfortunately, in practice you have no idea who you're going to face, so the true score falls in the middle.

So that, the "true score" of a move doesn't come from winning chances, drawing chances, or evaluation of position, all that matters is the ELO performance that the move would have against a pool of opponents.

Chess is all about ranking the moves of a position from best to worse, and playing the top one. A true score of a leaf node ends being arbitrary (by itself "0.25" has no meaning, you only put it there to mean some rating performance, so that it's a better one than "0.24" scores and worse than "0.26" scores, making non-leaf nodes to aim for the best.)

For non-leaf nodes the "true score" of a position is the leaf-node's "true score" if both sides play the best performing move, backsolved to the root. What I have is the opening position with 0.03 "true score" for 1.d4 and 0.00 (or worse) for the rest. The Spanish is the best try for 1.e4 but black has nothing better than the Marshall Counter-Attack, and white can't crack it. On the Italian black can force a "true score" that is better for black in the critical variation, no matter what white plays (which to me was a shocker.)

You can recognize critical variations playing against strong opponents that play the same moves you'd have played from the other side, because it means both of you arrived at the same leaf node (for optimal play for both sides), and any deviation from this would lead to worse performance (for this line. Of course the Spanish would have had a better performance than the Italian, so the Italian isn't critical from the opening position, but if you were forced to play it it has a critical line.)

The "David Vs. Goliath" scenario is very useful for detecting leaf nodes that are very good at elo performance. Just let Depth 20 Stockfish play one side of the board, and Depth 30 Stockfish the other. If Depth 20 Stockfish wins you know this variation is very good for that side and you can give it a high "true score."

Otherwise, if, let's say, your mainline leads to a leaf node that gives you a 60 elo advantage, but the opponent is 120 elo stronger, he'd only play 60 elo stronger than you, but they'll still beat you more often than not. I still find surprising that I'm able to gain such an elo advantage as black in the Italian.

What I find so funny is that not only people are ignorant of such things, they actually don't believe you when you tell them and find what you say ridiculous! I never expected becoming the conspiracy theorist of computer chess opening theory! :mrgreen: It is so funny, but I guess it's for the best.
Your beliefs create your reality, so be careful what you wish for.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Whatever is current - Amazing Leela

Post by zullil »

Ovyron wrote: Tue Jul 09, 2019 6:45 pm
What I find so funny is that not only people are ignorant of such things, they actually don't believe you when you tell them and find what you say ridiculous! I never expected becoming the conspiracy theorist of computer chess opening theory! :mrgreen: It is so funny, but I guess it's for the best.
Perhaps people don't believe you because your claims seem extraordinary and you are unable to provide any evidence that they are true. :mrgreen:
Uri Blass
Posts: 10269
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Whatever is current - Amazing Leela

Post by Uri Blass »

Ovyron wrote: Tue Jul 09, 2019 6:45 pm
Uri Blass wrote: Sat Jul 06, 2019 11:03 amI wonder what you mean by "true score".
Haha, I was the same guy that a few weeks ago was saying that there's no such thing as a "true score". But it clearly does, and private books prove it, as they aim, and succeed, at having the best possible "true score" once out of book, and if it's high enough, you get a "book win" (the game was decided before the engine had to decide what to move.)

So a true score means how likely is that you win the game at the leaf node. There's actually two metrics, how likely is the game to end in draw, and how likely are you to win if the game is decided. Had I been around (with this knowledge) back when computer chess started and it was decided to show scores in centipawns, I'd have pushed for engines to show two scores (the one for winning chances, and another for drawing chances). Without this, engines are prone to play the worse move, where drawing chances skyrocket and it doesn't matter to have more than the opponent, you should be playing the move that keeps drawing chances lower if you have the advantage, even if your chance of winning minus the chance of winning of the opponent is lower than with the other move.

I suspect Robert Houdart figured this out at some point, as after some threshold, Houdini 6 will only show scores between -0.10 and 0.10 if the drawing chances seem too high, and -0.11<>0.11 if someone has decent chances of winning.

The "true score" of a move depends on how strong is your opponent. Against a stronger opponent, you want to increase the true score of the moves that lead to a draw. Against a weaker opponent, you want to increase the "true score" of moves that lead to decided games (0% draw chance is optimal, because it ensures you'll win.) Unfortunately, in practice you have no idea who you're going to face, so the true score falls in the middle.

So that, the "true score" of a move doesn't come from winning chances, drawing chances, or evaluation of position, all that matters is the ELO performance that the move would have against a pool of opponents.

Chess is all about ranking the moves of a position from best to worse, and playing the top one. A true score of a leaf node ends being arbitrary (by itself "0.25" has no meaning, you only put it there to mean some rating performance, so that it's a better one than "0.24" scores and worse than "0.26" scores, making non-leaf nodes to aim for the best.)

For non-leaf nodes the "true score" of a position is the leaf-node's "true score" if both sides play the best performing move, backsolved to the root. What I have is the opening position with 0.03 "true score" for 1.d4 and 0.00 (or worse) for the rest. The Spanish is the best try for 1.e4 but black has nothing better than the Marshall Counter-Attack, and white can't crack it. On the Italian black can force a "true score" that is better for black in the critical variation, no matter what white plays (which to me was a shocker.)

You can recognize critical variations playing against strong opponents that play the same moves you'd have played from the other side, because it means both of you arrived at the same leaf node (for optimal play for both sides), and any deviation from this would lead to worse performance (for this line. Of course the Spanish would have had a better performance than the Italian, so the Italian isn't critical from the opening position, but if you were forced to play it it has a critical line.)

The "David Vs. Goliath" scenario is very useful for detecting leaf nodes that are very good at elo performance. Just let Depth 20 Stockfish play one side of the board, and Depth 30 Stockfish the other. If Depth 20 Stockfish wins you know this variation is very good for that side and you can give it a high "true score."

Otherwise, if, let's say, your mainline leads to a leaf node that gives you a 60 elo advantage, but the opponent is 120 elo stronger, he'd only play 60 elo stronger than you, but they'll still beat you more often than not. I still find surprising that I'm able to gain such an elo advantage as black in the Italian.

What I find so funny is that not only people are ignorant of such things, they actually don't believe you when you tell them and find what you say ridiculous! I never expected becoming the conspiracy theorist of computer chess opening theory! :mrgreen: It is so funny, but I guess it's for the best.

If I understand correctly your claims are based on this site
http://www.infinitychess.com/

I registered to this site but never played a game there and even do not know how to do it.

I do not see a direct link for playing.
I see a link play vs a computer can you beat it
When I click on it I do not get a game.


Frequently asked questions are about books but I do not want to download book as first step but only to play a game without playing automatically opening moves of the engine when I can use engines but do not play moves automatically from book and I have no idea how to do it.
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Whatever is current - Amazing Leela

Post by Ovyron »

zullil wrote: Tue Jul 09, 2019 7:17 pmPerhaps people don't believe you because your claims seem extraordinary and you are unable to provide any evidence that they are true. :mrgreen:
"Able, but not willing" is a bit different :wink:
Uri Blass wrote: Tue Jul 09, 2019 7:22 pmIf I understand correctly your claims are based on this site
http://www.infinitychess.com/

I registered to this site but never played a game there and even do not know how to do it.

I do not see a direct link for playing.
I see a link play vs a computer can you beat it
When I click on it I do not get a game.


Frequently asked questions are about books but I do not want to download book as first step but only to play a game without playing automatically opening moves of the engine when I can use engines but do not play moves automatically from book and I have no idea how to do it.
You have to register here:

http://www.infinitychess.com/Register

Then you need to download the GUI and install it on your system here:

http://www.infinitychess.com/Downloads

You'd need to set up your engine and book in the GUI. Sign up with your registered account.

Every day at 16:00 on the server time an 11 rounds tournament is played. Unfortunately, mostly everyone has their best lines corked most of the time, though on Saturdays and Sundays you'll see some people playing their best lines.

When hell breaks loose is on the Engine Masters Tourney, where for "almost nothing" (US$500) people are willing to uncork all their best lines. On the weekdays I have no problem drawing all of my games against what people play. On the EMT I was beaten so badly I ended at the bottom of the table, and the only difference is the lines people play.

I'd be great to see you play, if you do, all these discussions wouldn't have been for nothing :)
Your beliefs create your reality, so be careful what you wish for.
Uri Blass
Posts: 10269
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Whatever is current - Amazing Leela

Post by Uri Blass »

Ovyron wrote: Tue Jul 09, 2019 7:31 pm
zullil wrote: Tue Jul 09, 2019 7:17 pmPerhaps people don't believe you because your claims seem extraordinary and you are unable to provide any evidence that they are true. :mrgreen:
"Able, but not willing" is a bit different :wink:
Uri Blass wrote: Tue Jul 09, 2019 7:22 pmIf I understand correctly your claims are based on this site
http://www.infinitychess.com/

I registered to this site but never played a game there and even do not know how to do it.

I do not see a direct link for playing.
I see a link play vs a computer can you beat it
When I click on it I do not get a game.


Frequently asked questions are about books but I do not want to download book as first step but only to play a game without playing automatically opening moves of the engine when I can use engines but do not play moves automatically from book and I have no idea how to do it.
You have to register here:

http://www.infinitychess.com/Register

Then you need to download the GUI and install it on your system here:

http://www.infinitychess.com/Downloads

You'd need to set up your engine and book in the GUI. Sign up with your registered account.

Every day at 16:00 on the server time an 11 rounds tournament is played. Unfortunately, mostly everyone has their best lines corked most of the time, though on Saturdays and Sundays you'll see some people playing their best lines.

When hell breaks loose is on the Engine Masters Tourney, where for "almost nothing" (US$500) people are willing to uncork all their best lines. On the weekdays I have no problem drawing all of my games against what people play. On the EMT I was beaten so badly I ended at the bottom of the table, and the only difference is the lines people play.

I'd be great to see you play, if you do, all these discussions wouldn't have been for nothing :)
I still did not buy a new computer and I am not sure if my computer has the minimum requirements

Minimum
Intel Core 2 Quad Q6600
2 GB RAM
Windows 7 and later versions
.Net FrameWork 4.5.2
Flash player 8.0
100% compatible soundcard
DVD-ROM drive

My system is
Intel(R) Core(TM) i3-2120 CPU 3.3GH

I know I have 4 GB RAM that is enough but I am not sure about the other parts.
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Whatever is current - Amazing Leela

Post by Ovyron »

Their "minimum requirements" are just a suggestion, as it turns out. There's players on there with hardware that is below the minimum, and IC allows them to play (back when I had 1GB RAM I used to play, even though it's below the minimum.)

I can troubleshoot you any problems that you have.

EDIT - For instance, you don't need a DVD-ROM drive to play at all, I have no idea why they list it.
Your beliefs create your reality, so be careful what you wish for.
Uri Blass
Posts: 10269
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Whatever is current - Amazing Leela

Post by Uri Blass »

I played one game when I lost as expected.

I did not have a book and I used only 1 cpu stockfish with black and initially I did not understand how to use an engine at the site.
I did not play well and simply played the moves of the stockfish manually when stockfish ran on the site.

Unfortunately I did not save my game and I even cannot look at it because minimum knight rank is required to kibitz games.

I can add that I played in the engine hall.
I edited earlier my user to computer and I am not sure if I was right to do it when I basically played manually moves of the engine.
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Whatever is current - Amazing Leela

Post by Ovyron »

On InfinityChess's main window (what you see when you open the software) click on Database, on the new page that appears click Open Database, then open MyOnlineEngineGame.icdx, your played game should be on there.

My computer is very similar to yours (4CPU Q8200 3GB RAM), and because the GUI is very unstable, I have to use 3CPU for the engine, leaving the other to the GUI. And yet I've been able to avoid losing 22 games in a row against people in recent hardware running Stockfishes and Leelas (some that just are 10 elo stronger than me), because of the books I use, so you might do just fine.

Note: I didn't need a rank to do anything special, I was Rank Pawn for over 7 years, and I only upgraded to rank Knight (which requires playing 50 Rapid rated games) when I needed to chat to request the Eman engine.
Your beliefs create your reality, so be careful what you wish for.
Uri Blass
Posts: 10269
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Whatever is current - Amazing Leela

Post by Uri Blass »

thanks and I see how I can see my games.

After losing 2 games without book with black when I played e4 c5 I drew a game with e4 e5

Note that I saw that I have a small advantage of 0.13 so decided to use almost all my time with the idea that if stockfish shows a bigger score then a minimal advantage I will play stockfish's move and if not I force a draw.

stockfish did not increase the number so I decided to force a draw after 17 minutes to avoid losing on time and avoid risks.

Note that I do not know how to use the interface to play moves backward and forward in order to analyze.
Note also that I find it stupid that the opponent show the exact evaluation to me when the opponent move.

Maybe this data can help me and it certainly does not help the opponent to show the computer's evaluation.
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Whatever is current - Amazing Leela

Post by Ovyron »

So you're playing Freestyle chess, then? That's interesting, all the people there play in Engine mode, so the engine manages its time and plays the whole game without human intervention. I don't think IC's tourneys allow centaurs.
Uri Blass wrote:Note that I do not know how to use the interface to play moves backward and forward in order to analyze.
For anything else than playing online, the IC GUI is really poor (even, for playing online the IC GUI is really poor - say, much worse than Winboard - but it's the only way to play there.) The advice is to have open another GUI that you're already comfortable using to analyze with, and switch back and forth between it and IC to analyze and play moves, if you'll remain playing like centaur. But the meat and potatoes happens in engine mode, where you set up the engine and it does everything by itself without your intervention.
Your beliefs create your reality, so be careful what you wish for.