Something Hikaru Said

[Desperado]

Hello everybody.

I began to read this thread from the beginning and after reading some posts i realized that we might answer much earlier than expected. Here is the idea:

Why using a 32 men egtb if 31 would be sufficient for a knight handicap

[Uri Blass]

None of this seems reasonable to me. The percentage of GM errors, for example. A GM makes far FEWER errors against a weak program than against a strong program, but not because he actually makes fewer errors, but because the opponent doesn't notice them and doesn't punish them.

I disagree here.
If the opponent does not play well it is easier not to make mistakes.

I clearly have games against humans when I did no significant mistakes based on computer analysis(no move reduce the evaluation by more than 0.2 pawn).

It is not because I am so strong but because it is easier not to make mistakes when the opponent does not play well.
If the opponent play well I expect myself to do more mistakes.

Uri

"no significant mistakes based on computer analysis" is meaningless when we are talking about PERFECT computer play. ANY mistake will be significant there. This extrapolation about what happens today is meaningless when we talk about a perfect chess opponent.

It is not meaningless because the computer is clearly stronger than me and find many mistakes.

If the computer find that I play more mistakes in games that I play against stronger opponents it means that it is easier to make mistakes against stronger opponents.

I do not have the 32 piece tablebases but my speculation is that there are many games when the winner did not do a mistake that change the theoretical result and also there are draws with no mistakes.
Of course the side who did no mistake could do mistakes in case of playing against a stronger opponent.

[syzygy]

Now take the perfect engine instead of Komodo. First, it makes a human blunder 0.6% of the moves (some increase in blunder rate because of trickier positions), punishes all blunders and lengthens the game to 90 moves (some increase in game length). These small incremental factors add up to make a large difference:

You might be right that average game length will increase, and it is certainly not a bad idea to take conservative numbers which then still show that the GM will prevail.

But I somehow doubt that a 500 Elo stronger Komodo will put up that much more resistance. The Komodo we have now is already so much stronger than the 2500 Elo GM that I doubt that that extra 500 Elo is going to have much impact on Komodo - 2500 Elo GM play.

Consider for example a 2000 Elo human playing a 2800 Elo engine. Now let him play a 3300 Elo engine. Is the 2000 Elo human really going to notice the difference? Will average game length decrease by much?

I agree that the "punish rate" will increase as engine Elo goes up.

[syzygy]

None of this seems reasonable to me. The percentage of GM errors, for example. A GM makes far FEWER errors against a weak program than against a strong program, but not because he actually makes fewer errors, but because the opponent doesn't notice them and doesn't punish them.

And my calculation takes that fully into account.

[bob]

I don't know that the machine will be able to win, but there is absolutely ZERO evidence to support that it will always lose. Absolutely ZERO. To make such a claim, there must be something to support it.

what is the reason that this statement of larry would not be evidence that the machine will lose to knight handicap ?

is it because he has not proven that K/C ratio cannot be 64%, or something else. ?

http://www.talkchess.com/forum/viewtopi ... 49&t=58846

Let's say that there is some handicap that would produce an even score in a serious match between Komodo and Magnus Carlsen. I would estimate that this handicap would be in the 1 to 1.5 pawn range based on Komodo's eval after a long think. Let's say 1.25. Now both Komodo and Carlsen make errors of some average magnitude. We'll call Carlsen's error rate C, and Komodo's K. I think it's pretty obvious that K is much less than C, let's say K = .4xC. If a future engine drops the error rate to zero, then C - K increaases to 5/3 of it's former value, so the proper handicap should also increase in that ratio. That would put it at 208, a bit over two pawns but way below the roughly 3.5 value of knight odds. Of course there is a lot of uncertainty in the above, but I don't think the estimate would be way off. In order for the estimate to reach knight odds, the K/C ratio would have to be about 64%, which does not seem plausible to me

How can you prove anything with no way to verify? He said something like the thought Komodo was within 1 pawn of playing perfectly. Not from the games I have seen. It is very strong, but it is FAR short of playing perfectly, based on analysis of output I have seen... And that is just today, of course. What happens in the future, or even beyond the future with the hypothetical access to 31 piece endgame tables?

If you backed up in time, say 20 years, would anyone be claiming that a computer could give a GM anything and still win? Not in any discussion I ever took part in, even at ACM/ICGA/ICCA chess events. So the bar has moved higher in 20 years. Is there empirical evidence that suggests that we are "at the max point of computer chess strength" already?

10-20 years ago we had lots of claims that computers were GMs when they were not. Now we seem to be at the point where some think the computer plays almost perfect chess, which it can't. And then some also assume that a human can play a knight odds game perfectly. He can't play a non-odds game perfectly so what suggests that taking away one of the opponent's pieces suddenly increases his skill level so dramatically?

I don't know what the max odds will be, and don't really care either. But to claim there is some asymptote that can't be crossed seems just a tae premature when there is no evidence to support this.

(1)In my rather limited understanding of all this I believe (rather than arguing Komodo was within 1 pawn of playing perfectly.) that larry was arguing from a current komodo 1.25 advantage over super gm to a future 32 piece perfect engine, the advantage would only increase to about 2.2 advantage.

his argument was based that in a perfect engine C - K increases only to 5/3 of it's former value.


what was the fallacy in his argument.?

(2)do you know how about many moves it takes a gm to convert a knight handicap from 3 to 4 advantage?

(3)let's say a gm can convert a knight handicap from 3 to 4 in 9 moves. would a future 60,000 elo computer have to be able to get the score from 3 to 2 in less than 9 moves to win.?

What is fallacy? Where does the 5/3 come from? Who has any idea what a perfect player will be able to do? That's the fallacy as what happens today means no more than what happened 10 or 20 years ago.

(2) depends on the opponent. Against a beginner? 5 moves? Against a master? 15 moves? Against another GM? 30 moves? But all irrelevant when asking about "against a perfect opponent"...

(3) The "9 moves" is pure speculation. That's the problem. How would a USCF 2000 player fare against a top program today? And that is nowhere near the difference between a GM and the perfect player, which is almost certainly thousands of Elo yet to be seen.

[bob]

None of this seems reasonable to me. The percentage of GM errors, for example. A GM makes far FEWER errors against a weak program than against a strong program, but not because he actually makes fewer errors, but because the opponent doesn't notice them and doesn't punish them.

I disagree here.
If the opponent does not play well it is easier not to make mistakes.

I clearly have games against humans when I did no significant mistakes based on computer analysis(no move reduce the evaluation by more than 0.2 pawn).

It is not because I am so strong but because it is easier not to make mistakes when the opponent does not play well.
If the opponent play well I expect myself to do more mistakes.

Uri

"no significant mistakes based on computer analysis" is meaningless when we are talking about PERFECT computer play. ANY mistake will be significant there. This extrapolation about what happens today is meaningless when we talk about a perfect chess opponent.

Some extrapolations are meaningless some are not. That perfect engine is in the range 700-1500 ELO points stronger than Komodo is not meaningless. That only of fraction of that will go to improvement of Knight-odds chess is not meaningless. That this fraction is probably less than 50% is not meaningless.

Some other arguments are not meaningless. That Nakamura said what he said is not meaningless. It seems you promote complete ignorance on a subject where we have lots of empirical data and commons sense.

700-1500 Elo is certainly meaningless. Where does that come from? 20 years ago the assumption was that 2800 was the upper bound on Elo. That seems to have bitten the big banana. The only thing that bounds Elo is that the best player will be hard-pressed to get more than 800 above the second-best player. But then the second best can get to 800 below the 3rd.

As a simplistic point it might be interesting to go back and re-read all the chinook stuff to see if any human chess player managed to draw drawn games with any reliability. And of course, checkers is a far simpler game than chess... But we do have a perfect checkers player, so it would be interesting to see of Schaeffer ever ran such an experiment. I did a quick online search and did not find any results produced after the game solution was announced...

[bob]

None of this seems reasonable to me. The percentage of GM errors, for example. A GM makes far FEWER errors against a weak program than against a strong program, but not because he actually makes fewer errors, but because the opponent doesn't notice them and doesn't punish them.

And my calculation takes that fully into account.

How, when you have no idea how many mistakes a GM makes if the opponent is perfect and sees every one?

[syzygy]

None of this seems reasonable to me. The percentage of GM errors, for example. A GM makes far FEWER errors against a weak program than against a strong program, but not because he actually makes fewer errors, but because the opponent doesn't notice them and doesn't punish them.

And my calculation takes that fully into account.

How, when you have no idea how many mistakes a GM makes if the opponent is perfect and sees every one?

I have explained that. Kai understood it and you might as well if you take some time.

[Jesse Gersenson]

As a simplistic point it might be interesting to go back and re-read all the chinook stuff to see if any human chess player managed to draw drawn games with any reliability. And of course, checkers is a far simpler game than chess... But we do have a perfect checkers player, so it would be interesting to see of Schaeffer ever ran such an experiment. I did a quick online search and did not find any results produced after the game solution was announced...

The great Tinsely drew Chinook reliably. He had a plus score against the machine.

[duncan]

What is fallacy? Where does the 5/3 come from?

the 5/3 comes from here

""Let's say that there is some handicap that would produce an even score in a serious match between Komodo and Magnus Carlsen. I would estimate that this handicap would be in the 1 to 1.5 pawn range based on Komodo's eval after a long think. Let's say 1.25. Now both Komodo and Carlsen make errors of some average magnitude. We'll call Carlsen's error rate C, and Komodo's K. I think it's pretty obvious that K is much less than C, let's say K = .4xC. If a future engine drops the error rate to zero, then C - K increaases to 5/3 of it's former value, so the proper handicap should also increase in that ratio. ""

Who has any idea what a perfect player will be able to do? That's the fallacy as what happens today means no more than what happened 10 or 20 years ago.

(2) depends on the opponent. Against a beginner? 5 moves? Against a master? 15 moves? Against another GM? 30 moves? But all irrelevant when asking about "against a perfect opponent"...

I meant against komodo.

(3) The "9 moves" is pure speculation. That's the problem. How would a USCF 2000 player fare against a top program today? And that is nowhere near the difference between a GM and the perfect player, which is almost certainly thousands of Elo yet to be seen.

let's say a gm can convert a knight handicap against komodo from 3 to 4 in x moves. would a future 60,000 elo computer still have to be able to get the score from 3 to 2 in less than x moves to win.?