Something Hikaru Said

[Laskos]

Actually Naka seems right to me for GMs above ELO 2700, at least against simple 32 men tablebases and aside occasional obvious large blunders by GM.

It probably depends on the time control. At a one minute game even Hikaru would probably lose today. At game in two hours plus increment it will probably never happen.

Yes, I am talking here always only about 2 hour normal time control.

It may be possible that the GM extremely does not feel well during the game so in a single game everything is possible but I believe that no computer will get even a fide rating of 2400 in the future against humans if fide allow computers to play without knight b1 for rating(time control should be at least 90+30).

I am not sure even if 2200 is possible.

No, I believe 2200 will be possible in say 10 years. This is based on the more or less reliable estimates of today's Komodo play with such handicap (~2000). I think the perfect engine (32 men tablebases) can achieve ~2500 FIDE at Knight odds, I have such an estimate, but it's not reliable. And there are many kinds of very good engines, say a very strong engine fooling humans into traps and such, engine which will lose to perfect engine but achieve higher FIDE rating at Knight odds than the perfect engine, at least until human players learn how to play against it.

[Nordlandia]

Perhaps +2.75 eval is the magical sweet spot for IM level. Also known as | Double Exchange Odds. (note castling rights is disabled)

[d]1nbqkbn1/pppppppp/8/8/8/8/PPPPPPPP/RN1QK1NR w - - 0 1

Analysis by SugaR 2.0 64 POPCNT:

1.d4 c5
+- (2.71 --) Depth: 29/39 00:03:58 552MN
(, 10.01.2016)

Analysis by Komodo 9.3 64-bit:

1.d4 g6 2.c4
+- (2.94 ++) Depth: 24 00:01:17 145MN
(, 10.01.2016)

Analysis by Stockfish 090116 64 POPCNT:

1.c4 c5 2.Nc3 Nf6 3.h4 Nc6

+- (2.65) Depth: 24/33 00:01:03 146MN
(, 10.01.2016)

Average evaluation = 2.767 ~ 2.75 points.

New game
[d]rn1qk1nr/pppppppp/8/8/8/8/PPPPPPPP/1NBQKBN1 w - - 0 1

Analysis by SugaR 2.0 64 POPCNT:

1.Nf3
-+ (-2.42 ++) Depth: 28/35 00:01:53 261MN
(, 10.01.2016)

Analysis by Stockfish 090116 64 POPCNT:

1.e3 Nf6 2.d4 c5 3.a3 d6 4.Nf3 Nbd7

-+ (-2.51) Depth: 24/33 00:00:32 78683kN
(, 10.01.2016)


Analysis by Komodo 9.3 64-bit:

1.c4 c5 2.Nf3 Nc6

-+ (-2.60) Depth: 25 00:01:16 162MN
(, 10.01.2016)

2.766666666666667 + 2.51

~ 2.65 assessment from both sides averaged, - depending on starting player.

[Uri Blass]

Actually Naka seems right to me for GMs above ELO 2700, at least against simple 32 men tablebases and aside occasional obvious large blunders by GM.

It probably depends on the time control. At a one minute game even Hikaru would probably lose today. At game in two hours plus increment it will probably never happen.

Yes, I am talking here always only about 2 hour normal time control.

It may be possible that the GM extremely does not feel well during the game so in a single game everything is possible but I believe that no computer will get even a fide rating of 2400 in the future against humans if fide allow computers to play without knight b1 for rating(time control should be at least 90+30).

I am not sure even if 2200 is possible.

No, I believe 2200 will be possible in say 10 years. This is based on the more or less reliable estimates of today's Komodo play with such handicap (~2000). I think the perfect engine (32 men tablebases) can achieve ~2500 FIDE at Knight odds, I have such an estimate, but it's not reliable. And there are many kinds of very good engines, say a very strong engine fooling humans into traps and such, engine which will lose to perfect engine but achieve higher FIDE rating at Knight odds than the perfect engine, at least until human players learn how to play against it.

I think that it is easier to improve rating in normal chess and not rating without a knight.

Get komodo without a knight and some significantly weaker engine without a knight and give them a ccrl rating based on games and I expect that in most cases the gap in rating is going to be smaller than the rating gap between them in normal chess.

The gap is going to be smaller mainly in the case that the weak engine has some positional knowledge that you should go for complications when you are knight down and based on what I read if I remember correctly I guess that gaviota is a good candidate.

[bob]

In an easily won position a top GM should certainly be able to play perfectly. I believe that knight odds is sufficient for a position to be "easily won" in this sense.

if a computer playing a top gm at knight odds can get the score down from 3 to 2 before the super gm can get it up from 3 to 4, the computer will win. currently a computer cannot do that. how do you know it will never be able to do that. ?

Engine scores are not physical measurements that can be forced up or down simply by throwing enough computer cycles against them.

If a position is won, you don't even have to play any moves to get the score up (from you point of view). Just let the opponent computer think long enough and its score will eventually drop to mate. Making the computer stronger will only make it realise more quickly how hopelessly lost it is.

So if in a particular position the computer today cannot get the score down from 3 to 2, then a future computer in that same position will not be able to get the score down from 23 to 22.

99% of the time the problem is the HUMAN will get the score down from 3 to 2, then from 2 to 1, and eventually to lost. Why do you assume perfect play by the human?

[bob]

Engine scores are not physical measurements that can be forced up or down simply by throwing enough computer cycles against them.

If a position is won, you don't even have to play any moves to get the score up (from you point of view). Just let the opponent computer think long enough and its score will eventually drop to mate. Making the computer stronger will only make it realise more quickly how hopelessly lost it is.

So if in a particular position the computer today cannot get the score down from 3 to 2, then a future computer in that same position will not be able to get the score down from 23 to 22.

a computer is not told to evaluate the truth of a chess position and resign when hopeless. It is or can be told to equalise or at least minimise the difference where possible by relying on human error. the computer of the future will find many more human errors than the computer of today. hence many more opportunities to equalise. these many more opportunities many not be enough to overcome a knight in the hands of a super gm, but otoh it may be sufficient.

I'm not sure about the use of "many" above. When a human player fails to play the best move (which itself is hard to define when there is more than one optimal move), I think the odds are pretty good that today's Komodo will identify the error. There is room for improvement, but maybe not so much.
I would look at the problem this way. Let's say that there is some handicap that would produce an even score in a serious match between Komodo and Magnus Carlsen. I would estimate that this handicap would be in the 1 to 1.5 pawn range based on Komodo's eval after a long think. Let's say 1.25. Now both Komodo and Carlsen make errors of some average magnitude. We'll call Carlsen's error rate C, and Komodo's K. I think it's pretty obvious that K is much less than C, let's say K = .4xC. If a future engine drops the error rate to zero, then C - K increaases to 5/3 of it's former value, so the proper handicap should also increase in that ratio. That would put it at 208, a bit over two pawns but way below the roughly 3.5 value of knight odds. Of course there is a lot of uncertainty in the above, but I don't think the estimate would be way off. In order for the estimate to reach knight odds, the K/C ratio would have to be about 64%, which does not seem plausible to me. Probably this ratio can be estimated more precisely by comparing the error rates of Komodo with some engine about the level of Carlsen (maybe the pocket Hiarcs version that earned a 2900+ performance in 2009) as measured by the moves chosen by a "referee" such as Stockfish with 100s as much time. But that's a big project.

Larry:

Here's the obvious question that I suspect I know the answer to. Do you go over Komodo games and find places where it finds the right (or a better) move given longer than it had in the game? I have always seen this happen. Not so often nowadays as it was 20 years ago, but I still see it. And so long as that happens, it would seem (to me) there is room for significant progress. And this assumes we know what the correct move is in the first place and just wait long enough for the program to find it. What if WE are wrong?

[bob]

Someone should take a set of 1,000,000 GM games, and first locate positions where they are a piece ahead, and then analyze to see if they "played perfectly". I remain unconvinced after watching many of the old zonal and inter-zonal events were I watched GMs drop pieces and even queens on occasion. That's not perfect chess. I'm not drinking this "perfect play when a piece ahead" cool-aid without some sort of supporting evidence. "I believe" or "I think" is not evidence. If they can play perfectly when a piece ahead, they can play perfectly when material is even.

I'm fairly certain even you think a GM can win against a king and pawn with a full set of pieces, so somewhere there is a hard limit on what kind of odds a computer can give and still have a chance. Hikaru, and many of us, think that knight odds is beyond the limit. You apparently think it may be substantially more. Let's agree to disagree.

We are not talking about today, however. We are talking about way in the future. 20 years ago a GM could give pawn odds and win easily. 10 years ago it was give away nothing and come close to breaking even. Today we are nearly at the point of giving up a pawn is not enough to cause a program loss. What happens over the next 10 years, the next 100 years? Humans are getting NO stronger, but the hardware and software improvements continue to push computer ratings higher and higher.

I don't know whether the final equalizer will be more or less than a knight, which is the point. But I CERTAINLY am not willing to stipulate that a knight is beyond the upper bound. Time will tell, so there is little point in debating how many angels can dance on the head of a pin today.

[lkaufman]

Engine scores are not physical measurements that can be forced up or down simply by throwing enough computer cycles against them.

If a position is won, you don't even have to play any moves to get the score up (from you point of view). Just let the opponent computer think long enough and its score will eventually drop to mate. Making the computer stronger will only make it realise more quickly how hopelessly lost it is.

So if in a particular position the computer today cannot get the score down from 3 to 2, then a future computer in that same position will not be able to get the score down from 23 to 22.

a computer is not told to evaluate the truth of a chess position and resign when hopeless. It is or can be told to equalise or at least minimise the difference where possible by relying on human error. the computer of the future will find many more human errors than the computer of today. hence many more opportunities to equalise. these many more opportunities many not be enough to overcome a knight in the hands of a super gm, but otoh it may be sufficient.

I'm not sure about the use of "many" above. When a human player fails to play the best move (which itself is hard to define when there is more than one optimal move), I think the odds are pretty good that today's Komodo will identify the error. There is room for improvement, but maybe not so much.
I would look at the problem this way. Let's say that there is some handicap that would produce an even score in a serious match between Komodo and Magnus Carlsen. I would estimate that this handicap would be in the 1 to 1.5 pawn range based on Komodo's eval after a long think. Let's say 1.25. Now both Komodo and Carlsen make errors of some average magnitude. We'll call Carlsen's error rate C, and Komodo's K. I think it's pretty obvious that K is much less than C, let's say K = .4xC. If a future engine drops the error rate to zero, then C - K increaases to 5/3 of it's former value, so the proper handicap should also increase in that ratio. That would put it at 208, a bit over two pawns but way below the roughly 3.5 value of knight odds. Of course there is a lot of uncertainty in the above, but I don't think the estimate would be way off. In order for the estimate to reach knight odds, the K/C ratio would have to be about 64%, which does not seem plausible to me. Probably this ratio can be estimated more precisely by comparing the error rates of Komodo with some engine about the level of Carlsen (maybe the pocket Hiarcs version that earned a 2900+ performance in 2009) as measured by the moves chosen by a "referee" such as Stockfish with 100s as much time. But that's a big project.

Larry:

Here's the obvious question that I suspect I know the answer to. Do you go over Komodo games and find places where it finds the right (or a better) move given longer than it had in the game? I have always seen this happen. Not so often nowadays as it was 20 years ago, but I still see it. And so long as that happens, it would seem (to me) there is room for significant progress. And this assumes we know what the correct move is in the first place and just wait long enough for the program to find it. What if WE are wrong?

Certainly Komodo sometimes finds better moves with more time, and therefore certainly it can improve. But I think the situation is somewhat like the infinite series 1, 1.5,1.75, 1.875, 1.9375...approaching 2. In this case 1 might be the handicap we can give Carlsen now (let's say a pawn to keep it simple). Maybe in two years we can give him 1.5 pawns, then in two more years 1.75 pawns, then 1.875 pawns. The handicap will grow indefinitely, but will never exceed 2 pawns. Of course I'm just giving my opinion on how fast the series will converge; it is of course possible that it will reach 3.5 (knight odds) some day. But my feeling is that Komodo at tournament level is roughly one pawn away from perfect play, which would mean that we might eventually be able to give Carlsen two pawns or a bit more, but not a knight. There are tests we could run to try to confirm this, but it would detract too much from Komodo to do this. I think if you ask many grandmasters who use engines regularly, they would mostlly agree with me (and with Hikaru) on this. But we could all be wrong.

[bob]

Engine scores are not physical measurements that can be forced up or down simply by throwing enough computer cycles against them.

If a position is won, you don't even have to play any moves to get the score up (from you point of view). Just let the opponent computer think long enough and its score will eventually drop to mate. Making the computer stronger will only make it realise more quickly how hopelessly lost it is.

So if in a particular position the computer today cannot get the score down from 3 to 2, then a future computer in that same position will not be able to get the score down from 23 to 22.

a computer is not told to evaluate the truth of a chess position and resign when hopeless. It is or can be told to equalise or at least minimise the difference where possible by relying on human error. the computer of the future will find many more human errors than the computer of today. hence many more opportunities to equalise. these many more opportunities many not be enough to overcome a knight in the hands of a super gm, but otoh it may be sufficient.

I'm not sure about the use of "many" above. When a human player fails to play the best move (which itself is hard to define when there is more than one optimal move), I think the odds are pretty good that today's Komodo will identify the error. There is room for improvement, but maybe not so much.
I would look at the problem this way. Let's say that there is some handicap that would produce an even score in a serious match between Komodo and Magnus Carlsen. I would estimate that this handicap would be in the 1 to 1.5 pawn range based on Komodo's eval after a long think. Let's say 1.25. Now both Komodo and Carlsen make errors of some average magnitude. We'll call Carlsen's error rate C, and Komodo's K. I think it's pretty obvious that K is much less than C, let's say K = .4xC. If a future engine drops the error rate to zero, then C - K increaases to 5/3 of it's former value, so the proper handicap should also increase in that ratio. That would put it at 208, a bit over two pawns but way below the roughly 3.5 value of knight odds. Of course there is a lot of uncertainty in the above, but I don't think the estimate would be way off. In order for the estimate to reach knight odds, the K/C ratio would have to be about 64%, which does not seem plausible to me. Probably this ratio can be estimated more precisely by comparing the error rates of Komodo with some engine about the level of Carlsen (maybe the pocket Hiarcs version that earned a 2900+ performance in 2009) as measured by the moves chosen by a "referee" such as Stockfish with 100s as much time. But that's a big project.

Larry:

Here's the obvious question that I suspect I know the answer to. Do you go over Komodo games and find places where it finds the right (or a better) move given longer than it had in the game? I have always seen this happen. Not so often nowadays as it was 20 years ago, but I still see it. And so long as that happens, it would seem (to me) there is room for significant progress. And this assumes we know what the correct move is in the first place and just wait long enough for the program to find it. What if WE are wrong?

Certainly Komodo sometimes finds better moves with more time, and therefore certainly it can improve. But I think the situation is somewhat like the infinite series 1, 1.5,1.75, 1.875, 1.9375...approaching 2. In this case 1 might be the handicap we can give Carlsen now (let's say a pawn to keep it simple). Maybe in two years we can give him 1.5 pawns, then in two more years 1.75 pawns, then 1.875 pawns. The handicap will grow indefinitely, but will never exceed 2 pawns. Of course I'm just giving my opinion on how fast the series will converge; it is of course possible that it will reach 3.5 (knight odds) some day. But my feeling is that Komodo at tournament level is roughly one pawn away from perfect play, which would mean that we might eventually be able to give Carlsen two pawns or a bit more, but not a knight. There are tests we could run to try to confirm this, but it would detract too much from Komodo to do this. I think if you ask many grandmasters who use engines regularly, they would mostlly agree with me (and with Hikaru) on this. But we could all be wrong.

My problem with this is that is is (a) based on speculation; which is (b) based on what programs can do today or within the next few years. The original question concerned the impossible case of having 32 piece EGTBs available, something I don't believe will ever be possible. But if it were possible, I don't see anything that suggests that 2.0 or 3.0 is an absolute upper bound on this question. I'm not suggesting that knight odds WILL be surmountable for a computer. I've only said that I don't see anything that says a computer can't win with even larger handicaps, at some point in the future... IE at 100M nodes per second we know what they can do, and 1 giga-node per second is imaginable. But what about one tera-node per second? one peta-node? That's just a paltry 10^7 more than we can do today. But we are 10^8 nodes faster than my selective search program in 1970 that was doing one node per second...

[Laskos]

Actually Naka seems right to me for GMs above ELO 2700, at least against simple 32 men tablebases and aside occasional obvious large blunders by GM.

It probably depends on the time control. At a one minute game even Hikaru would probably lose today. At game in two hours plus increment it will probably never happen.

Yes, I am talking here always only about 2 hour normal time control.

It may be possible that the GM extremely does not feel well during the game so in a single game everything is possible but I believe that no computer will get even a fide rating of 2400 in the future against humans if fide allow computers to play without knight b1 for rating(time control should be at least 90+30).

I am not sure even if 2200 is possible.

No, I believe 2200 will be possible in say 10 years. This is based on the more or less reliable estimates of today's Komodo play with such handicap (~2000). I think the perfect engine (32 men tablebases) can achieve ~2500 FIDE at Knight odds, I have such an estimate, but it's not reliable. And there are many kinds of very good engines, say a very strong engine fooling humans into traps and such, engine which will lose to perfect engine but achieve higher FIDE rating at Knight odds than the perfect engine, at least until human players learn how to play against it.

I think that it is easier to improve rating in normal chess and not rating without a knight.

Get komodo without a knight and some significantly weaker engine without a knight and give them a ccrl rating based on games and I expect that in most cases the gap in rating is going to be smaller than the rating gap between them in normal chess.

The gap is going to be smaller mainly in the case that the weak engine has some positional knowledge that you should go for complications when you are knight down and based on what I read if I remember correctly I guess that gaviota is a good candidate.

Yes, the ELO gap is much smaller in Knight odds chess than in normal chess for same engines, and decreasing with strength.

[lkaufman]

Engine scores are not physical measurements that can be forced up or down simply by throwing enough computer cycles against them.

If a position is won, you don't even have to play any moves to get the score up (from you point of view). Just let the opponent computer think long enough and its score will eventually drop to mate. Making the computer stronger will only make it realise more quickly how hopelessly lost it is.

So if in a particular position the computer today cannot get the score down from 3 to 2, then a future computer in that same position will not be able to get the score down from 23 to 22.

a computer is not told to evaluate the truth of a chess position and resign when hopeless. It is or can be told to equalise or at least minimise the difference where possible by relying on human error. the computer of the future will find many more human errors than the computer of today. hence many more opportunities to equalise. these many more opportunities many not be enough to overcome a knight in the hands of a super gm, but otoh it may be sufficient.

I'm not sure about the use of "many" above. When a human player fails to play the best move (which itself is hard to define when there is more than one optimal move), I think the odds are pretty good that today's Komodo will identify the error. There is room for improvement, but maybe not so much.
I would look at the problem this way. Let's say that there is some handicap that would produce an even score in a serious match between Komodo and Magnus Carlsen. I would estimate that this handicap would be in the 1 to 1.5 pawn range based on Komodo's eval after a long think. Let's say 1.25. Now both Komodo and Carlsen make errors of some average magnitude. We'll call Carlsen's error rate C, and Komodo's K. I think it's pretty obvious that K is much less than C, let's say K = .4xC. If a future engine drops the error rate to zero, then C - K increaases to 5/3 of it's former value, so the proper handicap should also increase in that ratio. That would put it at 208, a bit over two pawns but way below the roughly 3.5 value of knight odds. Of course there is a lot of uncertainty in the above, but I don't think the estimate would be way off. In order for the estimate to reach knight odds, the K/C ratio would have to be about 64%, which does not seem plausible to me. Probably this ratio can be estimated more precisely by comparing the error rates of Komodo with some engine about the level of Carlsen (maybe the pocket Hiarcs version that earned a 2900+ performance in 2009) as measured by the moves chosen by a "referee" such as Stockfish with 100s as much time. But that's a big project.

Larry:

Here's the obvious question that I suspect I know the answer to. Do you go over Komodo games and find places where it finds the right (or a better) move given longer than it had in the game? I have always seen this happen. Not so often nowadays as it was 20 years ago, but I still see it. And so long as that happens, it would seem (to me) there is room for significant progress. And this assumes we know what the correct move is in the first place and just wait long enough for the program to find it. What if WE are wrong?

Certainly Komodo sometimes finds better moves with more time, and therefore certainly it can improve. But I think the situation is somewhat like the infinite series 1, 1.5,1.75, 1.875, 1.9375...approaching 2. In this case 1 might be the handicap we can give Carlsen now (let's say a pawn to keep it simple). Maybe in two years we can give him 1.5 pawns, then in two more years 1.75 pawns, then 1.875 pawns. The handicap will grow indefinitely, but will never exceed 2 pawns. Of course I'm just giving my opinion on how fast the series will converge; it is of course possible that it will reach 3.5 (knight odds) some day. But my feeling is that Komodo at tournament level is roughly one pawn away from perfect play, which would mean that we might eventually be able to give Carlsen two pawns or a bit more, but not a knight. There are tests we could run to try to confirm this, but it would detract too much from Komodo to do this. I think if you ask many grandmasters who use engines regularly, they would mostlly agree with me (and with Hikaru) on this. But we could all be wrong.

My problem with this is that is is (a) based on speculation; which is (b) based on what programs can do today or within the next few years. The original question concerned the impossible case of having 32 piece EGTBs available, something I don't believe will ever be possible. But if it were possible, I don't see anything that suggests that 2.0 or 3.0 is an absolute upper bound on this question. I'm not suggesting that knight odds WILL be surmountable for a computer. I've only said that I don't see anything that says a computer can't win with even larger handicaps, at some point in the future... IE at 100M nodes per second we know what they can do, and 1 giga-node per second is imaginable. But what about one tera-node per second? one peta-node? That's just a paltry 10^7 more than we can do today. But we are 10^8 nodes faster than my selective search program in 1970 that was doing one node per second...

Each extra doubling of speed (or each extra ply, which is similar) brings real but diminishing returns in elo due to the inherent drawishness of the initial position. If we start with a clearly won position like knight odds, we also get diminishing returns with increased speed/depth. An expert mathematician could probably extrapolate from data to estimate the elo gain that infinite speed (or 32 man tb) would obtain; in fact I think Kai did this to estimate about 800 elo remaining to perfect play. This would equate to adding another pawn and a fraction to a handicap, which gives an estimate of between 2 and 3 pawns as the proper handicap between a 2800 human and perfect play. You'll have to argue the accuracy of this extrapolation with Kai or another mathematician; it's beyond my ability to do this properly.