Can Komodo give knight odds in standard chess to a master?

[Lyudmil Tsvetkov]

In view of the match loss, I ran a test to see if Komodo can give knight odds to a decent engine. I chose Fritz 11 running on 1 core, with Komodo able to use all 4 cores on my laptop. Time limit was 15 min plus 5 seconds, one third of the human match level but still much slower than blitz. Contempt set at 100 (I used 150 vs. Gilden). Alternating knight handicap.

Fritz 11 on 1 pcu is rated 2853 on the CCRL 40/40 list, which actually seems a bit low since it is several versions improved from the Fritz that defeated then-World Champion Kramnik in a match. I would bet on Fritz 11 1 cpu in a match with Magnus Carlsen, though it might be close.

Much to my surprise, Komodo (same version that played Gilden) won by 13 to 7 (12 wins, 6 losses, 2 draws). So in rapid chess, Komodo can decisively defeat a Magnus Carlsen level engine giving it knight odds, but lost all three games to a nearly 75 year old player with a FIDE rating of 2114. Pretty amazing difference. Perhaps Fritz 11 doesn't know to simplify when ahead, I don't know about that, but this surely isn't the whole answer. In most of the winning games Komodo whipped up a strong attack on the king, something that was never even a dream against Gilden.

Must be something to learn from this.

15 min. is certainly way shorter than 45 min., but I wonder why you allocated 4 cores to Komodo and only a single one to Fritz?

I bet, even with shorter TC, 2 min. per game, but equal conditions, equal cores, Fritz will beat Komodo.

Probably so, but my Fritz 11 only runs on 1 core, and there are good reasons to think this is enough for it to win a match from Magnus Carlsen. The relevant CCRL rating for it is on one core. I could just as well have picked a four core engine with a similar rating if I had one installed.

Sure, I could have run Komodo on one core only and probably it would have lost the match. The point was to show that Komodo, running on a normal laptop, could give knight odds in a rapid game to a Carlsen-rated engine.

don't you also have Komodo running on 1 core?

But Fritz beat Kramnik on 8 cores or so, and not a single core, so if you want to check how stronger is Komodo than the engine that played Kramnik, then you should use 8 cores.

apart from that, Carlsen is much stronger than Kramnik.

and a 3-rd clue: no one knows how Carlsen plays against computers: his play is very much reminiscent of a computer one, in any case.

The Fritz that beat kramnik is clearly weaker than Fritz11 1 core.

not at all.

Fritz in Bahrain was Frtiz 8 +, a version between Fritz 8 and Fritz 9, so we can suppose 2.5 versions distance between Fritz Kramnik and Fritz 11.

that would make a bit over 100 elo increase(sometimes, Fritz updated versions with even less than 50 poinrs)

on the other hand, Fritz scales excellently with multiple cores, probably because of its heavy reliance on search, so one doubling in Fritz is know to be worth more than 70 points.

3 doublings would get it to more than 200 elo increase.

so, Fritz in Bahrain is at least 100 elo stronger than Larry's Fritz 11.

besides, Fritz in Bahrain used a wide opening book, which very much additionally skewed the result. (without that book, quite clearly it might also have lost)

[mehmet karaman]

Fritz in Bahrain was Frtiz 8 +, a version between Fritz 8 and Fritz 9, so we can suppose 2.5 versions distance between Fritz Kramnik and Fritz 11.

Release Dates
Fritz 7 : 2001-07
Fritz Bahrain : 2002-11
Deep Fritz 7.0 : 2003
Fritz 8 : 2003-9

https://chessprogramming.wikispaces.com/Fritz

[Lyudmil Tsvetkov]

Fritz in Bahrain was Frtiz 8 +, a version between Fritz 8 and Fritz 9, so we can suppose 2.5 versions distance between Fritz Kramnik and Fritz 11.

Release Dates
Fritz 7 : 2001-07
Fritz Bahrain : 2002-11
Deep Fritz 7.0 : 2003
Fritz 8 : 2003-9

https://chessprogramming.wikispaces.com/Fritz

yes, you are right, Mehmet.

thanks for the correction.

we tend to forget a lot about Fritz in these times of SF.

I checked again, and, as Kramnik played 2 matches vs Fritz, the first one was in 2002 against Fritz in Bahrain, which he drew, and the second one in 2006, against Fritz-Bonn, which he lost by 2/4.

Fritz-Bonn, as some sites would claim, was Deep Fritz 10.

but that only comes to even more strongly support my point, as Fritz-Bonn and Fritz 11 are, seemingly, only separated by a single version.

that would make some 50 elo difference in favour of Fritz 11, when you add the 200 elo lead in term of multiple cores, and another 200 elo because of the wide available book, especially useful against humans, my maths does it at Fritz-Bonn being some 350 elo stronger than Fritz 11.

so, in that case, Larry's Fritz-Komodo test might be well off target, and my estimate is that Carlsen has good winning chances against latest Komodo, playing without book.

[Modern Times]

when you add the 200 elo lead in term of multiple cores.

That seems optimistic to me.

[lkaufman]

Fritz in Bahrain was Frtiz 8 +, a version between Fritz 8 and Fritz 9, so we can suppose 2.5 versions distance between Fritz Kramnik and Fritz 11.

Release Dates
Fritz 7 : 2001-07
Fritz Bahrain : 2002-11
Deep Fritz 7.0 : 2003
Fritz 8 : 2003-9

https://chessprogramming.wikispaces.com/Fritz

yes, you are right, Mehmet.

thanks for the correction.

we tend to forget a lot about Fritz in these times of SF.

I checked again, and, as Kramnik played 2 matches vs Fritz, the first one was in 2002 against Fritz in Bahrain, which he drew, and the second one in 2006, against Fritz-Bonn, which he lost by 2/4.

Fritz-Bonn, as some sites would claim, was Deep Fritz 10.

but that only comes to even more strongly support my point, as Fritz-Bonn and Fritz 11 are, seemingly, only separated by a single version.

that would make some 50 elo difference in favour of Fritz 11, when you add the 200 elo lead in term of multiple cores, and another 200 elo because of the wide available book, especially useful against humans, my maths does it at Fritz-Bonn being some 350 elo stronger than Fritz 11.

so, in that case, Larry's Fritz-Komodo test might be well off target, and my estimate is that Carlsen has good winning chances against latest Komodo, playing without book.

On the CCRL 40/40 list, Fritz 11 single core is rate twenty elo HIGHER than Fritz 10 on four cores. I'm not sure they could really benefit from 8 cores back then, but if they did use eight in that match it would probably have made it just twenty to thirty elo above Fritz 11 at most. But Kramnik lost by 120 elo (true, tiny sample). Kramnik is only rated about 25 elo below Carlsen now, and then he was World Champion. As for the book, Kramnik got the right to look at the book during the games (!!), so it may have been better to use a tiny book that would not help him. I'm not saying that Fritz 11 would beat Carlsen now in a match, only that there would be no obvious favorite between them. The ratings on the CCRL 40/40 list for machines that have played serious matches with top players look fairly conservative to me, so there is no reason to think that Fritz 11 on one core is overrated at 2853, which is well above Carlsen now.

In case you forgot, Komodo beat GM Sergei Erenburg by 3.5 to 0.5 with almost no book (3 moves), despite 30 to 1 ratio time odds, playing on single core only, playing Black every game, not thinking on his time, and no tablebases. Erenberg is no Carlsen, about 250 elo separate them, but all these other handicaps add up to much more than that. I think the Erenburg conditions would make for a tossup match with Carlsen.

[Modern Times]

Deep Fritz 10 also scaled very poorly, even on just 4 threads. That was fixed in the 10.1 release.

[Lyudmil Tsvetkov]

Fritz in Bahrain was Frtiz 8 +, a version between Fritz 8 and Fritz 9, so we can suppose 2.5 versions distance between Fritz Kramnik and Fritz 11.

Release Dates
Fritz 7 : 2001-07
Fritz Bahrain : 2002-11
Deep Fritz 7.0 : 2003
Fritz 8 : 2003-9

https://chessprogramming.wikispaces.com/Fritz

yes, you are right, Mehmet.

thanks for the correction.

we tend to forget a lot about Fritz in these times of SF.

I checked again, and, as Kramnik played 2 matches vs Fritz, the first one was in 2002 against Fritz in Bahrain, which he drew, and the second one in 2006, against Fritz-Bonn, which he lost by 2/4.

Fritz-Bonn, as some sites would claim, was Deep Fritz 10.

but that only comes to even more strongly support my point, as Fritz-Bonn and Fritz 11 are, seemingly, only separated by a single version.

that would make some 50 elo difference in favour of Fritz 11, when you add the 200 elo lead in term of multiple cores, and another 200 elo because of the wide available book, especially useful against humans, my maths does it at Fritz-Bonn being some 350 elo stronger than Fritz 11.

so, in that case, Larry's Fritz-Komodo test might be well off target, and my estimate is that Carlsen has good winning chances against latest Komodo, playing without book.

On the CCRL 40/40 list, Fritz 11 single core is rate twenty elo HIGHER than Fritz 10 on four cores. I'm not sure they could really benefit from 8 cores back then, but if they did use eight in that match it would probably have made it just twenty to thirty elo above Fritz 11 at most. But Kramnik lost by 120 elo (true, tiny sample). Kramnik is only rated about 25 elo below Carlsen now, and then he was World Champion. As for the book, Kramnik got the right to look at the book during the games (!!), so it may have been better to use a tiny book that would not help him. I'm not saying that Fritz 11 would beat Carlsen now in a match, only that there would be no obvious favorite between them. The ratings on the CCRL 40/40 list for machines that have played serious matches with top players look fairly conservative to me, so there is no reason to think that Fritz 11 on one core is overrated at 2853, which is well above Carlsen now.

In case you forgot, Komodo beat GM Sergei Erenburg by 3.5 to 0.5 with almost no book (3 moves), despite 30 to 1 ratio time odds, playing on single core only, playing Black every game, not thinking on his time, and no tablebases. Erenberg is no Carlsen, about 250 elo separate them, but all these other handicaps add up to much more than that. I think the Erenburg conditions would make for a tossup match with Carlsen.

I see nothing like that.

when I check the full 40/40 list, I see Fritz 11 single core is some 74 elo higher rated than Fritz 10 single core, so maybe they made bigger gains between Fritz 10 and Fritz 11.

still, when you add 3 doublings, 3*70=210 , 210 - 74= 136 elo difference in strength in favour of Fritz 10 8 cores.(as far as I remember, they used the best hardware back then)

the book should have given another 200 elo at least.

please note, that books have quite different significance for Komodo and Fritz 10.

Komodo might well still do decently without a book, due to its sufficient knowledge, but not Fritz 10.

so, the available book has been a major boost for Fritz.

so, my estimate is Fritz 10 on a single core and without book would have performed somewhere around 2600 elo.

whether Fritz 11 at 2675 elo would beat Carlsen is quite debatable, for me it is obvious Fritz lags behind by some 200 elo points behind Carlsen.

only reason Kramnik performed relatively well vs Carlsen is he played most of their games at the early Carlsen period; as of late, Carlsen has been winning almost all of their encounters, as well as those vs Anand, so I guess he is stronger than both of them by at least 100 elo or so.

for those matches you need no Erenburg, you need good positional players, like Carlsen and Gilden.

[Uri Blass]

when you add the 200 elo lead in term of multiple cores.

That seems optimistic to me.

I totally agree but it is not important the discussion about the exact level of Fritz.

It is clear that Fritz11(1 core) does not know to play with knight advantage

1)With all the pieces it is obviously play at level of more than 2500 against humans even if you give it 15+5 time control and no pondering.

2)With knight advantage it cannot even do what humans with fide rating near 2100 can do and beat komodo.

It is obvious that Fritz11 has a huge weakness in its evaluation and the question is if people can take a weak engine like tscp(1698 ccrl rating) and change its evaluation so it can beat komodo when komodo start without a knight.

It will be nice to see some brute force engine that does not do null move pruning or LMR beat Komodo without a knight.

It is interesting if Komodo with the same evaluation without pruning that is probably significantly stronger than tscp(but I guess weaker than Fritz11) can do it.

[Lyudmil Tsvetkov]

when you add the 200 elo lead in term of multiple cores.

That seems optimistic to me.

I totally agree but it is not important the discussion about the exact level of Fritz.

It is clear that Fritz11(1 core) does not know to play with knight advantage

1)With all the pieces it is obviously play at level of more than 2500 against humans even if you give it 15+5 time control and no pondering.

2)With knight advantage it cannot even do what humans with fide rating near 2100 can do and beat komodo.

It is obvious that Fritz11 has a huge weakness in its evaluation and the question is if people can take a weak engine like tscp(1698 ccrl rating) and change its evaluation so it can beat komodo when komodo start without a knight.

It will be nice to see some brute force engine that does not do null move pruning or LMR beat Komodo without a knight.

It is interesting if Komodo with the same evaluation without pruning that is probably significantly stronger than tscp(but I guess weaker than Fritz11) can do it.

no book here, and Fritz does not know how to play without book, in distinction to Komodo.

for example, Fritz would still choose the Scandinavian on e4.

if Fritz played Kramnik without a book and Kramnik played 1.e4, then maybe Fritz would have played 1...d5, and Kramnik would have gotten very large advantage.

so, my claim Fritz gained a lot by its book is realistic, I think.

Komodo on 4 cores might have indeed won a match vs Fritz 11 by a margin, but that was without books, so probably Komodo on 1 core will manage to post the same score against an engine of, say, 2500 elo strength.

2500 elo is not that distant from Gilden's own strength, so I don't find Fritz underperforming by that much.

certainly, it underperforms, but not over-significantly.

please, don't get me wrong, I simply can not accept the fact Komodo is able to win at knight odds against a 2800-elo engine...

that seems completely incomprehensible to me, but I have never run such a test, so I can not claim with certainty this would not happen.

[Guenther]

2500 elo is not that distant from Gilden's own strength, so I don't find Fritz underperforming by that much.

certainly, it underperforms, but not over-significantly.

?? FM Larry Gilden, 2114 FIDE

You missread sth in this thread (you also once wrote GM Gilden strangely)...