Next Komodo vs. GM handicap match - "Standard chess&quo

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

How will Komodo score in 4 handicap games with Erenburg?

Poll ended at Fri Jun 24, 2016 10:33 pm

0.5 or less
1
6%
1 point
1
6%
1.5 points.
1
6%
2 points.
3
18%
2.5 points.
2
12%
3 points.
9
53%
 
Total votes: 17

Uri Blass
Posts: 10790
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Next Komodo vs. GM handicap match - "Standard chess

Post by Uri Blass »

lkaufman wrote:In view of the result, I decided to do a test to determine what engine would be a fair match for Komodo 1660 (dev. version used in the match, roughly plus ten elo from Komodo 10) at the given time odds, both using single core and no ponder. So 90' plus 30" to 3' plus 1" odds. After some quick tests to chose a suitable opponent, I settled on Robbolito 8.4, which I think was the best or nearly the best of the Robbolito/Ippolit series, at least on single core. This seemed outrageous to me at first, since when Robbolito came out it was dominant for a while, clearly stronger than Rybka 3 (which it appears to have been based loosely on but not directly cloned), and at that time strong enough that it could give Komodo double time handicap; that's how we tested it then. Well, after 13.4 games Komodo is losing by just 23.6 elo, which is roughly Robbo's edge over Rybka 3 at long tc. So subject to margin of error, it appears that this Komodo is roughly an even match with Rybka 3 giving Rybka 30 to 1 time odds with no ponder.
Now let's look at the match result. Sergey's rating plus the elo value of the 3.5 to 0.5 score is around 2930. Add 40 for White pieces, and 70 (according to Kai) for the 3 move book plus no tablebase rules. That gives 3040. Now look up the single core rating of Rybka 3 on CCRL 40/40 list. it is 3054. So the results agree remarkably well! Math really does work.
Conclusion: The level of ratings used by CCRL 40/40 is quite realistic, at least around the 3000 level.
I do not think that we have enough data to get the conclusions.

4 games are clearly not enough to get a rating against humans and I expect GM's to be better in the opening and not to get inferior positions in less than 20 moves with white.
Uri Blass
Posts: 10790
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Next Komodo vs. GM handicap match - "Standard chess

Post by Uri Blass »

I can add that 39.b4 in the last game is also not a blunder that I expect from a GM to make.

I think that in this position the simplest is Qd3 to keep everything defended.
b4 leave a4 undefended so even without seeing d3
I prefer not to play it if there is an equal alternative.
lkaufman
Posts: 6225
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: Next Komodo vs. GM handicap match - "Standard chess

Post by lkaufman »

Uri Blass wrote:
lkaufman wrote:In view of the result, I decided to do a test to determine what engine would be a fair match for Komodo 1660 (dev. version used in the match, roughly plus ten elo from Komodo 10) at the given time odds, both using single core and no ponder. So 90' plus 30" to 3' plus 1" odds. After some quick tests to chose a suitable opponent, I settled on Robbolito 8.4, which I think was the best or nearly the best of the Robbolito/Ippolit series, at least on single core. This seemed outrageous to me at first, since when Robbolito came out it was dominant for a while, clearly stronger than Rybka 3 (which it appears to have been based loosely on but not directly cloned), and at that time strong enough that it could give Komodo double time handicap; that's how we tested it then. Well, after 13.4 games Komodo is losing by just 23.6 elo, which is roughly Robbo's edge over Rybka 3 at long tc. So subject to margin of error, it appears that this Komodo is roughly an even match with Rybka 3 giving Rybka 30 to 1 time odds with no ponder.
Now let's look at the match result. Sergey's rating plus the elo value of the 3.5 to 0.5 score is around 2930. Add 40 for White pieces, and 70 (according to Kai) for the 3 move book plus no tablebase rules. That gives 3040. Now look up the single core rating of Rybka 3 on CCRL 40/40 list. it is 3054. So the results agree remarkably well! Math really does work.
Conclusion: The level of ratings used by CCRL 40/40 is quite realistic, at least around the 3000 level.
I do not think that we have enough data to get the conclusions.

4 games are clearly not enough to get a rating against humans and I expect GM's to be better in the opening and not to get inferior positions in less than 20 moves with white.
Well of course you are right, 4 games are just enough to show that the ratings aren't off by hundreds of elo points. This is just one more piece of evidence along with the many other human vs engine matches played this century. Meanwhile the result for Komodo against Robbo actually got better at the 30 to 1 handicap; now Komodo is only down by six elo after 176 games.
Komodo rules!
thekingman
Posts: 35
Joined: Mon Mar 16, 2015 6:17 am

Re: Next Komodo vs. GM handicap match - "Standard chess

Post by thekingman »

Great match, thanks for running it! Komodo just proved too strong for the human despite giving the human every possible advantage while still being "normal" chess... except one. Any thoughts on having a similar match with draw odds offered? I bet the human could do significantly better if not trying to win and just trying to survive.
lkaufman
Posts: 6225
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: Next Komodo vs. GM handicap match - "Standard chess

Post by lkaufman »

thekingman wrote:Great match, thanks for running it! Komodo just proved too strong for the human despite giving the human every possible advantage while still being "normal" chess... except one. Any thoughts on having a similar match with draw odds offered? I bet the human could do significantly better if not trying to win and just trying to survive.
I think Erenburg concluded before the match or at least during the first game that he had almost no chance to win a game, and so he was effectively playing as if it were draw odds. The problem with draw odds is that it puts all the emphasis on the opening book or lack thereof. Still, I won't rule it out.
Komodo rules!
nimh
Posts: 46
Joined: Sun Nov 30, 2014 12:06 am

Re: Next Komodo vs. GM handicap match - "Standard chess

Post by nimh »

I've been wondering about one thing - why did Erenburg play e4 only? Was he secially instructed to do so, or was he really optimistic about his abilities to keep up with K10 in open positions?
lkaufman
Posts: 6225
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: Next Komodo vs. GM handicap match - "Standard chess

Post by lkaufman »

nimh wrote:I've been wondering about one thing - why did Erenburg play e4 only? Was he secially instructed to do so, or was he really optimistic about his abilities to keep up with K10 in open positions?
Very simple; he always plays 1e4 in tournaments. So he knows it far better than he knows queenside openings.
Komodo rules!