Sedat Canbaz wrote:And in the previous match,we noticed not very satisfied results by the performance of Komodo 5 (against Houdini 2.0c)
Komodo had a bad start against all 4 engines (Critter, Stockfish, Rybka and Houdini).
Komodo's poor performance was most likely just a string of bad luck. Things will even out with more games - and in fact they already have partially.
There is no need for attributing this to opening choice, to the time control, to the sudden death, or to anything else.
Just plain variability of engine matches will do.
Quite frankly, we are puzzled. Our own distributed tester (mostly independent testers) show Komodo 5 to be 15 elo ahead of Houdini 1.5, after many thousand games. None of the testing agencies have confirmed this though, and we need to find out why. Perhaps it's something about the opening books, or the time controls, or AMD. Only by putting forth these various hypothesis can we expect to find out which is correct. Once we do, we can either
1. modify our tester to better predict what others will get.
or
2. Modify Komodo to fix whatever the problem may be, if it is easily fixable.
And this an interesting game with a very rare constellation: black queen on h8 forces white King on b1 to move his queen to a1! Wow, never seen anything like that.
Great game: After the opening (that gave good chances to both sides) first Houdini played more actively and gathered its pieces for attack. But Komodo defended ingenious and moved its pieces for a strong counter-attack. Best move of this game for me was:
[d]3r2k1/2r2pp1/3b4/3P1Q1q/1pB4P/pP6/P1P2R2/1K5R b - - 0 33
33. ..Qh8!
Sedat Canbaz wrote:Its too early for any conclusions...
Exactly. That's why it's surprising that you already attributed the poor results to the opening positions ("I think Komodo's performance suffers due to critical openings (especially with Blacks)"), instead of simply saying: "bad luck, but results should improve".
Sedat Canbaz wrote:Its too early for any conclusions...
Exactly. That's why it's surprising that you already attributed the poor results to the opening positions ("I think Komodo's performance suffers due to critical openings (especially with Blacks)"), instead of simply saying: "bad luck, but results should improve".
Dear Robert,
No...its not bad luck !
I still believe that Komodo 5's performance was not so good, due to Komodo is played with different openings
Really i dont want to critic Jeroen's work...
Once more i'd like to mention that Jeroen Noomen is really GREAT Master in Book Making
But anyway, it seems Komodo 5's playing strenght suffers (under these conditions) with Noomen Testsuite 2012's openings
Thanks for the clarifications, I now understand that you really suggest that Komodo doesn't play very well the positions from the Noomen Suite.
Let's wait and see what happens in the rest of the 4 matches.
Houdini wrote:Thanks for the clarifications, I now understand that you really suggest that Komodo doesn't play very well the positions from the Noomen Suite.
Let's wait and see what happens in the rest of the 4 matches.
Not at all...
Of course without final results...we can not be sure 100 % that Komodo will do better with Perfect 2012 book
Actually i have much more improved version-Perfect 2012b book, but i think its now too late to resume the current match
lkaufman wrote:Quite frankly, we are puzzled. Our own distributed tester (mostly independent testers) show Komodo 5 to be 15 elo ahead of Houdini 1.5, after many thousand games.
You know how it is and when the randomness monster strikes then it is as Robert said, your turn of being the unlucky one.
Just recently I had such an unlucky experience, an extreme one I must say as I (till now) haven't seen before. I removed a part from King Safety in order to measure its impact either good or bad. And the removal gave a 15 elo improvement after 4000 games. I could not believe it and started testing the removal again but now changing the eval values of King Safety with +1% and -1%, 2 peanuts changes with hardly any effect elo wise. 2 x 4000 games again and both matches showed the wrong of the initial change, instead of a 52.x% score I then got the opposite: two 47-48% results. And thus I almost removed well working code.
lkaufman wrote:Quite frankly, we are puzzled. Our own distributed tester (mostly independent testers) show Komodo 5 to be 15 elo ahead of Houdini 1.5, after many thousand games.
You know how it is and when the randomness monster strikes then it is as Robert said, your turn of being the unlucky one.
Just recently I had such an unlucky experience, an extreme one I must say as I (till now) haven't seen before. I removed a part from King Safety in order to measure its impact either good or bad. And the removal gave a 15 elo improvement after 4000 games. I could not believe it and started testing the removal again but now changing the eval values of King Safety with +1% and -1%, 2 peanuts changes with hardly any effect elo wise. 2 x 4000 games again and both matches showed the wrong of the initial change, instead of a 52.x% score I then got the opposite: two 47-48% results. And thus I almost removed well working code.
Hello dear Ed,
Yes...i am not a programmer and you can be right that with changing some values...your engine can start to play with 15 elo improvement
And from my experience i can say,
Depending on what kind of opening usage,any Engine can be performed between 0-200 Elo difference or sometimes even more
Btw (about the current match/opening issue),
Timo changed only the openings and now we see completely different results by Komodo 5: http://www.team-oh.de/livegames6/
One thing more,
There is no guarantee to see same Komodo 5's results e.g under different conditions (i mean if we change the hardware,the time control...)
