Still not enough games, 2 sigma (95%) error bars are about 10% in your results, so things may change. I am surprised at the huge difference between contempt 15 and 0, as, if I understand it, it matters only when Rybka is between 0.00 and -0.15. Can Rybka's play change seriously, as you describe it, by a contempt of 15?M ANSARI wrote:I think things will be pretty clear soon ... I have been doing a 300 game 60 minute +1sec on 3 Quads for almost a week. I am using neutral 7 move book. I am testing N4 against both Rybka 3_default contempt and Rybka 3_0 contempt. So far with 70% of the games played it is clear that R3 is quite a bit stronger ... but it is also seems that R3_0_cont is scoring higher. On the one computer I have at work where I am typing this post this is the score so far
N4_60_1_gaunt 2008
Naum 4 - Rybka 3_no_cont 10.5 - 21.5 +2/-13/=17 32.81%
Naum 4 - Rybka 3 14.5 - 17.5 +6/-9/=17 45.31%
Let me say that the reason I started testing contempt was when I was going through the games that R3 lost ... chesswise it was obvious that R3 was pushing too hard and that in many games it would lose simply because it had overreached. N4 is simply too strong and once R3's initiative evaporated, N4 had enough technique and strength to punish Rybka for speculative play. In many lost games I could not really find an evaluation mistake (although in many there were). This is what led me to believe that contempt was hurting Rybka's performance and I have a feeling this hunch will turn out to be correct.
I will collect all 300 games once the tourney is finished and tabulate them again. Still this is a very good performance for N4 but to test R3 at LTC it seems it will perform better at 0 contempt. By the way I have also tested the same thing at 16_1 time controls and on 30_1 time controls ... the results seem pretty much the same. The only time normal Rybka performed as good as Rybka 0 contempt was when I used Perfect 15 book. But I don't think that was an accurate test since 198 of the 200 games turned out to be B90 (the other two games were B92). Neutral books give a better idea of performance of an engine. I also did a 1000 game 1_1 test on Octa 4ghz using differnt cont settings ... the 30 cont and 15 cont scored almost 90% with lower 0 cont and 8 cont and -8 cont scoring in the 80's%. So contempt has a positive effect on R3 at fast time controls and this seems to decline linearily as TC go higher.
Kai
ps better use an opening set of positions (Noomen 30 maybe, so 60 games), switching black and white, with books my tests are wildly oscillating, and I don't know even how to estimate the error margins.
. But Playchess is not the only part of the chess world, I still think there are many ideas in R3.ctg that are still not well discovered, at least not at Playchess.