Naum4 - Rybka 3 - 100 games 60 min. test. After 16 games...

[Jeroen]

Here is a result between Rybka 3 and Naum 4 using a testsuite:

http://home.deds.nl/~ejwebspace/tourney/

[GenoM]

what do you like to imply with all these "results"? That Naum equals Rybka in strength?
it isn't even funny anymore

[pichy]

Jajajajajajaja, Jeroen, spirit of revenge?.

My forecast:

Rybka3 + Storm = 62,50%
Naum4 + RybkaII = 37,50%

With no more than 30% draws.

(I assume in a quad and 60 minutes game, with a minimum of 50 games).

Regards form Barcelona.

Tom.

... by the way, I have just ordered your RybkaIII book. Let's see how it behaves against Storm and Magnificient.

I would predict 55% to 45% in favor of Rybka 3 with Rybka 3.book including Draws

[Tomcass]

I do not like to imply anything, Evgenii. Thinks are very simple for me:

1.- I enjoy testing engines. This is my first driver.
2.- I like to try new books.
3.- After weeks testing engines I offer the results with my feelings to this chess community.

That's all. I am not saying that Naum4 equals Rybka3. If you read my forecast to Jeroen's test: Rybka 62,5 - Naum 37,5. How can I make this forecast - by the way, very close to reality! - if I think that they are equal?. I simply have found Naum4 to obtain impressive results UNDER CERTAIN circumstances. And this is what I have shared with our colleagues her.

Regards,

Tom.

[Dr.Wael Deeb]

I do not like to imply anything, Evgenii. Thinks are very simple for me:

1.- I enjoy testing engines. This is my first driver.
2.- I like to try new books.
3.- After weeks testing engines I offer the results with my feelings to this chess community.

That's all. I am not saying that Naum4 equals Rybka3. If you read my forecast to Jeroen's test: Rybka 62,5 - Naum 37,5. How can I make this forecast - by the way, very close to reality! - if I think that they are equal?. I simply have found Naum4 to obtain impressive results UNDER CERTAIN circumstances. And this is what I have shared with our colleagues her.

Regards,

Tom.

This is true....

[GenoM]

I do not like to imply anything, Evgenii. Thinks are very simple for me:

1.- I enjoy testing engines. This is my first driver.
2.- I like to try new books.
3.- After weeks testing engines I offer the results with my feelings to this chess community.

That's all. I am not saying that Naum4 equals Rybka3. If you read my forecast to Jeroen's test: Rybka 62,5 - Naum 37,5. How can I make this forecast - by the way, very close to reality! - if I think that they are equal?. I simply have found Naum4 to obtain impressive results UNDER CERTAIN circumstances. And this is what I have shared with our colleagues her.

Regards,

Tom.

Belka's under certain circumstances was stronger than R2.3.2a too, I heard.
So?

[M ANSARI]

N4 has a more accurate evaluation that Rybka (generally speaking and not all conditions) in

1. Bishop pair strength in open positions
2. Rook pair when bishop is still in the game vs. Queen and 3 pawns

These are the only 2 things I was able to find that Naum 4 is obviously better at than Rybka 3 ... ofcourse I do not include obvious Rybka 3 bugs that should be fixed with Rybka 3+ which should be out within a couple of weeks. Almost every single one of N4's wins that is not an over reaching of R3 (due to high contempt) is due one of the two things mentioned. N4 has as good evalution as R3 in many other situations but has a lot of weaknesses in understanding pawn promotions and does not probe these possibilities in search and thus suffers tremendous amount of games due to that.

By far N4's biggest weakness is very slow search. It loses a lot of games it shouldn't and draws a lot of games it shouldn't due to that. I would estimate that an 8 core N4 would have a terrible time against a R3 "0" contempt single core simply because of that. I don't know what causes the search to be so slow ... most likely the search does not have the same sophisticated search tricks for probing critical lines like Rybka does. But the slow search could possibly hide many positive aspects of N4 and thus I see this program as having an excellent future due to that. Improved hardware will make slow search not such a big problem in the future.

One thing to note is that when I mentioned some of Rybka 3's weaknesses to Vas he seemed already aware of every single one of them ... and I mean every single little detail ... and so Rybka 4 will probably have patched these tiny little gaps in evaluation and so the arms race continues ... and the improved quality of chess played continues.

[Uri Blass]

N4 has a more accurate evaluation that Rybka (generally speaking and not all conditions) in

1. Bishop pair strength in open positions
2. Rook pair when bishop is still in the game vs. Queen and 3 pawns

These are the only 2 things I was able to find that Naum 4 is obviously better at than Rybka 3 ... ofcourse I do not include obvious Rybka 3 bugs that should be fixed with Rybka 3+ which should be out within a couple of weeks. Almost every single one of N4's wins that is not an over reaching of R3 (due to high contempt) is due one of the two things mentioned. N4 has as good evalution as R3 in many other situations but has a lot of weaknesses in understanding pawn promotions and does not probe these possibilities in search and thus suffers tremendous amount of games due to that.

By far N4's biggest weakness is very slow search. It loses a lot of games it shouldn't and draws a lot of games it shouldn't due to that. I would estimate that an 8 core N4 would have a terrible time against a R3 "0" contempt single core simply because of that. I don't know what causes the search to be so slow ... most likely the search does not have the same sophisticated search tricks for probing critical lines like Rybka does. But the slow search could possibly hide many positive aspects of N4 and thus I see this program as having an excellent future due to that. Improved hardware will make slow search not such a big problem in the future.

One thing to note is that when I mentioned some of Rybka 3's weaknesses to Vas he seemed already aware of every single one of them ... and I mean every single little detail ... and so Rybka 4 will probably have patched these tiny little gaps in evaluation and so the arms race continues ... and the improved quality of chess played continues.

I guess that rybka3's main advantage is a very fast search.

Saying that Naum's biggest weakness is a very slow search suggest that other non rybka's programs has better search relative to Naum and I doubt if it is the case.

Maybe naum has an excellent future but the fact that rybka's search is significantly better does not prove it because we do not know if naum is going to improve the search and only the future is going to tell us more information.

Uri

[M ANSARI]

I am quite that if N4 could improve search that it will gain quite a few ELO points. I have only tested its search against R3, so I don't know how much that compares to other engines. With other engines I ususally see Rybka 3 out-evaluate it ... because it takes a long time for the other engines to start agreeing with Rybka's evaluation. With N4 once the position has settled N4 immediately starts evaluating similar to Rybka ... so it is very obvious that it is searching less than Rybka and that its evaluation is not to blame.

One thing I learned from watching so many games is that Rybka 3 can improve quite a bit by at LTC by doing a few things

1. Reduce contempt to around 7 (from default 15)
2. Add .25 evaluation to positions that have 2 bishops without blocked or doubled pawns
3. Add .15 evaluation more to 2 rooks + 3 pawns against Queen + Knight but keep it the same for Queen + Bishop