How strong will the new naum be relative to current rybka

[Tony Thomas]

I do not know under what kind of conditions Alex tests his engine. So I wont judge just yet. Under my conditions the new Naum is showing an improvement of almost 200 points but that could have been due to the fact that the previous versions didnt seem to like my time controls.

[ozziejoe]

Give naum some time folks. She may yet prove to be 100 better.

best
j

[genorb]

We should not be too naive. It is clear that when an author says that the improvement is 50 or 100 elo, this is just an estimate, otherwise the number would not be an "exact" one like this (it would be 47 or 96 or something like that). On the other hand if an author could give an accurate value for the improvement what would be the point of lists like CCRL and CEGT. An author has no time to test accurately his program to get a precise estimation of the improvement. It goes like this for many commercial product, we are clever enough to get the message no? For example when you buy a car, the constructor will tell you that the car uses a given quantity of gas per 100km. Even if the number is accurately dtermined, this is done for a very specific situation (I think they test it with a constant speed of 90km/h) and this number is then not very useful in practice (even if it gives an order a magnitude to compare with other cars).

[slobo]

We should not be too naive. It is clear that when an author says that the improvement is 50 or 100 elo, this is just an estimate, otherwise the number would not be an "exact" one like this (it would be 47 or 96 or something like that). On the other hand if an author could give an accurate value for the improvement what would be the point of lists like CCRL and CEGT. An author has no time to test accurately his program to get a precise estimation of the improvement. It goes like this for many commercial product, we are clever enough to get the message no?

It´s very strange that Uri Blass does not understand that point. He is not a beginner in programing chess engines.

[naum]

A lot of harsh words, but every single one of them from people who didn't even buy the engine, so they can't really judge it for themselves.
I didn't try to calculate the exact ELO based on my test results. I just estimated it around 100.
Here are the results I based my ELO estimate on. Conditions are 40/4', 1CPU 32-bit, no EGTBs, 64MB hash using Winboard on a moderate hardware.
vs Rybka232a -> +14 -28 =38
vs Toga 1.3.1 -> +39 -18 =23
vs Fruit 2.31 -> +36 -14 =30
vs Hiarcs 11.1 -> +34 -21 =25
vs Shredder 11 -> +27 -11 =22

I am not in the business of scamming people out of their money. What I can promise is that when all the dust settles, people who bought Naum will not be disappointed. I will make sure of that even if the results for this version don't live up to my estimated ELO.

Regards,
Alex

[Tony Thomas]

Give naum some time folks. She may yet prove to be 100 better.

best
j

Some people likes to just run matches against 30 different versions of the three top opponents. Of course, they would be disappointed. Me on the other hand who runs an engine versus the same set of opponents couldnt be any happier.

[Heinz Van Kempen]

Hi Alex ,

at least I am not disappointed up to now and despite of all the hope that Rybka ought to have tougher competition to make our hobby again more interesting with more suspense I was realistic enough to expect that Naum 3 would well compete with Zappa Mexico at longer time controls, what nonetheless is quite an achievement and what seems to be likely based on the few results we have so far and when I bear in mind our CEGT rating lists where Naum is one of the very few engines gaining with more time and CPU´s.

The bad thing is that I can deliver a decent amount of games only within a few weeks and not after one day already, like the Blitz and rapid chess testers .

There were times when engines improved by 10-30 ELO each year, if at all and years ago we all were very happy and cheered with a 50 ELO improvemnt in a shorter time. Seems in this respect the sudden impact and rise of Rybka the times of modesty are over. People are demanding and will be the same way disappointed if the next Rybka version will not give another considerable improvement. From all your postings I read I know that you are honest and also optimistic of course and maybe forgot that the small amount of test games you had could be misleading due to the usual statistical noise (as well as results from all testers could still be misleading at this point). I think if I were an engine author I would not claim anything about improvement and just wait for testing at all time controls, so no one will be disappointed and some maybe say: "Look, he just achieved another considerabe improvement like from Naum 2.2 over Naum 2.1".

My advice for those who do not own a quad or better is simply to run games with longer time controls when they want to see something better. After all computer chess competition should be also about quality and watching games and learnig from them and not about stats only. To be honest I am even tired of too many stats without having a look at some of the beautiful games.

I wish you good luck and success in the future. When all your versions give such improvements - as it seems by now - I will be very happy.

Best Regards
Heinz (CEGT testing team)

[ThatsIt]

Hi Alex !

[...snip...]
vs Rybka232a -> +14 -28 =38
vs Toga 1.3.1 -> +39 -18 =23
vs Fruit 2.31 -> +36 -14 =30
vs Hiarcs 11.1 -> +34 -21 =25
vs Shredder 11 -> +27 -11 =22

This means +123 in comparison to Naum 2.2 w32 1CPU
(CEGT-Blitz-Ratinglist). Your forecast seems to be
nearly perfect !

Best to you,
G.S.

[Graham Banks]

A lot of harsh words, but every single one of them from people who didn't even buy the engine, so they can't really judge it for themselves.
I didn't try to calculate the exact ELO based on my test results. I just estimated it around 100.
Here are the results I based my ELO estimate on. Conditions are 40/4', 1CPU 32-bit, no EGTBs, 64MB hash using Winboard on a moderate hardware.
vs Rybka232a -> +14 -28 =38
vs Toga 1.3.1 -> +39 -18 =23
vs Fruit 2.31 -> +36 -14 =30
vs Hiarcs 11.1 -> +34 -21 =25
vs Shredder 11 -> +27 -11 =22

I am not in the business of scamming people out of their money. What I can promise is that when all the dust settles, people who bought Naum will not be disappointed. I will make sure of that even if the results for this version don't live up to my estimated ELO.

Regards,
Alex

You are making very good progress with each new version Alex and I'm sure that many appreciate your efforts.
Don't be discouraged by negative comments.

Regards, Graham.

[genorb]

I do appreciate your work, I bought Naum 2.2 and Naum 3.0 and will buy any additional new version (especially if you continue to make discounts for existing customers which is a very nice commercial attitude unlike many others)