How strong will the new naum be relative to current rybka

[Yarget]

I bought Naum 3 earlier today and I have immidiately started to test the new Naum version under my special testconditions:

http://www.talkchess.com/forum/viewtopi ... ew=threads

I must say that my first-hand impression is very positive. While Naum 2.2 lost 9-11 to Zap!Chess Zanzibar in the Gambitgames Naum 3 performed much better and won the similar match with 14½-5½ (Rybka won this match 14-6 against Zap). One thing is a single result (20 games are ofcourse no basis for making judgements), another thing is the subjective impression you get by watching the games. I followed many of the games "live" this afternoon and several times Zap was simply outplayed in 30 or 40 moves. I didn't experience that too often with Naum 2.2 and it's my impression that Naum (among other things) is improved in tactics.

With great interest I'll (as always) follow Heinz' 40/120 ratinglist and I'm sure that Naum 3 will make a strong performance there. But even under my testconditions (short timecontrol and dual-core) I believe that Naum 3 will be one of Rybkas closest rivals.

Regards
Per

[Heinz Van Kempen]

Hi Per ,

first results with 40/120 for the Quad list are really not bad and also the active chess testers are seeing an improvement currently to 70 points over Naum 2.2.

People should just be patient to wait for enough results.

[Uri Blass]

A lot of harsh words, but every single one of them from people who didn't even buy the engine, so they can't really judge it for themselves.
I didn't try to calculate the exact ELO based on my test results. I just estimated it around 100.
Here are the results I based my ELO estimate on. Conditions are 40/4', 1CPU 32-bit, no EGTBs, 64MB hash using Winboard on a moderate hardware.
vs Rybka232a -> +14 -28 =38
vs Toga 1.3.1 -> +39 -18 =23
vs Fruit 2.31 -> +36 -14 =30
vs Hiarcs 11.1 -> +34 -21 =25
vs Shredder 11 -> +27 -11 =22

I am not in the business of scamming people out of their money. What I can promise is that when all the dust settles, people who bought Naum will not be disappointed. I will make sure of that even if the results for this version don't live up to my estimated ELO.

Regards,
Alex

I also looked at the 32 bit results

Now the results that I read are less disappointing

http://www.husvankempen.de/nunn/phpBB2/ ... .php?t=927

Naum2.2 had 2798 at blitz in CEGT and at the time that I wrote my post the only engine that Naum2.2 played 100 blitz games against it was Fritz11

Now at least Naum has more than +50 elo at blitz CEGT

Naum 3 w32 1CPU vs Rybka 2.3.2a w32 1CPU (100) 37.5-62.5 perf= 2872 (W.B.) 37.5
Naum 3 w32 1CPU vs Deep Shredder 11 w32 1CPU (100) 47.5-52.5 perf= 2844 (G.S.)
Naum 3 w32 1CPU vs Fritz 11 (100) 36.0-64.0 perf= 2808 (G.S.)


Naum 3 w32 1CPU vs Bright 0.2c 1CPU ( 20) 13.5- 6.5 perf= 2847 (G.S.)
Naum 3 w32 1CPU vs Fruit 070327 ( 20) 14.5- 5.5 perf= 2986 (G.S.)
Naum 3 w32 1CPU vs Hiarcs X54 w32 ( 20) 11.5- 8.5 perf= 2866 (G.S.)
Naum 3 w32 1CPU vs Rybka 1.2f w32 1CPU ( 20) 10.0-10.0 perf= 2891 (G.S.)

Uri

[geots]

A lot of harsh words, but every single one of them from people who didn't even buy the engine, so they can't really judge it for themselves.
I didn't try to calculate the exact ELO based on my test results. I just estimated it around 100.
Here are the results I based my ELO estimate on. Conditions are 40/4', 1CPU 32-bit, no EGTBs, 64MB hash using Winboard on a moderate hardware.
vs Rybka232a -> +14 -28 =38
vs Toga 1.3.1 -> +39 -18 =23
vs Fruit 2.31 -> +36 -14 =30
vs Hiarcs 11.1 -> +34 -21 =25
vs Shredder 11 -> +27 -11 =22

I am not in the business of scamming people out of their money. What I can promise is that when all the dust settles, people who bought Naum will not be disappointed. I will make sure of that even if the results for this version don't live up to my estimated ELO.

Regards,
Alex

You are making very good progress with each new version Alex and I'm sure that many appreciate your efforts.
Don't be discouraged by negative comments.

Regards, Graham.

Look, if you trashed every engine that could not beat Rybka- explain to me exactly who would be left for Rybka to play.

[Shaun]

A lot of harsh words, but every single one of them from people who didn't even buy the engine, so they can't really judge it for themselves.
I didn't try to calculate the exact ELO based on my test results. I just estimated it around 100.
Here are the results I based my ELO estimate on. Conditions are 40/4', 1CPU 32-bit, no EGTBs, 64MB hash using Winboard on a moderate hardware.
vs Rybka232a -> +14 -28 =38
vs Toga 1.3.1 -> +39 -18 =23
vs Fruit 2.31 -> +36 -14 =30
vs Hiarcs 11.1 -> +34 -21 =25
vs Shredder 11 -> +27 -11 =22

I am not in the business of scamming people out of their money. What I can promise is that when all the dust settles, people who bought Naum will not be disappointed. I will make sure of that even if the results for this version don't live up to my estimated ELO.

Regards,
Alex

Hi Alex,

when the dust settles I think we will see that at faster time controls there is a bigger improvement than at slower time controls, still early days so I may be wrong. Regardless at this level any increase is hard work/impressive... keep it up

I am not sure how easy it would be to add UCI parameters for more/less pruning but I believe that for fast time controls relatively more pruning is optimal if you give us the parameters I will certainly test the theory... although I need more computers as people keep releasing new engines.

I hope to post preliminary results for CCRL 40/4 tonight update running.

All the best

Shaun

[ThatsIt]

Code: Select all

01 Rybka 2.3.2a w32 1CPU       2959   13  13  2044   68.7 %   2822   34.4 %
02 Fritz 11                    2907    8   8  4290   60.5 %   2833   37.2 %

03 Naum 3 w32 1CPU             2861   32  32   308   52.6 %   2843   31.2 %

04 Deep Shredder 11 w32 1CPU   2859   10  10  3048   57.4 %   2807   30.8 %
05 Fruit 2.3.3f Test Beta      2856   10  10  2700   51.9 %   2843   36.0 %
06 Hiarcs 11.1 1CPU            2853   17  17  1164   61.0 %   2775   33.1 %
07 Loop 13.6 w32 1CPU          2843   14  14  1514   59.5 %   2776   33.6 %
08 Toga II 1.4 beta5c 1CPU     2840   16  15  1280   54.7 %   2807   34.1 %

-- Naum 2.2 w32 1CPU           2795   11  11  2492   50.3 %   2793   36.8 %

-- Naum 2.1 w32 1CPU           2770   16  16  1203   52.5 %   2752   32.1 %

[Ovyron]

Look, if you trashed every engine that could not beat Rybka- explain to me exactly who would be left for Rybka to play.

We would then be watching Rybka Vs. Rybka games until our eyes bled.

[Shaun]

Code: Select all

01 Rybka 2.3.2a w32 1CPU       2959   13  13  2044   68.7 %   2822   34.4 %
02 Fritz 11                    2907    8   8  4290   60.5 %   2833   37.2 %

03 Naum 3 w32 1CPU             2861   32  32   308   52.6 %   2843   31.2 %

04 Deep Shredder 11 w32 1CPU   2859   10  10  3048   57.4 %   2807   30.8 %
05 Fruit 2.3.3f Test Beta      2856   10  10  2700   51.9 %   2843   36.0 %
06 Hiarcs 11.1 1CPU            2853   17  17  1164   61.0 %   2775   33.1 %
07 Loop 13.6 w32 1CPU          2843   14  14  1514   59.5 %   2776   33.6 %
08 Toga II 1.4 beta5c 1CPU     2840   16  15  1280   54.7 %   2807   34.1 %

-- Naum 2.2 w32 1CPU           2795   11  11  2492   50.3 %   2793   36.8 %

-- Naum 2.1 w32 1CPU           2770   16  16  1203   52.5 %   2752   32.1 %

Gerhard,

Please can you confirm the time control

Thanks

Shaun

[Uri Blass]

A lot of harsh words, but every single one of them from people who didn't even buy the engine, so they can't really judge it for themselves.
I didn't try to calculate the exact ELO based on my test results. I just estimated it around 100.
Here are the results I based my ELO estimate on. Conditions are 40/4', 1CPU 32-bit, no EGTBs, 64MB hash using Winboard on a moderate hardware.
vs Rybka232a -> +14 -28 =38
vs Toga 1.3.1 -> +39 -18 =23
vs Fruit 2.31 -> +36 -14 =30
vs Hiarcs 11.1 -> +34 -21 =25
vs Shredder 11 -> +27 -11 =22

I am not in the business of scamming people out of their money. What I can promise is that when all the dust settles, people who bought Naum will not be disappointed. I will make sure of that even if the results for this version don't live up to my estimated ELO.

Regards,
Alex

Hi Alex,

when the dust settles I think we will see that at faster time controls there is a bigger improvement than at slower time controls, still early days so I may be wrong. Regardless at this level any increase is hard work/impressive... keep it up

I am not sure how easy it would be to add UCI parameters for more/less pruning but I believe that for fast time controls relatively more pruning is optimal if you give us the parameters I will certainly test the theory... although I need more computers as people keep releasing new engines.

I hope to post preliminary results for CCRL 40/4 tonight update running.

All the best

Shaun

I believe that basically speed improvement are more productive at short time control

Here are too examples and it means that if the main improvement that was done was something that is equivalent to speed improvement you are going to see bigger improvement at fast time control.


ccrl 40/40
Zappa 1.1 64-bit 2733 +24 −24 45.3% +33.1 37.8% 545
98.8%
Zappa 1.1 32-bit 2686 +32 −32 48.9% +8.7 32.6% 322

+47 elo

Zappa Mexico 64-bit 2935 +17 −17 56.7% −43.3 45.1% 1048
63.7%
Zappa Mexico 32-bit 2888 +19 −19 55.1% −31.3 42.4% 857
56.7

+47 elo


ccrl 40/4

Zappa 1.1 64-bit 2712 +15 −15 46.8% +21.4 32.8% 1568
100.0%
Zappa 1.1 32-bit 2638 +25 −25 43.2% +48.7 29.4% 575

+74 elo

Zappa Mexico 64-bit 2910 +12 −12 51.4% −10.6 36.0% 2410
100.0%
Zappa Mexico 32-bit 2853 +20 −20 50.9% −6.6 37.2% 870

+57 elo

[Shaun]

I believe that basically speed improvement are more productive at short time control

Yes if most of the improvements are from faster code then I would expect that too...

I would still like to be able to increase/decrease pruning...

Shaun

EDIT: P.S. We have a new number 2 in the CCRL 40/4 list ftp going as I type...