Arasan7.epd Test Results

[Marc MP]

For a change of eng-eng matches, I ran several freeware on arasan7.epd.
Athlon 1.4Ghz, 64M Hash, 8M for 3-4 mens, 20 sec per move. GUI: Arena, Chess Partner. Here are the results (out of 227).

Pro Deo 1.2 Q3 - Tactical Engine 152
Rybka 1.0 Beta Very Tactical 149
Rybka 1.0 Beta Very Positional 144
Slow Chess WV 2.1 Aggressive 143
Naum 2.0 142
Glaurung 1.2.1 141
Pharaon 3.5.1 141
Pro Deo 1.2 Polgar 138
Pro Deo 1.2 Rebel 136
Gambit Fruit 1.0 4bx 135
Spike 1.2 133
Toga 1.3 X4 no egbb 130
Movei00_8_403 130
Pro Deo 1.2 Tal 128
WildCat 7 128
Scorpio 1.91 126
Ruffian 1.0.5 115
Alaric 703 111
Fruit 2.1 110
Delfi 5.1 104
Colossus 2006f 99
The Baron 1.8.1 97

Is there an engine (a free one!) you would like to see on the list?

[Richard Allbert]

Arasan!

[Eelco de Groot]

It's a nice list Marc! How does your Glaurung Mammoth do on Arasan 7?

I can think of a few more strong programs to test: Toga 1.2.1a, Spike 1.1, Pro Deo 1.1

I will post a few results later with Hiarcs 11.1 settings, maybe in another thread, I have done them on an Athlon 2009 MHz but for the rest with about equal conditions.

So far here the programs seem to get more correct results than I would expect from just the increase in clock speed, maybe memory also plays a role? But I have not tested an identical program yet from your list.

My own Glaurung Express 6e settings did also fairly well but not as good as the Hiarcs 11.1 Combinations Ic settings. I think that one is going to do better than Pro Deo 1.2 Q3 on this computer.

Eelco

[Uri Blass]

Yes

Knightdreamer and trace.

Knightdreamer is known to be relatively strong in tactics and in some test suites it is significantly stronger than most of the commercial programs
if not better than all of them when trace is also known to be an engine with good results in tactical tests.

[Eelco de Groot]

The results for Rybka 1.0 Beta are a bit suspect I'm afraid, because Rybka Beta interprets "search for exactly 20 seconds" as "I should stop searching somewhere between one and two minutes". This is hardly fair on the other competitors, unless it is checked by hand from analysis file output which of the 227 positions Rybka solved within twenty seconds or the solution time can be checked in the results matrix or something similar. In position 9 for instance Rybka plays the solution 1.e5 after 56.2 seconds on my computer.

Shredder GUI gives a list which number of positions are solved within 1, 2 seconds etc. but I don't know if Arena or ChessPartner can give this information? Maybe you gave the corrected results Marc?

Rybka 1.0 Beta searches too long in both Shredder GUI and ChessPartner so I think it won't be much different in Arena?

Eelco

[Marc MP]

The results for Rybka 1.0 Beta are a bit suspect I'm afraid, because Rybka Beta interprets "search for exactly 20 seconds" as "I should stop searching somewhere between one and two minutes". This is hardly fair on the other competitors, unless it is checked by hand from analysis file output which of the 227 positions Rybka solved within twenty seconds or the solution time can be checked in the results matrix or something similar. In position 9 for instance Rybka plays the solution 1.e5 after 56.2 seconds on my computer.

Shredder GUI gives a list which number of positions are solved within 1, 2 seconds etc. but I don't know if Arena or ChessPartner can give this information? Maybe you gave the corrected results Marc?

Rybka 1.0 Beta searches too long in both Shredder GUI and ChessPartner so I think it won't be much different in Arena?

Eelco

Hi Eelco,

It looks like everythings is OK in Arena, but I just checked and effectively it doesn't work properly in Chess Partner. I don't think Arena gives the list per solution time however.

[Marc MP]

Here is the update. I plan to run a few more engines this week-end. If you have a special request let me know. The results are out of 227.

Fritz 8 170
Chess Tiger 2007 Gambit 163
Glaurung 1.2.1 Mammoth (M1) 160
Chess Tiger 2007 Normal 153
Pro Deo 1.2 Q3 - Tactical Engine 152
Rybka 1.0 Beta Very Tactical 149
Rybka 1.0 Beta Very Positional 144
Slow Chess WV 2.1 Aggressive 143
Naum 2.0 142
Glaurung 1.2.1 141
Pharaon 3.5.1 141
Toga 1.2.1a 139
Pro Deo 1.2 Polgar 138
Pro Deo 1.2 Rebel 136
Gambit Fruit 1.0 Beta 4bx 135
Spike 1.1 135
Spike 1.2 133
Toga 1.3 X4 no egbb 130
Movei00_8_403 130
Pro Deo 1.2 Tal 128
WildCat 7 128
Scorpio 1.91 126
Knight Dreamer 3.2 120
Trace 1.37a 118
Ruffian 1.0.5 115
Alaric 703 111
Fruit 2.1 110
List 5.12 107
Delfi 5.1 104
Colossus 2006f 99
The Baron 1.8.1 97
Arasan 9.5 95

I must say I'm quite amazed by Fritz 8 performance! Believe or not I never used it before with test suites because I didn't knew how to convert epd to cbf!! I just learned 2 hours ago!

The Glaurung settings "Mammoth" (M1 stands for the version number!) are an attempt for a "tactical engine" (like Pro Deo Q3). Maybe someone can improve on this?

Hash = 64
Aggressiveness = 260
Cowardice = 75
Passed pawns (middle game) = 240
Passed pawns (endgame) = 240
Pawn structure (middle game) = 100
Pawn structure (endgame) = 100
Mobility (middle game) = 220
Mobility (endgame) = 150
Space = 75
Development = 200
Null move reduction factor (middle game) = 3
Null move reduction factor (endgame) = 3
Late move reductions = All nodes
Reduce based on = Knowledge
Futility pruning = Non-PV nodes
Futility margin 0 = 150
Futility margin 1 = 225
Futility margin 2 = 450
Check extension = 60
One reply to check extension = 45
Two replies to check extension = 20
Mate threat extension = 45
Pawn push to 7th rank extension = 45
Threat depth = 3
Static evaluation cache = true
Static evaluation cache size = 8
Static null move pruning = true
Static pruning depth = 3
Checks in quiescence search = 3
Hash quiescence search = true
Number of threads = 1
Minimum tree split depth = 4
Position learning = false

[Marc MP]

I ran a few more:

Toga 1.2.1a BlueBerry Finder 147
Slow Chess WV 2.1 Normal 141
ET Chess 132
Frenzee 3.0 126
Crafty 21.5 (Blended) 86

Good results for the Toga settings! Also ET Chess doing good elo-wise.