Eigenmann Endgame Test

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, chrisw, Rebel

Walter Eigenmann

Eigenmann Endgame Test

Post by Walter Eigenmann »

.
New Engines:
Zappa 1.1 and Mint 2.3.

Total Engines: 90

Time: 60 sec/engine

Hard-/Software:
Athlon64/3000+ (128MB Hash) / 3-5men-Nalimov-, Shredder- & Bit-Bases /
TB-Cache: 64MB / GUIs: Fritz-10 & Arena-1.99b4 / Default-Settings

Downloads: cbh-, pgn-, epd- & xls-Files

Regards: Walter

.

Ranking Best Versions

Code: Select all

    Program                    Solutions    Time

01. Shredder 10                68/100      00:45:17
02. Rybka 2.2                  64/100      00:50:48
03. Fritz 10                   63/100      00:49:47
04. Deep Frenzee 3.0           61/100      00:54:04
05. Hiarcs 11.1                59/100      00:58.50
06. The Baron 2.12             58/100      00:59:48
07. SmarThink 1.00             55/100      01:01:32
08. Chess Tiger 2007           54/100      00:58:55
09. Toga II 1.3x4              54/100      01:01:58
10. Spike 1.2 Turin            53/100      00:59:42
11. SlowChess Blitz WV2.1      53/100      01:02:32
12. Glaurung 2 epsilon2        51/100      01:02:26
13. Delfi 5.1                  51/100      01:05:59
14. Ktulu 8                    50/100      01:02:55
15. Pharaon 3.5.1              50/100      01:03:32
16. Yace 0.99.87               50/100      01:04:44
17. LoopMP 12.32               50/100      01:06:02
18. Naum 2.0                   50/100      01:06:08
19. Ruffian 2.1.0              48/100      01:05:41
20. Fruit 2.2.1                48/100      01:06:29
21. Crafty 20.14               47/100      01:05:26
22. Strelka 1.0b               47/100      01:06:31
23. Colossus 2007a             47/100      01:06:55
24. Alaric 704                 46/100      01:06:14
25. Junior 10.1                45/100      01:06:14
26. Gandalf 6.0                44/100      01:06:00
27. Deep Sjeng 2.5             44/100      01:10:49
28. Scorpio 1.91               43/100      01:07:35
29. Patzer 3.8                 42/100      01:06:50
30. WildCat 7.0                40/100      01:13:10
31. Anaconda 2.0.1             40/100      01:16:26
32. Aristarch 4.50             39/100      01:13:41
33. Nimzo 8                    38/100      01:13:14
34. Pepito 1.59                38/100      01:15:16
35. ProDeo 1.4                 38/100      01:18:50
36. Chessmaster 9000           37/100      01:13:21
37. Jonny 2.83                 37/100      01:15:06
38. Movei 0.08.402             35/100      01:16:22
39. Amyan 1.597                35/100      01:16:58
40. Quark 2.35                 35/100      01:19:32
41. LGoliath Evolution         34/100      01:14:21
42. Arasan 9.5                 34/100      01:16:40
43. Gaia 3.5                   34/100      01:17:44
44. Tao 5.6                    31/100      01:19:42
45. AnMon 5.60                 28/100      01:22:54
46. DanaSah 3.03               27/100      01:21:13
47. SOS 5                      27/100      01:21:49
48. Zappa 1.1                  27/100      01:23.18
49. GreKo 5.0                  24/100      01:25:15
50. Ufim 8.02                  23/100      01:24:18
51. Hamsters 0.2               22/100      01:22:53
52. Chispa 4.0.3               22/100      01:25:44
53. Resp 0.19                  21/100      01:24:04
54. Homer 2.0                  20/100      01:26:44
55. BeoWulf 2.4                19/100      01:26:31
56. Queen 3.09                 19/100      01:26:43
57. Abrok 5.0                  19/100      01:28:31
58. Smirf MS-167d              18/100      01:27:44
59. Lime 62                    17/100      01:28:33
60. AliChess 4.08              16/100      01:27:46
61. BikJump 1.2.1              15/100      01:27:27
62. Mint 2.3                   15/100      01:28.13
63. FireFly 2.2.2              15/100      01:28:33
.
Michael Sherwin
Posts: 3196
Joined: Fri May 26, 2006 3:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Eigenmann Endgame Test

Post by Michael Sherwin »

Hi Walter,

Nice to see something different for a change. My program RomiChess has zero endgame knowledge and I am not even close to the point where it can be added. There is so much to add first, like 'material based' draw detection and a few dozen other things. Still it would be interesting to compare how well such an engine would do in your test. You can download RomiChess from the WBEC 'engine detail' pages. Or you can email me at mjsherwin@netzero_dot_net (o.n) and I will send the latest beta to you. If you are interested.

Mike
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Walter Eigenmann

Re: Eigenmann Endgame Test

Post by Walter Eigenmann »

.

Thx for your hint.

I will test RomiChess P3j (Djiksman/Default)

Regards: Walter

E-E-T:
http://www.beepworld.de/members38/eigenmann/e_e_t.htm

.
jdart
Posts: 4388
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Eigenmann Endgame Test

Post by jdart »

I took a look at this position recently (#39):

[D] k2N4/1qpK1p2/1p6/1P4p1/1P4P1/8/8/8 w - -

I think this is less a test of endgame knowledge than a null move test - if your program allows null move with reduced material, it will fail this.

--Jon
jarkkop
Posts: 198
Joined: Thu Mar 09, 2006 2:44 am
Location: Helsinki, Finland

Re: Eigenmann Endgame Test

Post by jarkkop »

Is this common in chess programs?
Do mean they will never get it or it takes 10 times longer?
jdart
Posts: 4388
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Eigenmann Endgame Test

Post by jdart »

>Do mean they will never get it or it takes 10 times longer?

Never. But I have tried scaling back null move and, while it "fixes" corner case like this, it caused overall performance to suffer.
Walter Eigenmann

Re: Eigenmann Endgame Test

Post by Walter Eigenmann »

jdart wrote:I took a look at this position recently (#39):

[D] k2N4/1qpK1p2/1p6/1P4p1/1P4P1/8/8/8 w - -

I think this is less a test of endgame knowledge than a null move test - if your program allows null move with reduced material, it will fail this.

--Jon
Of course.

The E-E-T contains many aspects of the ComputerChess Endgame stuff.

Regards: Walter

.
User avatar
smirobth
Posts: 2307
Joined: Wed Mar 08, 2006 8:41 pm
Location: Brownsville Texas USA

Re: Eigenmann Endgame Test

Post by smirobth »

Hi Walter,
Thanks for your interesting work. I have a few questions about your test. Are all 100 positions solved by at least one program?
If so, what is the smallest set of programs (including which programs) that solves all 100?
If not, how many positions are solved by at least one program, and what is the smallest set of programs (including which programs) that solves them?
Which four programs, when their results are combined, solve the maximum number of positions and which programs are they?

Or if answering would be too much of a pain, perhaps you have a download link to an xls file from which answers to such questions could be extracted?

Thanks.

-Robin
- Robin Smith
User avatar
Thomas Mayer
Posts: 385
Joined: Thu Mar 09, 2006 6:45 pm
Location: Nellmersbach, Germany

Re: Eigenmann Endgame Test

Post by Thomas Mayer »

Hi Jon,
jdart wrote:I took a look at this position recently (#39):

[D] k2N4/1qpK1p2/1p6/1P4p1/1P4P1/8/8/8 w - -

I think this is less a test of endgame knowledge than a null move test - if your program allows null move with reduced material, it will fail this.

--Jon
well, I think Walter is right in adding such a position because zugzwang IS endgame knowledge. E.g. Quark has a very shallow verification search which doesn't cost too much and uses it only in endgames. So it took some time (found at ply 24) to find the solution. Anyway it CAN solve that position. I always have Dieter Buerrsners reasoning for implementing verfication search in mind: "I simply do not like that there are positions that the engine can't solve even when it gets endless time"...

And when I remember correctly several half & full points were lost on different tournaments just because of such nullmove problems. (I have Rybka & Fritz to name some here in mind)

Quarks verification search is maybe 2%... so you can't count it in elo. Well here and there half a point is maybe also countless in elos - but you sleep better... ;)

Greets, Thomas
User avatar
Daniel Mehrmann
Posts: 858
Joined: Wed Mar 08, 2006 9:24 pm
Location: Germany
Full name: Daniel Mehrmann

Re: Eigenmann Endgame Test

Post by Daniel Mehrmann »

jdart wrote:I took a look at this position recently (#39):

[D] k2N4/1qpK1p2/1p6/1P4p1/1P4P1/8/8/8 w - -

I think this is less a test of endgame knowledge than a null move test - if your program allows null move with reduced material, it will fail this.

--Jon
I agree this is a nullmove test:

Homer with standard nullmove enabled:

Code: Select all

 13/25	 0:01 	-0.01 	1.Nxb7 Kxb7 2.Ke7 c5 3.bxc6+ Kxc6 4.Kxf7 Kb5 5.Kf6 Kxb4 (748.928) 623  TB:402
 14/24	 0:01 	-0.01 	1.Nxb7 Kxb7 2.Ke7 c5 3.bxc6+ Kxc6 4.Kxf7 Kb5 5.Kf6 Kxb4 (949.917) 614  TB:402
 14/27	 0:01 	-0.01 	1.Nxb7 Kxb7 2.Ke7 c5 3.bxc6+ Kxc6 4.Kxf7 Kb5 5.Kf6 Kxb4 (973.382) 605  TB:402
 15/28	 0:02 	-0.01 	1.Nxb7 Kxb7 2.Ke7 c5 3.bxc6+ Kxc6 4.Kxf7 Kb5 5.Kf6 Kxb4 (1.563.968) 582  TB:1.460
 15/28	 0:02 	-0.01 	1.Nxb7 Kxb7 2.Ke7 c5 3.bxc6+ Kxc6 4.Kxf7 Kb5 5.Kf6 Kxb4 (1.597.383) 578  TB:1.460
 16/30	 0:05 	-0.01 	1.Nxb7 Kxb7 2.Ke7 c5 3.bxc6+ Kxc6 4.Kxf7 Kb5 5.Kf6 Kxb4 (2.852.047) 562  TB:3.426
 16/30	 0:05 	-0.01 	1.Nxb7 Kxb7 2.Ke7 c5 3.bxc6+ Kxc6 4.Kxf7 Kb5 5.Kf6 Kxb4 (2.914.105) 559  TB:4.293
....Nxb7 still after 3 minutes
Homer without nullmove:

Code: Select all

 14/30	 0:03 	+0.49++	1.Nc6 (1.515.559) 502  TB:1.227
 14/30	 0:03 	+2.15 	1.Nc6 f6 2.Kd8 f5 3.gxf5 g4 4.f6 g3 5.Kd7 Qxc6+ 6.bxc6 g2 7.f7 g1Q 8.f8Q+ Ka7 9.Kxc7 Qd4 10.Kc8 Ka6 (1.843.947) 502  TB:1.658
 14/30	 0:04 	+2.15 	1.Nc6 f6 2.Kd8 f5 3.gxf5 g4 4.f6 g3 5.Kd7 Qxc6+ 6.bxc6 g2 7.f7 g1Q 8.f8Q+ Ka7 9.Kxc7 Qd4 10.Kc8 Ka6 (2.101.782) 496  TB:2.122
 15/31	 0:06 	+2.65++	1.Nc6 (3.183.606) 498  TB:3.325
 15/31	 0:07 	+2.49 	1.Nc6 f6 2.Kd8 Qb8+ 3.Nxb8 Kxb8 4.Kd7 Kb7 5.Ke6 Kc8 6.Kxf6 Kd7 7.Kxg5 Ke7 8.Kh6 Ke8 9.g5 Kf8 10.Kh7 Kf7 (3.533.254) 499  TB:3.325
best move: Nd8-c6 time: 0:09.859 min  n/s: 507.920  nodes: 4.653.056 TB: 4.938 
Best,
Daniel

ps: This is a weakness of course and need to fixed ;) So, i don't say its a bad test position.