EPD Endgame Suite

[Dann Corbit]

Which 169 positions have you removed from the original eg_msa.epd in the modified one?

Maybe I missed something... where do these positions come from? They are not in the epd file I posted. I do not have a record for a lot of the positions you posted.

2q5/5pk1/5p2/4p3/3pB2P/2b2QP1/4PP2/r4RK1
2q5/5pk1/5p2/4p3/3pB2P/2b2QP1/4PP2/R5K1
2q5/5pk1/5p2/4p3/3pB2P/2b2QP1/r3PP2/5RK1
2q5/5pk1/5p2/4p3/3pB2P/2bQ2P1/r3PP2/5RK1

Hmmm..

I seem to have mixed two different sets in my SQL query.

Let me try again:
http://cap.connx.com/chess-engines/new- ... a2.epd.bz2

There is clearly an error with this one:
5N2/4b2P/1p1p2k1/p2P2p1/P1P5/1P3R2/7r/3K4 b - - acd 23; bm Kh6; ce 32766; pm Kh6; pv Kh6 h8=Q#;

Since the analysis cannot be correct.

I see the problem with that one, multi-pv analysis showed a mate, which somehow got stored in the primary table.

Code: Select all

    27/2	02:36	 635,390,993	4,052,445	-M1	Kg6-h6 h7-h8Q+
    27/16	02:36	 635,390,993	4,052,471	-M8	Kg6-h5 h7-h8Q+ Kh5-g4 Qh8xh2 Kg4xf3 Qh2-e2+ Kf3-g3 Nf8-g6 Kg3-h3 Kd1-e1 g5-g4 Ke1-f1 g4-g3 Qe2-h5+ Be7-h4 Qh5xh4+
    27/23	02:36	 635,390,993	4,052,471	-1.01	Kg6-g7 Nf8-d7 Rh2xh7 Nd7xb6 Rh7-h8 c4-c5 d6xc5 Nb6-c4 Rh8-h4 Nc4xa5 Rh4-d4+ Kd1-e2 Rd4xd5 Rf3-d3 Rd5-f5 Na5-c4 Rf5-f8 a4-a5 Rf8-a8 Ke2-f3 Kg7-f7 Kf3-e4 Kf7-e6
    27/28	02:36	 635,390,993	4,052,496	-0.76	Be7xf8 Rf3xf8 Kg6xh7 Rf8-b8 g5-g4 Kd1-e1 Rh2-h6 Ke1-f2 Kh7-g6 Rb8xb6 Kg6-f5 Rb6-a6 Kf5-f4 Kf2-g1 Kf4-g3 c4-c5 Rh6-h2 Ra6xd6 Rh2-c2 Rd6-f6 Rc2-c1+ Rf6-f1 Rc1xc5 Rf1-d1 Rc5-c7 Kg1-f1 Kg3-f4 d5-d6

Same problem with this one:
B7/p5p1/3k2P1/3p1R2/2pPn3/8/2P5/r5K1 w - - acd 22; bm Rf1; ce 32720; pm Rf1; pv Rf1 Rxf1+ Kxf1 a5 Bc6 Kxc6 Ke2 a4 c3 a3 Kd1 a2 Kc2 a1=R Kb2 Rg1 Ka2 Nxc3+ Kb2 Ne4 Ka2 Rg2+ Kb1 Rxg6 Kc2 Rg1 Kb2 Rg2+ Kc1 g5 Kd1 g4 Ke1 c3 Kd1 Rd2+ Kc1 Rxd4 Kb1 g3 Kc2 g2 Kb3 g1=Q Ka3 Qa1+ Kb3 Qb2#;

Rf1 was the third choice and results in -mate but the other two choices are not nearly so polar.

[Dann Corbit]

Is the CSV still good? I see I will have have to chop ~250 positions for evaluating too high. As far as the positions that are dead drawn, do you mean that they are equal or they will always result in a draw? How do I identify the latter?

Not sure about the csv either.

There are two problems with incorrect analysis attached.
I can redo the csv file later, after I am sure that all issues have been resolved.

[gaard]

I see what you are saying now:

5N2/4b2P/1p1p2k1/p2P2p1/P1P5/1P3R2/7r/3K4 b - - acd 23; bm Kh6; ce 32766; pm Kh6; pv Kh6 h8=Q#;

[D]5N2/4b2P/1p1p2k1/p2P2p1/P1P5/1P3R2/7r/3K4 b - - 0 1

Code: Select all

  15	00:35.473	  15.268.541	430.584	+M1	1. ... Kh6 2.h8Q+
  15	00:35.469	  15.251.062	430.845	+M9	1. ... Kh5 2.h8Q+ Kg4 3.Qxh2 Kxf3 4.Qe2+ Kg3
  15	00:35.465	   5.986.816	425.895	+1.02	1. ... Kg7 2.Nd7
  15	00:35.458	   5.986.815	425.895	+0.83	1. ... Bxf8 2.Rxf8

[gaard]

The only positions I scrap based on evaluation are those over 1.2 pawns, or when the difference between the best move and the second best move is greater than .3. I keep positions with evaluations greater than -1.2 and less than 1.2 to maintain a representative sample.

[Dann Corbit]

I see what you are saying now:

5N2/4b2P/1p1p2k1/p2P2p1/P1P5/1P3R2/7r/3K4 b - - acd 23; bm Kh6; ce 32766; pm Kh6; pv Kh6 h8=Q#;

[D]5N2/4b2P/1p1p2k1/p2P2p1/P1P5/1P3R2/7r/3K4 b - - 0 1

Code: Select all

  15	00:35.473	  15.268.541	430.584	+M1	1. ... Kh6 2.h8Q+
  15	00:35.469	  15.251.062	430.845	+M9	1. ... Kh5 2.h8Q+ Kg4 3.Qxh2 Kxf3 4.Qe2+ Kg3
  15	00:35.465	   5.986.816	425.895	+1.02	1. ... Kg7 2.Nd7
  15	00:35.458	   5.986.815	425.895	+0.83	1. ... Bxf8 2.Rxf8

Yes, there are exactly two positions in the list that are falsely flagged as checkmate positions.

I need to update my Arena output file parser, which got confused by the new format.

I am glad I caught it now, before I have entered a zillion new rows of data into my database.

[Dann Corbit]

This version has been corrected for those errant positions at the top:
http://cap.connx.com/chess-engines/new- ... a3.epd.bz2

[gaard]

Thanks, Dann. For anyone that is interested, here is the corrected suite with positions removed that evaluate to 0 or greater than 150.

http://dl.dropbox.com/u/11904592/EG_MSb.epd
http://dl.dropbox.com/u/11904592/EG_MSb.pgn

[Laskos]

Thanks, Dann. For anyone that is interested, here is the corrected suite with positions removed that evaluate to 0 or greater than 150.

http://dl.dropbox.com/u/11904592/EG_MSb.epd
http://dl.dropbox.com/u/11904592/EG_MSb.pgn

Interesting suite, I run a test at 5s + 0.5s, 8,100 games

Code: Select all

    Program                            Score       %    Av.Op.  Elo    +   -    Draws

  1 Houdini 1.5a x64               : 1922.5/2700  71.2   3173   3330   13  13   14.3 %
  2 Ivanhoe B49jA_1-cpu            : 1514.5/2700  56.1   3192   3235   12  12   22.4 %
  3 Stockfish 2.0.1 JA 64bit       : 1382.0/2700  51.2   3198   3206   11  11   27.6 %
  4 Rybka 4_x64                    : 1279.5/2700  47.4   3202   3184   11  11   28.0 %
  5 Komodo64 1.3 JA                : 1035.5/2700  38.4   3213   3130   12  12   23.1 %
  6 Critter 0.90 64-bit            :  966.0/2700  35.8   3216   3115   12  12   21.4 %

Compared to general ratings, Houdini, Stockfish and Komodo overperform in the endgame, Rybka and Critter underperform. It seems that two complementary and good in endgame engines are Houdini and Stockfish.

Kai

[Don]

Thanks, Dann. For anyone that is interested, here is the corrected suite with positions removed that evaluate to 0 or greater than 150.

http://dl.dropbox.com/u/11904592/EG_MSb.epd
http://dl.dropbox.com/u/11904592/EG_MSb.pgn

Interesting suite, I run a test at 5s + 0.5s, 8,100 games

Code: Select all

    Program                            Score       %    Av.Op.  Elo    +   -    Draws

  1 Houdini 1.5a x64               : 1922.5/2700  71.2   3173   3330   13  13   14.3 %
  2 Ivanhoe B49jA_1-cpu            : 1514.5/2700  56.1   3192   3235   12  12   22.4 %
  3 Stockfish 2.0.1 JA 64bit       : 1382.0/2700  51.2   3198   3206   11  11   27.6 %
  4 Rybka 4_x64                    : 1279.5/2700  47.4   3202   3184   11  11   28.0 %
  5 Komodo64 1.3 JA                : 1035.5/2700  38.4   3213   3130   12  12   23.1 %
  6 Critter 0.90 64-bit            :  966.0/2700  35.8   3216   3115   12  12   21.4 %

Compared to general ratings, Houdini, Stockfish and Komodo overperform in the endgame, Rybka and Critter underperform. It seems that two complementary and good in endgame engines are Houdini and Stockfish.

Kai

I did a similar test comparing Robolitto and Komodo, and although you did not test Robolitto I came to the conclusion that Komodo has a better endgame than middlegame.

Don

[Laskos]

I did a similar test comparing Robolitto and Komodo, and although you did not test Robolitto I came to the conclusion that Komodo has a better endgame than middlegame.

Don

Yes, confirmed here too, Komodo's endgame seems superior to its middlegame (compared to other engines). The results of Komodo in this suite are

Code: Select all

5 Komodo64 1.3 JA           : 3130  2700 (+723,=625,-1352), 38.4 %

Houdini 1.5a x64              : 540 (+120,= 84,-336), 30.0 %
Rybka 4_x64                   : 540 (+217,=139,-184), 53.1 %
Ivanhoe B49jA_1-cpu           : 540 (+ 80,=113,-347), 25.3 %
Stockfish 2.0.1 JA 64bit      : 540 (+124,=152,-264), 37.0 %
Critter 0.90 64-bit           : 540 (+182,=137,-221), 46.4 %

Komodo even beats Rybka!

Kai