EPD Endgame Suite

[gaard]

This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd

[Mike S.]

How do you use these 1214 positions to get endgame ratings? I don't see any solution moves. Are they meant as starting positions for matches?

[Jouni]

Hi,

Are these from real games? What are numbers in the end of epd lines?

Jouni

[Laskos]

How do you use these 1214 positions to get endgame ratings? I don't see any solution moves. Are they meant as starting positions for matches?

That's the whole idea, not to have best move solutions. Otherwise it would be a test suite competition, which could be far away from real-game results.

Kai

[gaard]

Yes. They are taken from ICCF's _Complete Server Games 2010-04-03_. The last number is the move number of the position in the game it was taken from. The one before that has to do with the 50 move rule I believe.

[Dann Corbit]

This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd

There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.

[gaard]

This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd

There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.

How long did your analysis run? Would you mind posting the positions that are dead lost, dead drawn, and have a score greater than 120 centipawns so they can be removed?

[Dann Corbit]

This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd

There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.

How long did your analysis run? Would you mind posting the positions that are dead lost, dead drawn, and have a score greater than 120 centipawns so they can be removed?

I think my total analysis will probably done tomorrow. I will post a summary then.

[gaard]

Thanks, Dann. I trust your judgment. If you want to post the corrected suite too with those positions removed, that'd be great.

[Laskos]

This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd

There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.

These are not fatal flaws of an endgame positions suite, the real games towards the endgame are indeed quite often fairly decided. I present here the first results, which look interesting, after 1,800 games on 1 core at 5s + 0.5s.

Code: Select all

    Program                            Score     %    Av.Op.  Elo    +   -    Draws

  1 Houdini 1.5a x64               : 434.0/600  72.3   3171   3338   28  27   18.0 %
  2 Ivanhoe B49jA_1-cpu            : 313.0/600  52.2   3196   3211   24  24   25.0 %
  3 Stockfish 2.0.1 JA 64bit       : 310.0/600  51.7   3197   3209   23  23   29.0 %
  4 Rybka 4_x64                    : 291.5/600  48.6   3201   3191   23  23   31.2 %
  5 Critter 0.90 64-bit            : 236.5/600  39.4   3211   3137   25  25   23.8 %
  6 Komodo64 1.3 JA                : 215.0/600  35.8   3216   3115   25  25   23.0 %

The results are not flattened at all now and look more like I was expecting.

Code: Select all

Individual statistics:

1 Houdini 1.5a x64          : 3338  600 (+380,=108,-112), 72.3 %

Rybka 4_x64                   : 120 (+ 60,= 35,- 25), 64.6 %
Ivanhoe B49jA_1-cpu           : 120 (+ 79,= 20,- 21), 74.2 %
Komodo64 1.3 JA               : 120 (+ 85,= 11,- 24), 75.4 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 63,= 32,- 25), 65.8 %
Critter 0.90 64-bit           : 120 (+ 93,= 10,- 17), 81.7 %

2 Ivanhoe B49jA_1-cpu       : 3211  600 (+238,=150,-212), 52.2 %

Houdini 1.5a x64              : 120 (+ 21,= 20,- 79), 25.8 %
Rybka 4_x64                   : 120 (+ 52,= 39,- 29), 59.6 %
Komodo64 1.3 JA               : 120 (+ 71,= 27,- 22), 70.4 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 35,= 36,- 49), 44.2 %
Critter 0.90 64-bit           : 120 (+ 59,= 28,- 33), 60.8 %

3 Stockfish 2.0.1 JA 64bit  : 3209  600 (+223,=174,-203), 51.7 %

Houdini 1.5a x64              : 120 (+ 25,= 32,- 63), 34.2 %
Rybka 4_x64                   : 120 (+ 49,= 38,- 33), 56.7 %
Ivanhoe B49jA_1-cpu           : 120 (+ 49,= 36,- 35), 55.8 %
Komodo64 1.3 JA               : 120 (+ 47,= 38,- 35), 55.0 %
Critter 0.90 64-bit           : 120 (+ 53,= 30,- 37), 56.7 %

4 Rybka 4_x64               : 3191  600 (+198,=187,-215), 48.6 %

Houdini 1.5a x64              : 120 (+ 25,= 35,- 60), 35.4 %
Ivanhoe B49jA_1-cpu           : 120 (+ 29,= 39,- 52), 40.4 %
Komodo64 1.3 JA               : 120 (+ 67,= 31,- 22), 68.8 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 33,= 38,- 49), 43.3 %
Critter 0.90 64-bit           : 120 (+ 44,= 44,- 32), 55.0 %

5 Critter 0.90 64-bit       : 3137  600 (+165,=143,-292), 39.4 %

Houdini 1.5a x64              : 120 (+ 17,= 10,- 93), 18.3 %
Rybka 4_x64                   : 120 (+ 32,= 44,- 44), 45.0 %
Ivanhoe B49jA_1-cpu           : 120 (+ 33,= 28,- 59), 39.2 %
Komodo64 1.3 JA               : 120 (+ 46,= 31,- 43), 51.2 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 37,= 30,- 53), 43.3 %

6 Komodo64 1.3 JA           : 3115  600 (+146,=138,-316), 35.8 %

Houdini 1.5a x64              : 120 (+ 24,= 11,- 85), 24.6 %
Rybka 4_x64                   : 120 (+ 22,= 31,- 67), 31.2 %
Ivanhoe B49jA_1-cpu           : 120 (+ 22,= 27,- 71), 29.6 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 35,= 38,- 47), 45.0 %
Critter 0.90 64-bit           : 120 (+ 43,= 31,- 46), 48.8 %

Code: Select all

Games        :   1800 (finished)

White Wins   :    605 (33.6 %)
Black Wins   :    745 (41.4 %)
Draws        :    450 (25.0 %)
Unfinished   :      0

White Perf.  : 46.1 %
Black Perf.  : 53.9 %

The draw rate is a little low, and the curious thing is that Black performance is higher than White one. Again not a fatal flaw, because each position is played with reversed colours.

Interesting results anyway, and I will continue the test.

Kai