EPD Endgame Suite

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
gaard
Posts: 207
Joined: Mon Jun 07, 2010 1:13 am
Location: Ann Arbor, Michigan

EPD Endgame Suite

Post by gaard » Tue Feb 15, 2011 6:03 am

This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd

User avatar
Mike S.
Posts: 1461
Joined: Thu Mar 09, 2006 4:33 am

Re: EPD Endgame Suite

Post by Mike S. » Tue Feb 15, 2011 4:41 pm

How do you use these 1214 positions to get endgame ratings? I don't see any solution moves. Are they meant as starting positions for matches?
Regards, Mike

Jouni
Posts: 1788
Joined: Wed Mar 08, 2006 7:15 pm

Re: EPD Endgame Suite

Post by Jouni » Tue Feb 15, 2011 5:26 pm

Hi,

Are these from real games? What are numbers in the end of epd lines?

Jouni

User avatar
Laskos
Posts: 8047
Joined: Wed Jul 26, 2006 8:21 pm

Re: EPD Endgame Suite

Post by Laskos » Tue Feb 15, 2011 6:02 pm

Mike S. wrote:How do you use these 1214 positions to get endgame ratings? I don't see any solution moves. Are they meant as starting positions for matches?
That's the whole idea, not to have best move solutions. Otherwise it would be a test suite competition, which could be far away from real-game results.

Kai

User avatar
gaard
Posts: 207
Joined: Mon Jun 07, 2010 1:13 am
Location: Ann Arbor, Michigan

Re: EPD Endgame Suite

Post by gaard » Tue Feb 15, 2011 6:25 pm

Yes. They are taken from ICCF's _Complete Server Games 2010-04-03_. The last number is the move number of the position in the game it was taken from. The one before that has to do with the 50 move rule I believe.

Dann Corbit
Posts: 8670
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: EPD Endgame Suite

Post by Dann Corbit » Wed Feb 16, 2011 8:28 pm

gaard wrote:This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd
There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.

User avatar
gaard
Posts: 207
Joined: Mon Jun 07, 2010 1:13 am
Location: Ann Arbor, Michigan

Re: EPD Endgame Suite

Post by gaard » Wed Feb 16, 2011 8:44 pm

Dann Corbit wrote:
gaard wrote:This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd
There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.
How long did your analysis run? Would you mind posting the positions that are dead lost, dead drawn, and have a score greater than 120 centipawns so they can be removed?

Dann Corbit
Posts: 8670
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: EPD Endgame Suite

Post by Dann Corbit » Wed Feb 16, 2011 8:52 pm

gaard wrote:
Dann Corbit wrote:
gaard wrote:This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd
There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.
How long did your analysis run? Would you mind posting the positions that are dead lost, dead drawn, and have a score greater than 120 centipawns so they can be removed?
I think my total analysis will probably done tomorrow. I will post a summary then.

User avatar
gaard
Posts: 207
Joined: Mon Jun 07, 2010 1:13 am
Location: Ann Arbor, Michigan

Re: EPD Endgame Suite

Post by gaard » Wed Feb 16, 2011 9:55 pm

Thanks, Dann. I trust your judgment. If you want to post the corrected suite too with those positions removed, that'd be great.

User avatar
Laskos
Posts: 8047
Joined: Wed Jul 26, 2006 8:21 pm

Re: EPD Endgame Suite

Post by Laskos » Thu Feb 17, 2011 8:19 am

Dann Corbit wrote:
gaard wrote:This is what I am now using to establish endgame ratings. You may also find is useful. For analyzing these positions, I used Rybka 4 exclusively with a fixed depth of 10. All positions meet the criteria:

1) Differ in more then 6 ranks from every other positions
2) White material is less than 15
3) Black material is less than 15
4) White plus Black material is greater than 17
5) Do not evaluate to 0
6) Do not evaluate greater than 120 centipawns
7) Do not evaluate greater than 30 centipawns difference from the next best move

http://dl.dropbox.com/u/11904592/EG_MSa.epd
There are about 100 positions that are dead lost.
There are a little over 100 positions that are dead drawn.

There are 278 positions with a score of more than one pawn up or down.
There are 131 positions that have a score of more than two pawns up or down.
These are not fatal flaws of an endgame positions suite, the real games towards the endgame are indeed quite often fairly decided. I present here the first results, which look interesting, after 1,800 games on 1 core at 5s + 0.5s.

Code: Select all

    Program                            Score     %    Av.Op.  Elo    +   -    Draws

  1 Houdini 1.5a x64               : 434.0/600  72.3   3171   3338   28  27   18.0 %
  2 Ivanhoe B49jA_1-cpu            : 313.0/600  52.2   3196   3211   24  24   25.0 %
  3 Stockfish 2.0.1 JA 64bit       : 310.0/600  51.7   3197   3209   23  23   29.0 %
  4 Rybka 4_x64                    : 291.5/600  48.6   3201   3191   23  23   31.2 %
  5 Critter 0.90 64-bit            : 236.5/600  39.4   3211   3137   25  25   23.8 %
  6 Komodo64 1.3 JA                : 215.0/600  35.8   3216   3115   25  25   23.0 %
The results are not flattened at all now and look more like I was expecting.

Code: Select all

Individual statistics:

1 Houdini 1.5a x64          : 3338  600 (+380,=108,-112), 72.3 %

Rybka 4_x64                   : 120 (+ 60,= 35,- 25), 64.6 %
Ivanhoe B49jA_1-cpu           : 120 (+ 79,= 20,- 21), 74.2 %
Komodo64 1.3 JA               : 120 (+ 85,= 11,- 24), 75.4 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 63,= 32,- 25), 65.8 %
Critter 0.90 64-bit           : 120 (+ 93,= 10,- 17), 81.7 %

2 Ivanhoe B49jA_1-cpu       : 3211  600 (+238,=150,-212), 52.2 %

Houdini 1.5a x64              : 120 (+ 21,= 20,- 79), 25.8 %
Rybka 4_x64                   : 120 (+ 52,= 39,- 29), 59.6 %
Komodo64 1.3 JA               : 120 (+ 71,= 27,- 22), 70.4 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 35,= 36,- 49), 44.2 %
Critter 0.90 64-bit           : 120 (+ 59,= 28,- 33), 60.8 %

3 Stockfish 2.0.1 JA 64bit  : 3209  600 (+223,=174,-203), 51.7 %

Houdini 1.5a x64              : 120 (+ 25,= 32,- 63), 34.2 %
Rybka 4_x64                   : 120 (+ 49,= 38,- 33), 56.7 %
Ivanhoe B49jA_1-cpu           : 120 (+ 49,= 36,- 35), 55.8 %
Komodo64 1.3 JA               : 120 (+ 47,= 38,- 35), 55.0 %
Critter 0.90 64-bit           : 120 (+ 53,= 30,- 37), 56.7 %

4 Rybka 4_x64               : 3191  600 (+198,=187,-215), 48.6 %

Houdini 1.5a x64              : 120 (+ 25,= 35,- 60), 35.4 %
Ivanhoe B49jA_1-cpu           : 120 (+ 29,= 39,- 52), 40.4 %
Komodo64 1.3 JA               : 120 (+ 67,= 31,- 22), 68.8 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 33,= 38,- 49), 43.3 %
Critter 0.90 64-bit           : 120 (+ 44,= 44,- 32), 55.0 %

5 Critter 0.90 64-bit       : 3137  600 (+165,=143,-292), 39.4 %

Houdini 1.5a x64              : 120 (+ 17,= 10,- 93), 18.3 %
Rybka 4_x64                   : 120 (+ 32,= 44,- 44), 45.0 %
Ivanhoe B49jA_1-cpu           : 120 (+ 33,= 28,- 59), 39.2 %
Komodo64 1.3 JA               : 120 (+ 46,= 31,- 43), 51.2 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 37,= 30,- 53), 43.3 %

6 Komodo64 1.3 JA           : 3115  600 (+146,=138,-316), 35.8 %

Houdini 1.5a x64              : 120 (+ 24,= 11,- 85), 24.6 %
Rybka 4_x64                   : 120 (+ 22,= 31,- 67), 31.2 %
Ivanhoe B49jA_1-cpu           : 120 (+ 22,= 27,- 71), 29.6 %
Stockfish 2.0.1 JA 64bit      : 120 (+ 35,= 38,- 47), 45.0 %
Critter 0.90 64-bit           : 120 (+ 43,= 31,- 46), 48.8 %

Code: Select all

Games        :   1800 (finished)

White Wins   :    605 (33.6 %)
Black Wins   :    745 (41.4 %)
Draws        :    450 (25.0 %)
Unfinished   :      0

White Perf.  : 46.1 %
Black Perf.  : 53.9 %
The draw rate is a little low, and the curious thing is that Black performance is higher than White one. Again not a fatal flaw, because each position is played with reversed colours.

Interesting results anyway, and I will continue the test.

Kai

Post Reply