Stockfish endgame play: SIMPLY STUNNING

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Jouni
Posts: 3232
Joined: Wed Mar 08, 2006 8:15 pm

Stockfish endgame play: SIMPLY STUNNING

Post by Jouni »

I took 10 endgame positions from Nunn and played 2 round tournament with 5 engines. Result:

Code: Select all


              
1   Stockfish 1.3 JA      10.5 - 9.511.0 - 9.011.5 - 8.514.0 - 6.0**      47.0/80
2   Naum 4                9.5 - 10.511.0 - 9.012.5 - 7.512.5 - 7.5 **     45.5/80
3   Rybka 3 1-cpu 32-bit  9.0 - 11.09.0 - 11.010.5 - 9.512.5 - 7.5  **    41.0/80
4   bright-0.4a           8.5 - 11.57.5 - 12.59.5 - 10.512.5 - 7.5   **   38.0/80
5   Fritz 10              6.0 - 14.07.5 - 12.57.5 - 12.57.5 - 12.5    **  28.5/80
Stockfish won ALL matches even if it's only engine without tablebase support (except KPK) ! Is secret awesome search depth? Games were blitz and single CPU, but gives you impression definitely.

Jouni
Top
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Stockfish endgame play: SIMPLY STUNNING

Post by mcostalba »

Jouni wrote:I took 10 endgame positions from Nunn and played 2 round tournament with 5 engines. Result:

Code: Select all


              
1   Stockfish 1.3 JA      10.5 - 9.511.0 - 9.011.5 - 8.514.0 - 6.0**      47.0/80
2   Naum 4                9.5 - 10.511.0 - 9.012.5 - 7.512.5 - 7.5 **     45.5/80
3   Rybka 3 1-cpu 32-bit  9.0 - 11.09.0 - 11.010.5 - 9.512.5 - 7.5  **    41.0/80
4   bright-0.4a           8.5 - 11.57.5 - 12.59.5 - 10.512.5 - 7.5   **   38.0/80
5   Fritz 10              6.0 - 14.07.5 - 12.57.5 - 12.57.5 - 12.5    **  28.5/80
Stockfish won ALL matches even if it's only engine without tablebase support (except KPK) ! Is secret awesome search depth? Games were blitz and single CPU, but gives you impression definitely.

Jouni
Its secrect, in my opinion, is that Glaurung end base knwoledge is fantastic, possibly the best around.

To that Stockfish added just a bit of extra speed. It would be interesting to know Glaurung 2.2 how it goes with this test. I foreseen a very good result either.
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Stockfish endgame play: SIMPLY STUNNING

Post by bob »

Jouni wrote:I took 10 endgame positions from Nunn and played 2 round tournament with 5 engines. Result:

Code: Select all


              
1   Stockfish 1.3 JA      10.5 - 9.511.0 - 9.011.5 - 8.514.0 - 6.0**      47.0/80
2   Naum 4                9.5 - 10.511.0 - 9.012.5 - 7.512.5 - 7.5 **     45.5/80
3   Rybka 3 1-cpu 32-bit  9.0 - 11.09.0 - 11.010.5 - 9.512.5 - 7.5  **    41.0/80
4   bright-0.4a           8.5 - 11.57.5 - 12.59.5 - 10.512.5 - 7.5   **   38.0/80
5   Fritz 10              6.0 - 14.07.5 - 12.57.5 - 12.57.5 - 12.5    **  28.5/80
Stockfish won ALL matches even if it's only engine without tablebase support (except KPK) ! Is secret awesome search depth? Games were blitz and single CPU, but gives you impression definitely.

Jouni
Run the _same_ test again and compare results. You might be surprised.
Top
Hart

Re: Stockfish endgame play: SIMPLY STUNNING

Post by Hart »

I would like to run a similar tournament but with far more than 10 positions. Is there a simple way to search for similar endgame positions with ChessBase to put together a larger base?
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Stockfish endgame play: SIMPLY STUNNING

Post by bob »

Hart wrote:I would like to run a similar tournament but with far more than 10 positions. Is there a simple way to search for similar endgame positions with ChessBase to put together a larger base?
10 positions is hopeless. 80 games is hopeless. The error bar is so large the final results can be anything and any program could win one of those matches...
Top
Hart

Re: Stockfish endgame play: SIMPLY STUNNING

Post by Hart »

Hence, the reason for a much larger base. In ChessBase I can select games with moves in a range of 80-120 but in some positions after 80 moves there looks to be too many pieces on the board to qualify as an endgame. Is there a more straightforward way to control the selection?
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Stockfish endgame play: SIMPLY STUNNING

Post by bob »

Hart wrote:Hence, the reason for a much larger base. In ChessBase I can select games with moves in a range of 80-120 but in some positions after 80 moves there looks to be too many pieces on the board to qualify as an endgame. Is there a more straightforward way to control the selection?
I have not found one... Can you specify max number of non-pawns or anything similar in a search? I could probably make Crafty do that and just spit out fens as it encounters the first position in any game that meets the criteria.
Top
User avatar
Mike S.
Posts: 1480
Joined: Thu Mar 09, 2006 5:33 am

Re: Stockfish endgame play: SIMPLY STUNNING

Post by Mike S. »

For comparison: Single Stockfish 1.3 solved very good, in the PET endgame test (by Peter McKenzie, known from the Lambchop engine) with 50 positions:

PET endgame test(50), CPU D945(~P4 dual)2*3.4 GHz
max. solving time: 10 sec., 5 extra plies(= max.)
all engines 32-bit, Windows Vista, ChessProgram10

Code: Select all

Engine             cores  hash(MB) solved
-----------------------------------------
Bright 0.4a&#40;2&#41;       2      256      39 <-- new 3+4+5 bitbases
Bright 0.4a&#40;1&#41;       2      256      37 <-- old 3+4 bitbases
Stockfish 1.3 MTE    1      128      33 <--
Bright 0.4a          1      128      32
Stockfish 1.3        1      128      32 <--
Rybka 2.3.2a         1      128      31
Glaurung 2.2         1      128      30
Glaurung 2.2         2      256      29
Rybka 2.2n2          1      128      28
Rybka 2.2n2          2      256      27
Stockfish 1.3        2      256      25 <-- ?!
Shredder 11 SE&#40;*)    1      128      24
Stockfish 1.3 MTE    2      256      22 <-- ?!
-----------------------------------------
*) this version does not use any endgame tables
(The mp version 1.3 has problems.)


P.S. MTE is an alternative or experimental setting, but not made with having the endgame in mind, especially. The non-default settings are:

[OPTIONS]
Knight Check Bonus=2
Mate Threat Extension (PV nodes)=2
Mate Threat Extension (non-PV nodes)=1
Passed Pawn Extension (non-PV nodes)=1
Regards, Mike
Top
User avatar
Mike S.
Posts: 1480
Joined: Thu Mar 09, 2006 5:33 am

Re: Stockfish endgame play: SIMPLY STUNNING

Post by Mike S. »

Addendum: The PET is:

Code: Select all

8/7p/2k1Pp2/pp1p2p1/3P2P1/4P3/P3K2P/8 w - - bm e4;             id "PET001&#58; Pawn endgame";
8/1p4kP/5pP1/3p4/8/4P3/7K/8 w - - bm e4;                       id "PET002&#58; Pawn endgame";
8/8/8/pPk3pp/2P1p3/1pK3P1/5PP1/8 w - - bm g4;                  id "PET003&#58; Pawn endgame";
6k1/3p4/3p4/3P4/4P1p1/6P1/8/K7 w - - bm Kb1;                   id "PET004&#58; Pawn endgame";
8/p7/1p2k1p1/8/1p2p1Pp/3pP2P/PP6/5K2 w - - bm b3;              id "PET005&#58; Pawn endgame";
8/p7/1p2k1p1/8/1p2p1Pp/3pP2P/PP6/5K2 b - - bm b3;              id "PET006&#58; Pawn endgame";
8/7p/p3pk2/1p5P/3K1P2/1P6/P7/8 w - - bm a4;                    id "PET007&#58; Pawn endgame";
8/8/1p3p2/p3k1pp/4P2P/P3K1P1/1P6/8 w - - bm g4;                id "PET008&#58; Pawn endgame";
8/8/p3k1p1/1pp5/4KP2/1P6/P5P1/8 w - - bm g4;                   id "PET009&#58; Pawn endgame";
8/1p3p2/p4kpp/1PPp4/P7/7P/5K1P/8 b - - bm Ke6 Ke7;             id "PET010&#58; Pawn endgame";
8/6p1/7p/p1N5/6PP/2k5/8/3K4 b - - bm a4;                       id "PET011&#58; pawns vs N";
8/8/8/5Bp1/7k/8/4pPKP/8 w - - bm Bg4 h3;                       id "PET012&#58; pawns vs B";
8/8/2p5/pkp3R1/7B/P7/2p3K1/8 w - - bm a4+;                     id "PET013&#58; pawns vs R+B";
8/6k1/6p1/2n2PP1/3N3p/3pK3/8/8 b - - bm d2;                    id "PET014&#58; Knight endgame";
8/1N6/1p1P1n2/p1p1p3/P1P5/1P2k3/2K5/8 w - - bm Nxc5;           id "PET015&#58; Knight endgame";
4k3/B1p4p/8/p2n1p2/5P2/5P2/1P2K2P/8 w - - am Be3;              id "PET016&#58; bishop vs knight";
8/8/p2k1p2/1p1p3p/1P1P3p/P3NPP1/5K2/1b6 w - - bm Ng2;          id "PET017&#58; bishop vs knight";
8/3k4/3b4/1K6/7p/5B1P/PP6/8 w - - bm b4;                       id "PET018&#58; opposite bishops";
8/8/4kpp1/3p1b2/p6P/2B5/6P1/6K1 b - - bm Bh3;                  id "PET019&#58; opposite bishops";
8/4k1p1/1p2B2p/4p3/8/4P2P/1PP1KbP1/8 b - - am Bxe3;            id "PET020&#58; opposite bishops";
5k2/8/4b3/1B5P/8/8/5pPP/5K2 b - - bm Bh3;                      id "PET021&#58; bishop endgame";
8/2B5/1p1p4/1PkP1p2/P4P2/5P2/1p6/bK6 w - - bm Bd8;             id "PET022&#58; bishop endgame";
8/2p1bpp1/2Pp4/p2P1P1p/2K2B1P/k5P1/8/8 w - - bm f6;            id "PET023&#58; bishop endgame";
8/4ppbk/p5pp/3pP3/3B4/5P1P/PP3P2/6K1 b - - am g5;              id "PET024&#58; bishop endgame";
4K3/2k1Bp1N/6p1/5PP1/8/7p/b7/8 w - - bm Bf6;                   id "PET025&#58; BN vs B";
8/5k2/4p3/4Pp1p/5P1P/3Rn2K/6p1/8 b - - bm g1=B;                id "PET026&#58; rook vs knight";
5k2/8/8/3R4/6K1/8/3b2pP/8 w - - bm Rf5+;                       id "PET027&#58; rook vs bishop";
8/4kp2/4p1p1/2p1r3/PpP5/3R4/1P1K1PP1/8 w - - bm g4;            id "PET028&#58; rook endgame";
1r3k2/5pp1/3p2p1/8/3P4/P6P/2R2P1K/8 b - - bm Ra8;              id "PET029&#58; rook endgame";
5k2/p1p4R/1pr5/3p1pP1/P2P1P2/2P2K2/8/8 w - - bm Kg3;           id "PET030&#58; rook endgame";
8/1r3pkp/8/5p2/8/5PP1/1P1R3P/6K1 b - - bm f4;                  id "PET031&#58; rook endgame";
8/5pk1/r5pp/P7/3R3P/6P1/5PK1/8 w - - bm Ra4;                   id "PET032&#58; rook endgame";
8/4k3/2P2p2/6p1/1P4R1/4K3/7r/8 w - - bm Rd4;                   id "PET033&#58; rook endgame";
3r4/7p/Rp4k1/5p2/4p3/2P5/PP3P1P/5K2 b - - bm Rd2;              id "PET034&#58; rook endgame";
8/8/p2k3p/1P1p1pp1/P1r5/1R1K1PP1/7P/8 b - - bm a5;             id "PET035&#58; rook endgame";
8/1r3pkp/R7/6Pp/8/4P3/5PK1/8 b - - bm Re7;                     id "PET036&#58; rook endgame";
8/pR4pk/1b6/2p5/N1p5/8/PP1r2PP/6K1 b - - bm Rxb2;              id "PET037&#58; RN vs RB";
4k3/2p1b3/4p1p1/1pp1P3/5PP1/1PBK4/r1P2R2/8 b - - bm c4+;       id "PET038&#58; RB vs RB";
1r6/1pb1k1p1/4p2p/1p1p4/3Pp2P/1R2P1PB/1P2PK2/8 b - - bm b4;    id "PET039&#58; RB vs RB";
7r/3bk3/2p2p2/2P1P3/1P3P2/6Kp/R7/5B2 w - - bm e6;              id "PET040&#58; RB vs RB";
1r6/4k3/r2p2p1/2pR1p1p/2P1pP1P/pPK1P1P1/P7/1B6 b - - bm Rxb3+; id "PET041&#58; RR vs RB";
8/8/5P1k/8/2K5/pr2r3/4R3/2R5 w - - bm f7;                      id "PET042&#58; double rook endgame";
r5k1/5pp1/1P5p/3R4/2r5/P3P3/2p2PPP/2R3K1 b - - bm Rxa3;        id "PET043&#58; double rook endgame";
1r6/Rp2rp2/1Pp2kp1/N1Pp3p/3Pp1nP/4P1P1/R4P2/6K1 w - - bm Nxb7 Rxb7; id "PET044&#58; RRN vs RRN";
5k2/4p3/3p2Q1/3Pq1pP/5pP1/2P4K/8/8 w - - bm Qf5+;              id "PET045&#58; Queen endgame";
8/8/1P1k2p1/8/5P1p/4Qb2/1q4PK/8 w - - bm Qe5+;                 id "PET046&#58; Q+B vs Q";
8/p4k2/1p1p1qp1/1P1P4/P5P1/7P/4Q1K1/2b5 b - - am Qb2;          id "PET047&#58; Q+B vs Q";
6qk/1p4pn/8/3p4/2P5/p3P1P1/P1Q3K1/1B6 w - - am Qxh7+;          id "PET048&#58; Q+B vs Q+N";
1Q6/Nn3pbp/2N2p2/1Pqk1P1P/8/8/6P1/7K w - - am Qxb7;            id "PET049&#58; QNN vs QBN";
7k/7p/1p4p1/n2p4/1K1p3R/2NB4/3Q1P2/q1r5 w - - bm Rxh7+;        id "PET050&#58; QNBR vs QNR";
The alternative solution Rxb7 to #44 was added after the test with the other engines (see posting above) but before the Stockfish 1.3 tests, which I did last.

You can copy and paste this into any text editor, and save it as PET.epd. Fritz users can create a new empty database and append that EPD to it (see Edit menu in the F12 database screen), and then use it as a testsuite (see Extras/Analysis...).
Regards, Mike
Top
Jouni
Posts: 3232
Joined: Wed Mar 08, 2006 8:15 pm

Re: Stockfish endgame play: SIMPLY STUNNING

Post by Jouni »

I don't believe You need 10 000 games to verify difference. If You can't see it with less games, it's not significant at all. I played same 5 engines
in 2 round tournament with 3 totally different books. Very little difference
in points after 80 games, see here!

Code: Select all


book1
             
1   Rybka 3 1-cpu 32-bit  11.5 - 8.516.0 - 4.017.0 - 3.017.0 - 3.0**      61.5/80
2   Naum 4                8.5 - 11.513.5 - 6.513.5 - 6.515.0 - 5.0 **     50.5/80
3   bright-0.4a           4.0 - 16.06.5 - 13.511.5 - 8.510.5 - 9.5  **    32.5/80
4   Stockfish 1.3 JA      3.0 - 17.06.5 - 13.58.5 - 11.510.0 - 10.0   **   28.0/80
5   Fritz 10              3.0 - 17.05.0 - 15.09.5 - 10.510.0 - 10.0    **  27.5/80

book2

               
1   Rybka 3 1-cpu 32-bit  12.0 - 8.018.0 - 2.016.0 - 4.017.0 - 3.0**      63.0/80
2   Naum 4                8.0 - 12.013.0 - 7.016.0 - 4.013.0 - 7.0 **     50.0/80
3   Fritz 10              2.0 - 18.07.0 - 13.011.5 - 8.59.5 - 10.5  **    30.0/80
4   bright-0.4a           4.0 - 16.04.0 - 16.08.5 - 11.512.5 - 7.5   **   29.0/80
5   Stockfish 1.3 JA      3.0 - 17.07.0 - 13.010.5 - 9.57.5 - 12.5    **  28.0/80


book3
                
1   Rybka 3 1-cpu 32-bit  14.0 - 6.016.0 - 4.016.5 - 3.515.5 - 4.5**      62.0/80
2   Naum 4                6.0 - 14.013.0 - 7.015.5 - 4.514.0 - 6.0 **     48.5/80
3   Stockfish 1.3 JA      4.0 - 16.07.0 - 13.011.0 - 9.010.5 - 9.5  **    32.5/80
4   bright-0.4a           3.5 - 16.54.5 - 15.59.0 - 11.012.5 - 7.5   **   29.5/80
5   Fritz 10              4.5 - 15.56.0 - 14.09.5 - 10.57.5 - 12.5    **  27.5/80


Jouni
Top

Return to “Computer Chess Club: General Topics”