TalkChess.com

Stockfish 11 disagrees with some of the results reported at https://www.chessprogramming.org/Chess960_Perft_Results

Here is a list showing what the wiki has followed by the results from SF where the *s indicate differences:

nrkb1qnr/ppppp1p1/6bp/5p2/1PP1P1P1/8/P2P1P1P/NRKBBQNR w HBhb - 1 9 ; 32 ; 761 ; 24586 ; 632916 ; 20671433 ; 568524724
nrkb1qnr/ppppp1p1/6bp/5p2/1PP1P1P1/8/P2P1P1P/NRKBBQNR w HBhb - 1 9 ; 32 ; 761 ; 24586 ; 632916 ; *20671639 ; *568530649

nrk1bbnr/p1q1pppp/1ppp4/8/3P3P/4K3/PPP1PPP1/NR1QBBNR w hb - 0 9 ; 30 ; 719 ; 21683 ; 541389 ; 16278120 ; 423649784
nrk1bbnr/p1q1pppp/1ppp4/8/3P3P/4K3/PPP1PPP1/NR1QBBNR w hb - 0 9 ; 30 ; 719 ; 21683 ; 541389 ; 16278120 ; *423649784

nrknbbqr/pp3p1p/B3p1p1/2pp4/4P3/2N3P1/PPPP1P1P/NRK1B1QR w HBhb - 0 9 ; 29 ; 683 ; 19755 ; 501807 ; 14684565 ; 394951291
nrknbbqr/pp3p1p/B3p1p1/2pp4/4P3/2N3P1/PPPP1P1P/NRK1B1QR w HBhb - 0 9 ; 29 ; 683 ; 19755 ; 501807 ; *14684581 ; *394951813

nrk2bbr/pppqpppp/3p4/8/1P3nP1/3P4/P1P1PP1P/NRKNQBBR w HBhb - 1 9 ; 24 ; 814 ; 19954 ; 670162 ; 17603960 ; 592121050
nrk2bbr/pppqpppp/3p4/8/1P3nP1/3P4/P1P1PP1P/NRKNQBBR w HBhb - 1 9 ; 24 ; 814 ; 19954 ; 670162 ; *17603963 ; *592121279

nrknqrbb/1p2ppp1/2pp4/Q6p/P2P3P/8/1PP1PPP1/NRKN1RBB w FBfb - 0 9 ; 34 ; 513 ; 16111 ; 303908 ; 9569590 ; 206509331
nrknqrbb/1p2ppp1/2pp4/Q6p/P2P3P/8/1PP1PPP1/NRKN1RBB w FBfb - 0 9 ; 34 ; 513 ; 16111 ; *303945 ; *9570555 ; *206564265

nrkbnrbq/ppppppp1/8/8/7p/PP3P2/2PPPRPP/NRKBN1BQ w Bfb - 0 9 ; 16 ; 353 ; 6189 ; 156002 ; 3008668 ; 82706705
nrkbnrbq/ppppppp1/8/8/7p/PP3P2/2PPPRPP/NRKBN1BQ w Bfb - 0 9 ; 16 ; 353 ; 6189 ; 156002 ; *3008674 ; *82706885

nrknrbbq/p4ppp/2p1p3/1p1p4/1P2P3/2P5/P1NP1PPP/1RKNRBBQ w EBeb - 0 9 ; 29 ; 728 ; 21915 ; 587668 ; 18231199 ; 511686397
nrknrbbq/p4ppp/2p1p3/1p1p4/1P2P3/2P5/P1NP1PPP/1RKNRBBQ w EBeb - 0 9 ; 29 ; 728 ; 21915 ; 587668 ; *18231203 ; *511686549

nrknr1bb/pppp1p2/7p/2qPp1p1/8/1P5P/P1P1PPP1/NRKNRQBB w EBeb - 0 9 ; 20 ; 714 ; 14336 ; 500458 ; 11132758 ; 386064577
nrknr1bb/pppp1p2/7p/2qPp1p1/8/1P5P/P1P1PPP1/NRKNRQBB w EBeb - 0 9 ; 20 ; 714 ; 14336 ; 500458 ; *11132835 ; *386066686

nrk1brnq/pp1p1pp1/7p/b1p1p3/1P6/6P1/P1PPPPQP/NRKBBRN1 w FBfb - 2 9 ; 29 ; 675 ; 20352 ; 492124 ; 15316285 ; 389051744
nrk1brnq/pp1p1pp1/7p/b1p1p3/1P6/6P1/P1PPPPQP/NRKBBRN1 w FBfb - 2 9 ; 29 ; 675 ; 20352 ; *492169 ; *15317759 ; *389125787

nrkbrnbq/4pppp/1ppp4/p7/2P1P3/3P2N1/PP3PPP/NRKBR1BQ w EBeb - 0 9 ; 29 ; 591 ; 17132 ; 384358 ; 11245508 ; 270967202
nrkbrnbq/4pppp/1ppp4/p7/2P1P3/3P2N1/PP3PPP/NRKBR1BQ w EBeb - 0 9 ; 29 ; 591 ; 17132 ; 384358 ; *11245548 ; *270968394

nr1rnqbb/ppp1pp1p/3k2p1/3p4/1P5P/3P1N2/P1P1PPP1/NRKR1QBB w DB - 1 9 ; 25 ; 758 ; 18547 ; 543643 ; 13890077 ; 402109399
nr1rnqbb/ppp1pp1p/3k2p1/3p4/1P5P/3P1N2/P1P1PPP1/NRKR1QBB w DB - 1 9 ; 25 ; 758 ; 18547 ; 543643 ; *13890131 ; *402111471

qrkn1rbb/pp2pppp/2p5/3p4/P2Qn1P1/1P6/2PPPP1P/1RKNNRBB w FBfb - 0 9 ; 38 ; 943 ; 35335 ; 868165 ; 31909835 ; 798405123
qrkn1rbb/pp2pppp/2p5/3p4/P2Qn1P1/1P6/2PPPP1P/1RKNNRBB w FBfb - 0 9 ; 38 ; 943 ; 35335 ; *868166 ; *31909865 ; *798409577

brk1nbrq/1ppppn1p/6p1/p4p2/P5P1/5R2/1PPPPP1P/BRKNNB1Q w Bgb - 0 9 ; 29 ; 922 ; 27709 ; 879527 ; 27463717 ; 888881062
brk1nbrq/1ppppn1p/6p1/p4p2/P5P1/5R2/1PPPPP1P/BRKNNB1Q w Bgb - 0 9 ; 29 ; 922 ; 27709 ; *879545 ; *27464736 ; *888960660

1rknrbbn/p1pp1p1p/8/1p2p1p1/4qPP1/2P5/PP1PP1BP/QRKNR1BN w EBeb - 0 9 ; 28 ; 1309 ; 36355 ; 1568968 ; 44576409 ; 1846382333
1rknrbbn/p1pp1p1p/8/1p2p1p1/4qPP1/2P5/PP1PP1BP/QRKNR1BN w EBeb - 0 9 ; 28 ; 1309 ; 36355 ; 1568968 ; *44576411 ; *1846382409

brk1nbrn/pp1ppppp/2p5/7P/5P2/q2P4/PPP1P1P1/BRKQNBRN w GBgb - 1 9 ; 15 ; 471 ; 8716 ; 276424 ; 5960901 ; 190316951
brk1nbrn/pp1ppppp/2p5/7P/5P2/q2P4/PPP1P1P1/BRKQNBRN w GBgb - 1 9 ; 15 ; 471 ; 8716 ; 276424 ; *5960947 ; *190318307

brknq1nb/pp2prpp/8/2pP1p2/6P1/2N5/PPPP1P1P/BRK1QRNB w FBb - 1 9 ; 33 ; 830 ; 27897 ; 764915 ; 26262884 ; 765831403
brknq1nb/pp2prpp/8/2pP1p2/6P1/2N5/PPPP1P1P/BRK1QRNB w FBb - 1 9 ; 33 ; 830 ; 27897 ; 764915 ; *26262909 ; *765832147

brknrbq1/1p1p1ppp/p3p1n1/2p5/8/1P1BPP2/P1PP2PP/BRKNR1QN w EBeb - 0 9 ; 36 ; 786 ; 27868 ; 655019 ; 22852433 ; 577223409
brknrbq1/1p1p1ppp/p3p1n1/2p5/8/1P1BPP2/P1PP2PP/BRKNR1QN w EBeb - 0 9 ; 36 ; 786 ; 27868 ; 655019 ; *22852434 ; *577223443

brkbnr2/1ppppp1p/7n/p5N1/P2q4/8/1PPPPPPP/BRKBNRQ1 w FBfb - 1 9 ; 22 ; 869 ; 19234 ; 679754 ; 16453359 ; 567287944
brkbnr2/1ppppp1p/7n/p5N1/P2q4/8/1PPPPPPP/BRKBNRQ1 w FBfb - 1 9 ; 22 ; 869 ; 19234 ; 679754 ; *16453369 ; *567288190

1rkbrqnn/p1pp1ppp/1p6/8/P2Pp3/8/1PPKPPQP/BR1BR1NN w eb - 0 9 ; 28 ; 916 ; 24892 ; 817624 ; 22840279 ; 759318058
1rkbrqnn/p1pp1ppp/1p6/8/P2Pp3/8/1PPKPPQP/BR1BR1NN w eb - 0 9 ; 28 ; 916 ; 24892 ; *817627 ; *22840366 ; *759329118

brkbrnqn/ppp2p2/4p3/P2p2pp/6P1/5P2/1PPPP2P/BRKBRNQN w EBeb - 0 9 ; 25 ; 548 ; 14563 ; 348259 ; 9688526 ; 247750144
brkbrnqn/ppp2p2/4p3/P2p2pp/6P1/5P2/1PPPP2P/BRKBRNQN w EBeb - 0 9 ; 25 ; 548 ; 14563 ; *348261 ; *9688570 ; *247755418

brkr1qnb/pppp2pp/2B1p3/5p2/2n5/6PP/PPPPPPN1/BRKR1QN1 w DBdb - 1 9 ; 27 ; 854 ; 23303 ; 741626 ; 20558538 ; 667089231
brkr1qnb/pppp2pp/2B1p3/5p2/2n5/6PP/PPPPPPN1/BRKR1QN1 w DBdb - 1 9 ; 27 ; 854 ; 23303 ; 741626 ; *20558544 ; *667089439

brkbrnn1/pp1pppp1/7q/2p5/6Pp/4P1NP/PPPP1P2/BRKBR1NQ w EBeb - 2 9 ; 30 ; 978 ; 29593 ; 942398 ; 29205057 ; 936568065
brkbrnn1/pp1pppp1/7q/2p5/6Pp/4P1NP/PPPP1P2/BRKBR1NQ w EBeb - 2 9 ; 30 ; 978 ; 29593 ; 942398 ; *29205065 ; *936574810

brkrnb1q/pp1p1ppp/2p1p3/5n2/1P6/5N1N/P1PPPPPP/BRKR1B1Q w DBdb - 1 9 ; 31 ; 897 ; 27830 ; 810187 ; 25423729 ; 755334868
brkrnb1q/pp1p1ppp/2p1p3/5n2/1P6/5N1N/P1PPPPPP/BRKR1B1Q w DBdb - 1 9 ; 31 ; 897 ; 27830 ; 810187 ; *25423794 ; *755337544

Note: There might be more as I was only running the tests through depth 5, unless it found a discrepancy, in which case I also ran depth 6.

For what it's worth, I tried the first 6 positions in my engine and it always matched the second line (matched the *s) so my guess is that one is correct.

(I did find out that I don't automatically assume FRC mode so numbers were different until I did new FRC game due to lacking castling rights, but then they matched neither numbers so discrepancy must be due to something else.)

Minic is matching first line (without the *).

I recently struggle to get castling right in FRC, maybe there are still mistakes.

Considering the case where king and rook does not move is something.
Also do not forget to allow that king can be at destination square of rook, and rook at the destination square of king ... which is tricky.

zenpawn wrote: ↑Mon May 11, 2020 2:25 am Stockfish 11 disagrees with some of the results reported at https://www.chessprogramming.org/Chess960_Perft_Results

That's because you're doing something wrong!
Can you figure it out?

Hint: do you SF expect to know that you are giving a Chess960 fen?

Rubi switches to FRC/Chess960 automatically if it gets X-FEN castle chars. Makes things like this easier. At least if you only support normal chess and FRC and no other variants.

I wonder if you all prefer an auto tool that could find out if a Pertf of an engine (for a given position) correct or not then narrows down to print out the leave positions in which the engine may have problems (disagreement)?

Joerg Oster wrote: ↑Mon May 11, 2020 10:06 am

zenpawn wrote: ↑Mon May 11, 2020 2:25 am Stockfish 11 disagrees with some of the results reported at https://www.chessprogramming.org/Chess960_Perft_Results

That's because you're doing something wrong!
Can you figure it out?

Hint: do you SF expect to know that you are giving a Chess960 fen?

If that were the case, it wouldn't have matched any of them.

Let's go with the first position I flagged above:
The source of the results on the wiki is Ethereal. FWIW, my engine matches Stockfish's results, which is how I found these in the first place.

zenpawn wrote: ↑Mon May 11, 2020 11:53 am

Code: Select all

setoption true UCI_Chess960
 

Try correct UCI syntax:
setoption name UCI_Chess960 value true

RubiChess wrote: ↑Mon May 11, 2020 12:12 pm

zenpawn wrote: ↑Mon May 11, 2020 11:53 am

Code: Select all

setoption true UCI_Chess960
 

Try correct UCI syntax:
setoption name UCI_Chess960 value true

Crap! That way, it does indeed report 20671433.

Guess we can delete this post.

I too auto assume FRC if there are castle-eligible rooks that are not on A1/A8/H1/H8.
I did not know that this dataset made its way to the wiki. Cool to know, and hopefully useful.