We need to scrap Perft()

Discussion of chess software programming and technical issues.

Moderator: Ras

R. Tomasi
Posts: 307
Joined: Wed Sep 01, 2021 4:08 pm
Location: Germany
Full name: Roland Tomasi

Re: We need to scrap Perft()

Post by R. Tomasi »

These are my preliminary results for kiwi-pete:

Code: Select all

set-fen r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0

New position: r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0

cperft 1

  g2h3
  d5e6
  e2a6
  e5g6
  e5d7
  e5f7
  f3f6
  f3h3

depth:  1 nodes:   8.00 N

 Leafs:             8
 Captures:          8
 En passant:        0
 Castles:           0
 Promotions:        0
 Checks:            0
 Checkmates:        0
 Double pushes:     0
 Queenside castles: 0
 Kingside castles:  0

cperft 2

  g2h3  : 7
  d5e6  : 7
  e2a6  : 6
  e5g6  : 8
  e5d7  : 11
  e5f7  : 9
  f3f6  : 7
  f3h3  : 7
  e1g1  : 7
  e1c1  : 7
  a2a4  : 9
  g2g4  : 7
  c3b1  : 6
  c3d1  : 6
  c3a4  : 7
  c3b5  : 6
  e5d3  : 7
  e5c4  : 8
  e5g4  : 8
  e5c6  : 8
  a2a3  : 8
  b2b3  : 7
  g2g3  : 6
  d5d6  : 6
  a1b1  : 7
  a1c1  : 7
  a1d1  : 7
  h1f1  : 7
  h1g1  : 7
  f3d3  : 7
  f3e3  : 7
  f3g3  : 7
  f3f4  : 7
  f3f5  : 9
  d2c1  : 7
  d2e3  : 7
  d2f4  : 7
  d2g5  : 7
  d2h6  : 9
  e2d1  : 6
  e2f1  : 7
  e2d3  : 7
  e2c4  : 8
  e2b5  : 7
  f3g4  : 8
  f3h5  : 10
  e1d1  : 7
  e1f1  : 7

depth:  2 nodes:   96.0 N

 Leafs:             351
 Captures:          351
 En passant:        1
 Castles:           0
 Promotions:        0
 Checks:            3
 Checkmates:        0
 Double pushes:     0
 Queenside castles: 0
 Kingside castles:  0

cperft 3

  g2h3  : 285
  d5e6  : 428
  e2a6  : 274
  e5g6  : 306
  e5d7  : 324
  e5f7  : 277
  f3f6  : 432
  f3h3  : 320
  e1g1  : 368
  e1c1  : 367
  a2a4  : 379
  g2g4  : 319
  c3b1  : 392
  c3d1  : 392
  c3a4  : 432
  c3b5  : 401
  e5d3  : 235
  e5c4  : 228
  e5g4  : 279
  e5c6  : 342
  a2a3  : 418
  b2b3  : 360
  g2g3  : 278
  d5d6  : 385
  a1b1  : 368
  a1c1  : 368
  a1d1  : 368
  h1f1  : 368
  h1g1  : 369
  f3d3  : 316
  f3e3  : 367
  f3g3  : 367
  f3f4  : 324
  f3f5  : 464
  d2c1  : 367
  d2e3  : 408
  d2f4  : 324
  d2g5  : 402
  d2h6  : 387
  e2d1  : 338
  e2f1  : 379
  e2d3  : 357
  e2c4  : 349
  e2b5  : 370
  f3g4  : 407
  f3h5  : 397
  e1d1  : 361
  e1f1  : 356

depth:  3 nodes:   2.14 kN

 Leafs:             17102
 Captures:          17102
 En passant:        45
 Castles:           0
 Promotions:        0
 Checks:            777
 Checkmates:        1
 Double pushes:     0
 Queenside castles: 0
 Kingside castles:  0

cperft 4

  g2h3  : 14544
  d5e6  : 16538
  e2a6  : 12680
  e5g6  : 16601
  e5d7  : 21915
  e5f7  : 18350
  f3f6  : 15125
  f3h3  : 17277
  e1g1  : 15667
  e1c1  : 14505
  a2a4  : 18003
  g2g4  : 13488
  c3b1  : 13803
  c3d1  : 13852
  c3a4  : 16725
  c3b5  : 14126
  e5d3  : 13323
  e5c4  : 15132
  e5g4  : 14910
  e5c6  : 16861
  a2a3  : 18451
  b2b3  : 14779
  g2g3  : 12486
  d5d6  : 13487
  a1b1  : 15030
  a1c1  : 14980
  a1d1  : 14479
  h1f1  : 14767
  h1g1  : 15104
  f3d3  : 15232
  f3e3  : 16550
  f3g3  : 16859
  f3f4  : 16284
  f3f5  : 21874
  d2c1  : 14826
  d2e3  : 16338
  d2f4  : 15204
  d2g5  : 16471
  d2h6  : 18169
  e2d1  : 12130
  e2f1  : 15853
  e2d3  : 15545
  e2c4  : 17231
  e2b5  : 15349
  f3g4  : 18005
  f3h5  : 21605
  e1d1  : 14512
  e1f1  : 13973

depth:  4 nodes:  100.0 kN

 Leafs:             758998
 Captures:          757163
 En passant:        1929
 Castles:           0
 Promotions:        9667
 Checks:            21808
 Checkmates:        43
 Double pushes:     0
 Queenside castles: 0
 Kingside castles:  0

cperft 5

  g2h3  : 587193
  d5e6  : 906592
  e2a6  : 546412
  e5g6  : 628936
  e5d7  : 702356
  e5f7  : 601241
  f3f6  : 811865
  f3h3  : 738834
  e1g1  : 764112
  e1c1  : 697335
  a2a4  : 803098
  g2g4  : 599927
  c3b1  : 774802
  c3d1  : 776984
  c3a4  : 922214
  c3b5  : 824017
  e5d3  : 490559
  e5c4  : 479500
  e5g4  : 539339
  e5c6  : 712328
  a2a3  : 910233
  b2b3  : 709572
  g2g3  : 554960
  d5d6  : 756218
  a1b1  : 727832
  a1c1  : 728110
  a1d1  : 698615
  h1f1  : 713892
  h1g1  : 749350
  f3d3  : 656471
  f3e3  : 792976
  f3g3  : 825372
  f3f4  : 735213
  f3f5  : 1049250
  d2c1  : 714460
  d2e3  : 859600
  d2f4  : 677032
  d2g5  : 833151
  d2h6  : 766991
  e2d1  : 601631
  e2f1  : 778227
  e2d3  : 738691
  e2c4  : 737175
  e2b5  : 746630
  f3g4  : 864621
  f3h5  : 866822
  e1d1  : 684452
  e1f1  : 659702

depth:  5 nodes:   4.19 MN

 Leafs:             35044893
 Captures:          35043416
 En passant:        73365
 Castles:           0
 Promotions:        3961
 Checks:            2503771
 Checkmates:        29893
 Double pushes:     0
 Queenside castles: 0
 Kingside castles:  0
R. Tomasi
Posts: 307
Joined: Wed Sep 01, 2021 4:08 pm
Location: Germany
Full name: Roland Tomasi

Re: We need to scrap Perft()

Post by R. Tomasi »

So, assuming we have now two reference implementations that agree on the same numbers, there remain the open questions:

1.) Underpromotions: yes/no?
2.) "cperft" or a different name for the command, such that it would not be confused with a checking-moves perft that could be done in similar spirit?
Last edited by R. Tomasi on Sat Dec 18, 2021 5:15 pm, edited 1 time in total.
Mergi
Posts: 127
Joined: Sat Aug 21, 2021 9:55 pm
Full name: Jen

Re: We need to scrap Perft()

Post by Mergi »

R. Tomasi wrote: Sat Dec 18, 2021 5:06 pm These are my preliminary results for kiwi-pete:

Code: Select all

cperft 4
 Leafs:             758998
I give up, i'm getting 751289 here.
R. Tomasi
Posts: 307
Joined: Wed Sep 01, 2021 4:08 pm
Location: Germany
Full name: Roland Tomasi

Re: We need to scrap Perft()

Post by R. Tomasi »

Mergi wrote: Sat Dec 18, 2021 5:15 pm
R. Tomasi wrote: Sat Dec 18, 2021 5:06 pm These are my preliminary results for kiwi-pete:

Code: Select all

cperft 4
 Leafs:             758998
I give up, i'm getting 751289 here.
So much for having two agreeing implementations... :cry:
Mergi
Posts: 127
Joined: Sat Aug 21, 2021 9:55 pm
Full name: Jen

Re: We need to scrap Perft()

Post by Mergi »

R. Tomasi wrote: Sat Dec 18, 2021 5:16 pm
Mergi wrote: Sat Dec 18, 2021 5:15 pm
R. Tomasi wrote: Sat Dec 18, 2021 5:06 pm These are my preliminary results for kiwi-pete:

Code: Select all

cperft 4
 Leafs:             758998
I give up, i'm getting 751289 here.
So much for having two agreeing implementations... :cry:
Nevermind. Got it now.

758998

Stopped trying to be clever with generating only certain moves and went fool-proof :D

Code: Select all

    uint64_t QPerft(Board &board) {
        MoveGen move_gen(board);
        uint64_t total_nodes = 0;
        while (Move m = move_gen.GetAnyMove()) {
            if (!board.MakeMove(m)) {
                board.UnmakeMove(m);
                continue;
            }
            if (board.CapturedPiece() || GetMoveType(m) == PROMOTE_QUEEN) {
                ++total_nodes;
            }
            board.UnmakeMove(m);
        }
        return total_nodes;
    }

R. Tomasi
Posts: 307
Joined: Wed Sep 01, 2021 4:08 pm
Location: Germany
Full name: Roland Tomasi

Re: We need to scrap Perft()

Post by R. Tomasi »

Mergi wrote: Sat Dec 18, 2021 5:24 pm Nevermind. Got it now.
Excellent :D
Mergi
Posts: 127
Joined: Sat Aug 21, 2021 9:55 pm
Full name: Jen

Re: We need to scrap Perft()

Post by Mergi »

Here's my results for perft 4 and 5 if you want to verify it later. Sadly doing it like this is way too slow (barely 10m nodes/s), so anything more is out of reach for me.

Honestly though, i fail to see the usefulness of this. QSearch generators for most engines are way too different to be able to have some general testing method. For example, what about check evasions? My generator does generate all quiet moves, if in check. Everyone has a slightly different approach. Testing only captures with this "cperft" seems rather pointless.

Code: Select all

perft 4

f3f6: 811865
f3h3: 738834
e2a6: 546412
e5f7: 601241
e5d7: 702356
e5g6: 628936
d5e6: 906592
g2h3: 587193
f3h5: 866822
f3f5: 1049250
f3g4: 864621
f3f4: 735213
f3g3: 825372
f3e3: 792976
f3d3: 656471
h1g1: 749350
h1f1: 713892
a1d1: 698615
a1c1: 728110
a1b1: 727832
e2b5: 746630
e2c4: 737175
e2d3: 738691
e2f1: 778227
e2d1: 601631
d2h6: 766991
d2g5: 833151
d2f4: 677032
d2e3: 859600
d2c1: 714460
e5c6: 712328
e5g4: 539339
e5c4: 479500
e5d3: 490559
c3b5: 824017
c3a4: 922214
c3d1: 776984
c3b1: 774802
e1f1: 659702
e1d1: 684452
e1c1: 697335
e1g1: 764112
g2g4: 599927
a2a4: 803098
d5d6: 756218
g2g3: 554960
b2b3: 709572
a2a3: 910233

Nodes total: 35044893



perft 5

f3f6: 29739654
f3h3: 38619482
e2a6: 25222381
e5f7: 36738818
e5d7: 43700095
e5g6: 33588741
d5e6: 36805812
g2h3: 29182687
f3h5: 44275343
f3f5: 48325023
f3g4: 38328527
f3f4: 34944559
f3g3: 37620160
f3e3: 35921407
f3d3: 31493285
h1g1: 31192332
h1f1: 29763377
a1d1: 28872053
a1c1: 30538217
a1b1: 30779653
e2b5: 31712703
e2c4: 35361043
e2d3: 32431650
e2f1: 33329584
e2d1: 23307756
d2h6: 35632897
d2g5: 35642720
d2f4: 31581775
d2e3: 35564052
d2c1: 29999102
e5c6: 34783151
e5g4: 27368675
e5c4: 28601176
e5d3: 25231542
c3b5: 31099353
c3a4: 37068080
c3d1: 29237608
c3b1: 29172905
e1f1: 26489041
e1d1: 28728047
e1c1: 28820961
e1g1: 32932256
g2g4: 25128757
a2a4: 37909363
d5d6: 28004168
g2g3: 24489562
b2b3: 29630049
a2a3: 40345981

Nodes total: 1565255563
R. Tomasi
Posts: 307
Joined: Wed Sep 01, 2021 4:08 pm
Location: Germany
Full name: Roland Tomasi

Re: We need to scrap Perft()

Post by R. Tomasi »

Mergi wrote: Sat Dec 18, 2021 5:38 pm Here's my results for perft 4 and 5 if you want to verify it later. Sadly doing it like this is way too slow (barely 10m nodes/s), so anything more is out of reach for me.

Honestly though, i fail to see the usefulness of this. QSearch generators for most engines are way too different to be able to have some general testing method. For example, what about check evasions? My generator does generate all quiet moves, if in check. Everyone has a slightly different approach. Testing only captures with this "cperft" seems rather pointless.

Code: Select all

perft 4

f3f6: 811865
f3h3: 738834
e2a6: 546412
e5f7: 601241
e5d7: 702356
e5g6: 628936
d5e6: 906592
g2h3: 587193
f3h5: 866822
f3f5: 1049250
f3g4: 864621
f3f4: 735213
f3g3: 825372
f3e3: 792976
f3d3: 656471
h1g1: 749350
h1f1: 713892
a1d1: 698615
a1c1: 728110
a1b1: 727832
e2b5: 746630
e2c4: 737175
e2d3: 738691
e2f1: 778227
e2d1: 601631
d2h6: 766991
d2g5: 833151
d2f4: 677032
d2e3: 859600
d2c1: 714460
e5c6: 712328
e5g4: 539339
e5c4: 479500
e5d3: 490559
c3b5: 824017
c3a4: 922214
c3d1: 776984
c3b1: 774802
e1f1: 659702
e1d1: 684452
e1c1: 697335
e1g1: 764112
g2g4: 599927
a2a4: 803098
d5d6: 756218
g2g3: 554960
b2b3: 709572
a2a3: 910233

Nodes total: 35044893



perft 5

f3f6: 29739654
f3h3: 38619482
e2a6: 25222381
e5f7: 36738818
e5d7: 43700095
e5g6: 33588741
d5e6: 36805812
g2h3: 29182687
f3h5: 44275343
f3f5: 48325023
f3g4: 38328527
f3f4: 34944559
f3g3: 37620160
f3e3: 35921407
f3d3: 31493285
h1g1: 31192332
h1f1: 29763377
a1d1: 28872053
a1c1: 30538217
a1b1: 30779653
e2b5: 31712703
e2c4: 35361043
e2d3: 32431650
e2f1: 33329584
e2d1: 23307756
d2h6: 35632897
d2g5: 35642720
d2f4: 31581775
d2e3: 35564052
d2c1: 29999102
e5c6: 34783151
e5g4: 27368675
e5c4: 28601176
e5d3: 25231542
c3b5: 31099353
c3a4: 37068080
c3d1: 29237608
c3b1: 29172905
e1f1: 26489041
e1d1: 28728047
e1c1: 28820961
e1g1: 32932256
g2g4: 25128757
a2a4: 37909363
d5d6: 28004168
g2g3: 24489562
b2b3: 29630049
a2a3: 40345981

Nodes total: 1565255563
FWIW our numbers agree. I'll let others decide usefulness or not. Having a table with these numbers would have helped me quite a bit when writing my movegen.
User avatar
Ras
Posts: 2696
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: We need to scrap Perft()

Post by Ras »

Mergi wrote: Sat Dec 18, 2021 5:38 pmFor example, what about check evasions?
Since that should also be part of main search, regular perft covers that aspect. That's also why I don't really see the need for a checkperft. Without checkperft, we could stay with cperft for the capture / QS test.
Rasmus Althoff
https://www.ct800.net
Mergi
Posts: 127
Joined: Sat Aug 21, 2021 9:55 pm
Full name: Jen

Re: We need to scrap Perft()

Post by Mergi »

Ras wrote: Sat Dec 18, 2021 6:18 pm
Mergi wrote: Sat Dec 18, 2021 5:38 pmFor example, what about check evasions?
Since that should also be part of main search, regular perft covers that aspect. That's also why I don't really see the need for a checkperft. Without checkperft, we could stay with cperft for the capture / QS test.
But what you want to do with cperft is also done for the most part in regular perft as well. Captures are already displayed on the chess wiki for most positions. The only differene is that in cperft you add to it non-capture queen promotions, which you cannot infer from the wiki (non-capture promotions are not distinguishable in the tables there).

If you want to test only your very specific qsearch generator, all you have to do is first explore that position with your normal movegen, enumerate all the captures, promotions and non-capture promotions, and then explore the same position with the specialized movegen and compare the numbers.

It only way this would kind of make sense to me is if you didn't stop in ply + 1, but instead explored all the captures and promotions fully.