Are the perft results wrong on the CPW?

[Sapling]

First time poster here, when I was building Sapling I really enjoyed coding & optimizing the move generator, ofc it's bearing on the strength of an engine is very minimal but, Afterall I was building the engine primarily for fun.

I've just started a new project called 'The Grand Chess Tree' - I'll create another post about it when it's in a more polished state, but suffice to say it's a specialized move generation engine.

So far I've traversed to perft(9), but I'm noticing from depth 7 the discovered checks and mates are not matching the results in the CPW for the initial position

depth(7) all match except mates:
mine vs cpw
mates: 435,765 vs 435,767
then for depth(8) the discovered start to diverge also:
discovered: 847,043 vs 847,039
mates: 9,851,894 vs 9,852,036

One thing to note is that the checks, discovered checks and mates are all calculated separately from the move generation, and all other counts seem to match which indicates a miss match in the classification of a leaf node.

I'm wondering if anyone else has ran a perft with discovered & mate counts past depth 7 so I can confer?

[Ajedrecista]

Hello Tim:

Welcome to TalkChess! I can not confirm any value such as checkmates, discovered checks, etc.; only raw perft values (also divided perft) and unique positions, i.e. getting rid of transpositions. It looks like you have implemented en passant, which is usually subject of these discrepancies. I would take a look at castlings, too.

I am aware that the CPW has got an interesting external link, which is:

Statistics on chess games

I do not know if the CPW values were copied from that site, or verified by other sources.

FYI, similar threads were posted at TalkChess a few years ago:

Level 11 Perft statistics

Perft statistics - chessprogrammingwiki

Good luck with your research!

Regards from Spain.

Ajedrecista.

[tttony]

Wellcome!

First don't worry too much about that, just focus on getting a bug free move generator and compare the nodes result with perft and divide functions

I remember having this problem too, but not with your case, it was with the Kiwipete position in depth 5 I get for double checks 2645 but in the wiki says 2637, even in the wiki site notes about some programmers getting 2645

My question is, how we count when a move can give discovered check and double check at the same time?

I just created this position:

[d]2B3B1/3R1R2/Q1N1k1NR/3R1R2/2B1N1B1/8/4R3/4K3 w - - 0 1

Old method I counted like this:

Code: Select all

if (is_discovered_check) 
    discovered_check++
else if (is_double_check)
    double_check++

And I get this result for the testing position:

Code: Select all

 Depth          Nodes  ... Checks   Disc. Checks  Double Checks     Checkmates     Elapsed ms
     1             88  ...     50             36             14             50              0
     2              0  ...      0              0              0              0              0

The new method:

Code: Select all

if (is_discovered_check) 
    discovered_check++
if (is_double_check)
    double_check++

I get this result:

Code: Select all

 Depth          Nodes  ... Checks   Disc. Checks  Double Checks     Checkmates     Elapsed ms
     1             88  ...     50             50             14             50              0
     2              0  ...      0              0              0              0              0

And according to me, by checking several times counting manually I get that result too

But with this new method I get the nodes wrong in start position

[Sapling]

Thanks for the warm welcome

Ajedrecista those are really helpful resources, especially the wismuth link - where he gives a more formal definition of direct / discovered / double check + a break down table.

ttttony, I also got 2645 for the double checks for Kiwipete when incrementing single / double / discovered independently (your new method).

At depth 8 I got all the same results as the CPW but 847,043 discovered checks instead of the expected 847,039 on the cpw.
The fact that it was only off by 4 really through me, and that everything was matching exactly before depth 7.

Looking at the break down in Wismuth's table that is exactly how many single discovered mates exist at depth 8.

So I went through the table in the CPW to find how the numbers match Wismuth's table and essentially I've found that the CPW lists:

Checks as the sum of all checks of any kind (including single, double, discovered, mate)
Double Checks only count when it's NOT a mate
Discovery Checks only count when it's NOT a mate

After changing the logic in my generator to match that I'm getting the correct results that match the CPW including perft(5) for Kiwipete! Though the Wismuth's table is so useful I may actually change my output to match it.

Thanks for the help guys!

[sovaz1997]

I don't think it's necessary to know how many checkmate situations there are
I think the most important thing is the following: how much legal moves are available. 1) perft for depth = N and 2) all perfts for depth = N - 1 after all moves in target position.

What I have in my engine:

Code: Select all

b1a3: 4856835
b1c3: 5708064
g1f3: 5723523
g1h3: 4877234
a2a3: 4463267
a2a4: 5363555
b2b3: 5310358
b2b4: 5293555
c2c3: 5417640
c2c4: 5866666
d2d3: 8073082
d2d4: 8879566
e2e3: 9726018
e2e4: 9771632
f2f3: 4404141
f2f4: 4890429
g2g3: 5346260
g2g4: 5239875
h2h3: 4463070
h2h4: 5385554
Perft 6: 119060324; speed: 40Mnps; time: 2946.798s

Thus, if you have differences in the calculation of the total perft, then in some nodes there will also be a difference

And you just need to gradually go down recursively to the position in which there will be an incorrect perft with a depth=1 (i. e. count of possible moves from position) and find manually what kind of move it is

Yeah, for this you need to have other engine, to compare perft with it. I don't remember exactly, but Stockfish also did this output of perft as I remember.

[phhnguyen]

I am aware that the CPW has got an interesting external link, which is:

Statistics on chess games

I do not know if the CPW values were copied from that site, or verified by other sources.

Clearly, the information in CPW was not updated from that page. CPW has some extra information such as Captures, E.p., Castles, Promotions which are not on the page.

This information was updated by Alessandro Iavicoli without pointing to the source. Perhaps, it is from his own information. If you know the source, I will update CPW.

[towforce]

I just created this position:

[d]2B3B1/3R1R2/Q1N1k1NR/3R1R2/2B1N1B1/8/4R3/4K3 w - - 0 1

What was black's last move?

[Sapling]

Clearly, the information in CPW was not updated from that page. CPW has some extra information such as Captures, E.p., Castles, Promotions which are not on the page.

This information was updated by Alessandro Iavicoli without pointing to the source. Perhaps, it is from his own information. If you know the source, I will update CPW.

If you're interested in adding more information to the table I've filled out full statistics to startpos perft(11) and i'm currently 20% into perft(12).

This week I will also start filling out a similar table for Kiwipete.

Either way I do think there should be a note as to what the checks / double checks / discovery checks are actually counting in the CPW table as after perft(7) people get the wrong results because they are counting differently, even though their engine is generating moves correctly. And I think this defeats the purpose of perft.

(Check my reply on Jan 30th for the details)

[chrisw]

First time poster here, when I was building Sapling I really enjoyed coding & optimizing the move generator, ofc it's bearing on the strength of an engine is very minimal but, Afterall I was building the engine primarily for fun.

I've just started a new project called 'The Grand Chess Tree' - I'll create another post about it when it's in a more polished state, but suffice to say it's a specialized move generation engine.

So far I've traversed to perft(9), but I'm noticing from depth 7 the discovered checks and mates are not matching the results in the CPW for the initial position

depth(7) all match except mates:
mine vs cpw
mates: 435,765 vs 435,767
then for depth(8) the discovered start to diverge also:
discovered: 847,043 vs 847,039
mates: 9,851,894 vs 9,852,036

One thing to note is that the checks, discovered checks and mates are all calculated separately from the move generation, and all other counts seem to match which indicates a miss match in the classification of a leaf node.

I'm wondering if anyone else has ran a perft with discovered & mate counts past depth 7 so I can confer?

You can find a bunch of EPD suites here, specially designed for PERFT destruction testing ...

https://github.com/ChrisWhittington/Chess-EPDs

[Sapling]

You can find a bunch of EPD suites here, specially designed for PERFT destruction testing ...

https://github.com/ChrisWhittington/Chess-EPDs

Thanks Chris that's amazing! I'm currently working on the GPU implementation for the grand chess tree, so I'm sure those EPD suites will be super useful this week!