Mailboxes to bitboards transition

[emadsen]

Got my bit boards routine working and surprised by the results compared to my mailboxes. I was expecting very little reduction in speed, thinking my mailbox engine was pretty efficient. Results as follows:

Average over 10,000,000 runs.
Mailboxes = 2,272,507 nodes per second
bitboards = 13,843,647 nodes per second

I don't think I have a standard bitboard since I only use bit operations on the sliding pieces.

What bitboard technique are you using for sliders? Have you implemented magic bitboards? If so, 13M NPS is a bit slow. I can't remember exactly, but when I first got move generation functioning, prior to implementing all the bookkeeping necessary to sort moves, prior to staged move generation, etc... I think I got 50M NPS in C# using magic bitboards for sliders. On an AMD Ryzen 1950X.

Most of the time is spent on search and evaluation. Better search and/or eval, less cpu time will be spent on movegen. Movegen is minor piece on any decent engine. More so with staged movegens, where most moves aren't even generated, or engines that simply try the move from TT without generating any moves at all.

This is not a novelty. Bob Hyatt said it long ago:

"Speed here is not so important. I doubt anyone's move generator takes more than 10% of total search time, which means a 20% improvement in perft numbers is only a 2% overall speed gain. I would not worry about anything but matching the node counts exactly..."
https://www.chessprogramming.org/Perft#Quotes

That's correct. Perft is a regression test.

[lithander]

I used 10,000,000 runs on 300 puzzle mates from 4 - 18 ply to insure accuracy of the engine, averaging about 59 pseudo moves per game. My 5 year old i7 has 1 core running at 2.59 GHz. The 10,000,000 runs were needed to eliminate noise from the OS.

The engine is finding the best move for positions at 13M Nps? I think in that case it is pretty fast. 13M Nps for perft would be rather slow but if it's solving mates the cost for search and evaluation is already included.

If everyone could agree on the same set of 300 puzzles one could compare the speed of engines that way. But it would be only the best-mate-finding speed so again nothing that really predicts the performance of the engine in actual play.

[spirch]

going to point out that he is using a 2015 MOBILE intel cpu

Chessnut1071, if you can find a more recent computer, i would try it there too that could remove some bad feedback

in any case, good work on improving / learning new thing with your code, keep doing it!

[dangi12012]

I used 10,000,000 runs on 300 puzzle mates from 4 - 18 ply to insure accuracy of the engine, averaging about 59 pseudo moves per game. My 5 year old i7 has 1 core running at 2.59 GHz. The 10,000,000 runs were needed to eliminate noise from the OS.

The engine is finding the best move for positions at 13M Nps? I think in that case it is pretty fast. 13M Nps for perft would be rather slow but if it's solving mates the cost for search and evaluation is already included.

If everyone could agree on the same set of 300 puzzles one could compare the speed of engines that way. But it would be only the best-mate-finding speed so again nothing that really predicts the performance of the engine in actual play.

Well there would be a way. If someone would take a good sample of 1000 positions and 1000 positions with poison and generate the evaluation by spending a trillion with SF on them.
poison is a position where the evaluation drastically changes after depth 20.

Then you can get the standard error for your own evaluation with 1 Billion nodes against the "known" reference 1 Trillion evals.
That would be your score.
You can also see how "node" is not well defined when switching engines. And time is not a factor here at all but thats important since nps are the actual engine speed.

You see there are many variables that would need discussion but that would be a way to approximate "strength" without self play. If your own engine evalues correctly but only uses 10x less nodes that would correspond to +200 elo.

[dangi12012]

Oh sorry there was a mistake in my previous post.
A 10x increase in node count is ~200 elo.
A 1000x increase would be 600 elo.
I dont know if this is true for any increase in elo (probably not)
and the more I think of it - there more i see flaws.

So really the only good way to get good elo numbers is to play games in a pool of well known engines with a known elo score.

[Chessnut1071]

Got my bit boards routine working and surprised by the results compared to my mailboxes. I was expecting very little reduction in speed, thinking my mailbox engine was pretty efficient. Results as follows:

Average over 10,000,000 runs.
Mailboxes = 2,272,507 nodes per second
bitboards = 13,843,647 nodes per second

I don't think I have a standard bitboard since I only use bit operations on the sliding pieces.

What bitboard technique are you using for sliders? Have you implemented magic bitboards? If so, 13M NPS is a bit slow. I can't remember exactly, but when I first got move generation functioning, prior to implementing all the bookkeeping necessary to sort moves, prior to staged move generation, etc... I think I got 50M NPS in C# using magic bitboards for sliders. On an AMD Ryzen 1950X.

Most of the time is spent on search and evaluation. Better search and/or eval, less cpu time will be spent on movegen. Movegen is minor piece on any decent engine. More so with staged movegens, where most moves aren't even generated, or engines that simply try the move from TT without generating any moves at all.

This is not a novelty. Bob Hyatt said it long ago:

"Speed here is not so important. I doubt anyone's move generator takes more than 10% of total search time, which means a 20% improvement in perft numbers is only a 2% overall speed gain. I would not worry about anything but matching the node counts exactly..."
https://www.chessprogramming.org/Perft#Quotes

That's correct. Perft is a regression test.

I have a very simple bitboard, a precomputed move vector (8) for each of the 64 squares. I just use the intrinsics TrailingZeros and LeadigZeros after I XOR out the last move to find the next move. I stop at a blocker.

btw, that 13.8 million was in debug mode. Also, I capture the following data for each move:
1 check 2 double check 3 discovered check 4 pin 5 ent passant 6 pawn promotion 7 move direction 8 capture 9 defender.
My speed should about 2x in release mode, however, I have to copy all my precomputed tables into the release directory before I can test there.

I haven't tried magic bitboards because I don't know enough about them to program that yet. Once I have the bitboards fully implemented without error, I'll try that next.

We're planning on using a chess engine as one of the algorithms to test on a quantum computer. Translating these C# instruction to machine code is well above my pay grade. Are any of you developers doing an engine at the assembler level?

[hgm]

Your numbers are suspect. Normally engines do not much better than 1-2 Mnps during search, whether mailbox or bitboard.

[Chessnut1071]

Your numbers are suspect. Normally engines do not much better than 1-2 Mnps during search, whether mailbox or bitboard.

??? supect ???

10,000,000 engine call x AVG 59 pseudo moves = 590,000,000 (nodes) / 21.57 seconds ( release mode ) = 27.352,804 nodes per second

Actually, from what I'm hearing, my engine is super SLOW! Are you judging from legal moves or pseudo moves?

[hgm]

That is moves per second, not nodes per second. So you were indeed exaggerating your speed by a factor 59.

It also doesn't seem to be the speed of an engine, but of perft.

[emadsen]

I have a very simple bitboard, a precomputed move vector (8) for each of the 64 squares. I just use the intrinsics TrailingZeros and LeadigZeros after I XOR out the last move to find the next move. I stop at a blocker.

btw, that 13.8 million was in debug mode. Also, I capture the following data for each move:
1 check 2 double check 3 discovered check 4 pin 5 ent passant 6 pawn promotion 7 move direction 8 capture 9 defender.

It's difficult for me to understand your move generation logic without seeing your code. In previous discussions- if I recall correctly- I and other developers have asked how your code accounts for every permutation of occupancy along a move direction ray. I don't recall seeing that code. Though "I stop at a blocker" suggests your move generator depends on a loop or does some bookkeeping to track blockers in each direction. Sharing your code would clarify this.

??? supect ???

10,000,000 engine call x AVG 59 pseudo moves = 590,000,000 (nodes) / 21.57 seconds ( release mode ) = 27.352,804 nodes per second

Actually, from what I'm hearing, my engine is super SLOW! Are you judging from legal moves or pseudo moves?

"Engine calls" is not a phrase anyone on this forum is familiar with. It does not have a clear, unambiguous definition chess engine programmers agree upon. Also unclear is the significance of 59 average pseudo-legal moves. How did you measure that number? Just because the root position has 59 pseudo-legal moves does not mean, on average, every position in the search tree has 59 pseudo-legal moves. Likely less.

The standard metric used by TalkChess forum members is Nodes Per Second (NPS), measured by incrementing the node count whenever Board.PlayMove(move) is called. Correct me if I'm wrong (forum members), but I believe that's the metric most engine developers use. It measures the rate at which an engine examines legally reachable (well, other than via null move) positions.

Also, it's helpful to state whether you've measured NPS during perft (pure move generation) or during search (usually alpha / beta minimax + depth reductions + pruning).