CCT10 Crafty release postponed?

[bob]

Hi Bob,
I was always thinking that the 64-bit version of crafty should be stronger
than the 32-bit version. From results of some testers I am not shure now.
Did those testers something wrong?
kind regards
Bernhard

I have not followed those discussions so I don't know what they are doing. Clearly the 64 bit version is significantly faster than the 32 bit version, on the same hardware. In fact, the executable itself is about 1/2 the size because of all the double instructions needed to do 64 bit operations. The executable on my 32 bit xeon is 1.1 megs or so, on my 64 bit boxes it is about .5 megs. So the savings from doing two 32 bit operations vs one 64 bit operation is significant. Which means that it should certainly be stronger on a 64 bit platform. How much faster it is, I really don't know at present. I have not tried 32 bit vs 64 bit on the same processor since this discussion came up a few years ago. I'd guess 35% faster is pretty close, and that makes a measurable improvement in playing level...

[Uri Blass]

Bob, I don't see the recent CCT10 Crafty in your FTP site there yet. You said that it would be released right after the tournament.

Just anxious

Jarkko Pesonen

Here is where we are. I have almost finished 22.0. This will have a couple of eval tweaks, but the main thing is it is significantly smaller. The white/black duplication in evaluate.c, movgen.c, make.c, etc no longer exists. I lack one module, EvaluatePassedPawnRaces() which will be done within a day or two. We have been discussing this and we plan on releasing version 22.0 as soon as that is done. We have not been doing many changes because this has shredded a lot of code and I have been carefully testing because these changes are very significant. I am running about 1,200 positions searched to fixed depth, and require that _every_ position match the pre-22.0 version node counts exactly.

I will post a note here this week when it is ready, as we want to get this out so we can resume making changes that will affect its playing strength.

I will note that this version is a bit faster, probably due to reduced cache footprint, and a little code was cleaned up here and there. I plan on continuing the clean-up since it is now about 1/2 the work, not having to change both the black and then the white code. So the first few 22.0 versions will each be cleaner than the last, in addition to having new ideas included. I am really working toward going thru every "kernel" module (the stuff used within the search) with a fine-toothed comb to look for bugs, or improvements.

It will definitely be out this week, once I get this last module done. To date, we are lighter by about 3,500 lines of code, with another 200+ getting removed from this last module...

Note that strelka have seperate code for white and black and I wonder if changing it can make it faster.

How much speed improvement do you get from seperate code for white and black?

Is it a good idea in terms of speed for all things or maybe at least for pawns it is better to have seperate code for white and black?

Is it depedent on the size of the program when basically it is better to have seperate code for white and black only if the program is relatively small?

How do you work about changing the code?
I think that a possible way may be by the following steps:

1)Add data structures that you may need instead of the existing data structure but do not delete the old data structure.
Example:If you have a special bitboard for white pieces that you call
whiteBB and special bitboard for black pieces that you call blackBB then you can add special bitboard array colorBB[2];

2)Add functions that do not include white/black duplication
do not delete old functions.

3)Add code to check if you really duplicate everything correctly(information should be the same for example if you have whiteBB bitboard and colorBB[WHITE] bitboard after step 1 then you can check that they are always the same).
steps 1-3 should be done first to parts that do not include the evaluation and later you can also add evaluation code that is based on the arrays that you add and you may have evaluate based on the old data and evaluate1 based on the new data when you check that they return the same result.
4)Check that you get the same result in many positions.
5)Delete irrelevant code because there are functions and arrays that you simply do not need because the changes that you did caused you to do many things twice.

Uri

[bob]

Bob, I don't see the recent CCT10 Crafty in your FTP site there yet. You said that it would be released right after the tournament.

Just anxious

Jarkko Pesonen

Here is where we are. I have almost finished 22.0. This will have a couple of eval tweaks, but the main thing is it is significantly smaller. The white/black duplication in evaluate.c, movgen.c, make.c, etc no longer exists. I lack one module, EvaluatePassedPawnRaces() which will be done within a day or two. We have been discussing this and we plan on releasing version 22.0 as soon as that is done. We have not been doing many changes because this has shredded a lot of code and I have been carefully testing because these changes are very significant. I am running about 1,200 positions searched to fixed depth, and require that _every_ position match the pre-22.0 version node counts exactly.

I will post a note here this week when it is ready, as we want to get this out so we can resume making changes that will affect its playing strength.

I will note that this version is a bit faster, probably due to reduced cache footprint, and a little code was cleaned up here and there. I plan on continuing the clean-up since it is now about 1/2 the work, not having to change both the black and then the white code. So the first few 22.0 versions will each be cleaner than the last, in addition to having new ideas included. I am really working toward going thru every "kernel" module (the stuff used within the search) with a fine-toothed comb to look for bugs, or improvements.

It will definitely be out this week, once I get this last module done. To date, we are lighter by about 3,500 lines of code, with another 200+ getting removed from this last module...

Note that strelka have seperate code for white and black and I wonder if changing it can make it faster.

How much speed improvement do you get from seperate code for white and quote]


Nothing significant. 2-3% at best. My main concern was that it might slow down a bit since I had to add some array references I had thought it might be a percent or two slower, but was surprised to find it became faster.

Is it a good idea in terms of speed for all things or maybe at least for pawns it is better to have seperate code for white and black?

I have successfully done it for everything. There is one move generator that works for either side, one Make/Unmake that works for either side, and one evaluation for each type of piece that works for both sides. There are still a few things that have to include both sides, such as when determining if someone has a pawn that can run, because both sides can do that and the analysis has to consider both at the same time. But for almost everything, the redundancy has been removed.

In GenerateMoves() there are just 3-4 lines that are duplicated to move the pawns, since they require shifts in two different directions and I have not tried to get rid of that yet... although I will as the cleanup continues. But first I just want to get a pretty clean 22.0 onto my ftp box, then 22.1 will be a cleaner version of that...

Is it depedent on the size of the program when basically it is better to have seperate code for white and black only if the program is relatively small?

I suspect the principal issue is cache footprint. And while the correlated branch prediction hardware can do pretty well on the if (wtm) and if (!wtm) type tests, they are still branches that have to be predicted and this new version gets rid of that...

How do you work about changing the code?
I think that a possible way may be by the following steps:

1)Add data structures that you may need instead of the existing data structure but do not delete the old data structure.
Example:If you have a special bitboard for white pieces that you call
whiteBB and special bitboard for black pieces that you call blackBB then you can add special bitboard array colorBB[2];

2)Add functions that do not include white/black duplication
do not delete old functions.

3)Add code to check if you really duplicate everything correctly(information should be the same for example if you have whiteBB bitboard and colorBB[WHITE] bitboard after step 1 then you can check that they are always the same).
steps 1-3 should be done first to parts that do not include the evaluation and later you can also add evaluation code that is based on the arrays that you add and you may have evaluate based on the old data and evaluate1 based on the new data when you check that they return the same result.
4)Check that you get the same result in many positions.
5)Delete irrelevant code because there are functions and arrays that you simply do not need because the changes that you did caused you to do many things twice.

Uri

I just keep the old version in one directory, and copy everything to a new directory. I then make one set of changes, say rewriting GenerateCaptures() using the new approach I then run a simple shell script that searches about 1200 positions to fixed depth and compare node counts. If they don't match exactly, I find out why. I eventually search those 1200 positions to depth=12, which takes an hour or so. A perfect match and the new version becomes the "base" version and I start again on the next module rewrite. The evaluation stuff has been a bit slower, as some of it is getting rewritten as I go because some was a bit on the kludgey side. My intent is for 22.x to eventually be as clean as I can make it...

That makes it easier for us to make changes since we don't have to do the changes twice and then deal with the asymmetric bugs that introduces.