Debugging trick

[michiguel]

I compiled my engine with different compilers (linux intel and gcc) and I observed that the 64 bit versions (gcc vs intel) had very slightly different node counts when searching the starting position. The 32-bit versions were the same, and equal to the gcc 64 bit. The oddball was Intel64. The difference showed up after ~15 s of search, which meant that I had to dump a huge tree to find out the differences. But, I did something else. I ran an epdtest of all the STS positions (600) at depth 3. Then, I picked a position that gave me different node counts. In this case, it was very easy to find the difference after dumping a very small tree. Then, I fixed a stupid bug that was in the evaluation for a long time, undetected.

I think it will be a good routine test to use different compilers and run large number of positions at very shallow depths to look for differences.

Miguel

[stegemma]

...
I think it will be a good routine test to use different compilers and run large number of positions at very shallow depths to look for differences.

Miguel

Maybe the problem occurs depending on different random numbers generators? If so, could be an advice to use different random seed at any test, and a set of predefined ones too.

[rvida]

...
I think it will be a good routine test to use different compilers and run large number of positions at very shallow depths to look for differences.

Miguel

Maybe the problem occurs depending on different random numbers generators? If so, could be an advice to use different random seed at any test, and a set of predefined ones too.

No, I think this is due to different stack/data layout. An uninitialized variable can have different value with different compiler.

[michiguel]

...
I think it will be a good routine test to use different compilers and run large number of positions at very shallow depths to look for differences.

Miguel

Maybe the problem occurs depending on different random numbers generators? If so, could be an advice to use different random seed at any test, and a set of predefined ones too.

No, I think this is due to different stack/data layout. An uninitialized variable can have different value with different compiler.

That's what it was. I was accessing an element of an array that was just outside.

Miguel

[Onno Garms]

There are several tools that spot uninitialized memory reads, e.g. Valgrind, BoundChecker, Purify. They will report an error even if the value of the unitialized variable does not cause problems.

Having said so, comparing node counts on a large number of test positions is very useful for differential testing. I.e., if you made some changes that are supposed not to change the search tree, verify that by comparing node counts.

[michiguel]

There are several tools that spot uninitialized memory reads, e.g. Valgrind, BoundChecker, Purify. They will report an error even if the value of the unitialized variable does not cause problems.

I used valgrind before, and apparently did not catch this. I used an index that was the product of a calculation and the return of a function. I should have used an assert(), but I did not.

Miguel

Having said so, comparing node counts on a large number of test positions is very useful for differential testing. I.e., if you made some changes that are supposed not to change the search tree, verify that by comparing node counts.

[Gian-Carlo Pascutto]

I used valgrind before, and apparently did not catch this. I used an index that was the product of a calculation and the return of a function. I should have used an assert(), but I did not.

There are some limitations, IIRC with stack variables and alignment.

std::tr1::array also catches those.