Removing bugs from bug free functions

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Cardoso
Posts: 314
Joined: Thu Mar 16, 2006 6:39 pm
Location: Portugal
Full name: Alvaro Cardoso
Contact:

Removing bugs from bug free functions

Post by Cardoso » Tue Jun 16, 2020 1:44 am

Over the years I've been experiencing some weird behavior from the MVS c++ compiler.
I have a bug free function the causes the engine to crash.
So what do I do? I discovered accidentally more than a decade ago that if I insert some dummy code that doesn't change the function results and that references a another function, the engine won't crash and works flawlessly! Just a day ago I had "deja vu" again. In the case below if I comment out the line:

Code: Select all

if (4 == 5) PrintGeneralText(" 4 == 5 ");
my error checking debug code detects incoherences in the board representation and the engine crashes later because the board data structure gets corrupted.

Have any of you had this experience?

Code: Select all

BB CanBeCapturedByEnemyMen(POS* pos, BB move, int color, int square) {
	const int opponent_color = Flip(color);
	const BB EnemyMen = Men(opponent_color);
	BB result = 0;
	
	if (4 == 5) PrintGeneralText(" 4 == 5 ");

		if ((MmtLeft[color][square] & EnemyMen) // there is an enemy man on our left
			&& (MmtLeft[opponent_color][square] & Free)  // the enemy can capture to his left
			) result = MmtLeft[color][square];

		if ((MmtRight[color][square] & EnemyMen) // there is an enemy man on our right
			&& (MmtRight[opponent_color][square] & Free)  // the enemy can capture to his right
			) result |= MmtRight[color][square];

	return result;
}
adsf

bob
Posts: 20916
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Removing bugs from bug free functions

Post by bob » Tue Jun 16, 2020 4:26 am

That isn't a bug-free function most likely. Typically when you can add code and hide the bug, look for uninitialized data that moves around on the stack depending on what extra code you insert...

In your case, function parameters can be wrong as well. IE you reference arrays using subscripts that might be invalid. I'd start by inserting code (or asserts) to verify that every argument is valid every time you call the thing.

RubiChess
Posts: 204
Joined: Fri Mar 30, 2018 5:20 am

Re: Removing bugs from bug free functions

Post by RubiChess » Tue Jun 16, 2020 4:54 am

Cardoso wrote:
Tue Jun 16, 2020 1:44 am
Over the years I've been experiencing some weird behavior from the MVS c++ compiler.
I have a bug free function the causes the engine to crash.
I'm using MSVC for many years now and never saw this.
If the engined crashed, there was a bug in the engine.
I suppose your function is not as bug free as you believe.

Regards.

Sven
Posts: 3877
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Re: Removing bugs from bug free functions

Post by Sven » Tue Jun 16, 2020 8:14 am

Cardoso wrote:
Tue Jun 16, 2020 1:44 am
Over the years I've been experiencing some weird behavior from the MVS c++ compiler.
I have a bug free function the causes the engine to crash.
So what do I do? I discovered accidentally more than a decade ago that if I insert some dummy code that doesn't change the function results and that references a another function, the engine won't crash and works flawlessly! Just a day ago I had "deja vu" again. In the case below if I comment out the line:

Code: Select all

if (4 == 5) PrintGeneralText(" 4 == 5 ");
my error checking debug code detects incoherences in the board representation and the engine crashes later because the board data structure gets corrupted.

Have any of you had this experience?

Code: Select all

BB CanBeCapturedByEnemyMen(POS* pos, BB move, int color, int square) {
	const int opponent_color = Flip(color);
	const BB EnemyMen = Men(opponent_color);
	BB result = 0;
	
	if (4 == 5) PrintGeneralText(" 4 == 5 ");

		if ((MmtLeft[color][square] & EnemyMen) // there is an enemy man on our left
			&& (MmtLeft[opponent_color][square] & Free)  // the enemy can capture to his left
			) result = MmtLeft[color][square];

		if ((MmtRight[color][square] & EnemyMen) // there is an enemy man on our right
			&& (MmtRight[opponent_color][square] & Free)  // the enemy can capture to his right
			) result |= MmtRight[color][square];

	return result;
}
Why "... & Free"? Why not "... & Men(color)"? The enemy usually can't capture on an empty square?

Furthermore, as a minor remark, the pos and move parameters are unused so you might want to check whether you can remove them.
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)

Dann Corbit
Posts: 11510
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Removing bugs from bug free functions

Post by Dann Corbit » Tue Jun 16, 2020 10:23 am

suggestion: turn your warnings up to the highest level possible

I guess you may see warnings about uninitialized things or undefined behavior.

Default levels for C and C++ compilers are not high enough to actually show you your mistakes except for the really obvious ones.

Some of the things that don't get warnings on default level compiles are the most outrageous sorts of things that should have screaming warnings in all caps with exclamation points.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Cardoso
Posts: 314
Joined: Thu Mar 16, 2006 6:39 pm
Location: Portugal
Full name: Alvaro Cardoso
Contact:

Re: Removing bugs from bug free functions

Post by Cardoso » Tue Jun 16, 2020 3:19 pm

Thank you for all the input.
Why "... & Free"? Why not "... & Men(color)"? The enemy usually can't capture on an empty square?
My engine is a spanish checkers engine.
Also Men() and Free are #defines to substitute bitboard variables in pos data structure, for easier code reading.
The only variable not needed is "BB move" but I keep it and added "int ply" for debug purposes.
Also I dumped
BB MmtLeft[2][32];
BB MmtRight[2][32];
into the log file and checked every bitboard one by one, every 128 of them visually.
Then I added the error checking code below, including CheckPositionCoherence() which is an all things checking function
that I only use at the root that checks all position bitboards against the pos->board[] array, if the move is valid, if the hashkey matches the position, material, empty and occupied bitboards, and other things. This is a good thing to have at the root to check everything after the search unwinds. But I added it in this function for the moment just to be sure.

Code: Select all

	if (color != 0 && color != 1) PrintGeneralText("CanBeCapturedByEnemyMen error: color=" + my_Str(color));
	if (square < 0  || square > 31) PrintGeneralText("CanBeCapturedByEnemyMen error: square out of bounds square" + my_Str(square) );
	if (opponent_color == color) PrintGeneralText("CanBeCapturedByEnemyMen error: opponent_color == color");
	UndoMove(pos, ply, move, color);
	   CheckPositionCoherence(pos, ply, color, move, 0, "CanBeCapturedByEnemyMen() 1");
        DoMove(pos, ply, move, color);
	if (BBSetMask(square) & ExtSquares) PrintGeneralText("CanBeCapturedByEnemyMen error: square is on the borders, square=" + my_Str(square));
I've found no errors whatsoever in this function.
After turning all warnings, and investigating a long list of them, I found no problems at all in this function.

Cardoso
Posts: 314
Joined: Thu Mar 16, 2006 6:39 pm
Location: Portugal
Full name: Alvaro Cardoso
Contact:

Re: Removing bugs from bug free functions

Post by Cardoso » Tue Jun 16, 2020 3:43 pm

Update on this, I removed all the error checking code and compiled the engine with all the optimizations disabled.
The engine ran perfectly without crashes, only slower.
So it looks to be a problem with the compiler optimizations, witch is what I've been seeing on all these years.
But as soon as I enable O1 optimizations the CheckPositionCoherence() starts to report errors.

Cardoso
Posts: 314
Joined: Thu Mar 16, 2006 6:39 pm
Location: Portugal
Full name: Alvaro Cardoso
Contact:

Re: Removing bugs from bug free functions

Post by Cardoso » Tue Jun 16, 2020 4:46 pm

Another update, after turning on maximum optimizations O2, and without error checking, CheckPositionCoherence() at the root detects errors and the engine crashes as expected, but after adding the code below in CanBeCapturedByEnemyMen() all problems are gone, except speed of course wich is lower since CheckPositionCoherence() is very slow.
So no more errors after referencing other functions before executing the main code of CanBeCapturedByEnemyMen().

Code: Select all

	UndoMove(pos, ply, move, color);
	CheckPositionCoherence(pos, ply, color, move, 0, "CanBeCapturedByEnemyMen() 1");
        DoMove(pos, ply, move, color);
So I guess I'm going back to the faster way of fixing this with the code:

Code: Select all

if (4 == 5) PrintGeneralText(" 4 == 5 ");

Ras
Posts: 1423
Joined: Tue Aug 30, 2016 6:19 pm
Full name: Rasmus Althoff
Contact:

Re: Removing bugs from bug free functions

Post by Ras » Tue Jun 16, 2020 6:51 pm

Cardoso wrote:
Tue Jun 16, 2020 3:43 pm
So it looks to be a problem with the compiler optimizations, witch is what I've been seeing on all these years.
It's a problem with your code that has undefined behaviour somewhere, and not necessarily in the function where things finally crash. Introducing nonsense code does not fix anything, it's just papering over the actual bugs.
Rasmus Althoff
https://www.ct800.net

Cardoso
Posts: 314
Joined: Thu Mar 16, 2006 6:39 pm
Location: Portugal
Full name: Alvaro Cardoso
Contact:

Re: Removing bugs from bug free functions

Post by Cardoso » Tue Jun 16, 2020 8:09 pm

Thanks Rasmus,
I accept that!
But with optimizations disabled and the nonsense code disabled the engine doesn't crash and no errors are reported.
Now I turned full optimization again and activated the nonsense code and made a test using CheckPositionCoherence() at every ply in the search and eval, and it didn't report any errors.
I'll give up for today, I'll try to find the problem another day.

Post Reply