I have found a "good bug" in my modified version of Sunfish.
When the move is a TT move, I give it a bonus of 10 pawns to ensure it goes to the front of the move list. However, when doing this, I forgot that the function that gives move scores is also the eval.
When I fixed the bug, the engine ends up being 2 ply slower, and chooses a different move as a result, as well as searching almost 3 times more nodes (but does give a sane-looking evaluation score).
Do I fix the bug and take the 2 ply hit, or keep the bug and be forced to ignore the search score?
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 1059 0 25 - 4 0
2 642 9 154 - 4 0
3 300 17 336 - 4 0
4 601 64 2311 - 4 0
5 1059 133 7890 - 4 0
6 940 200 19122 - 4 0
7 0 322 56014 - 4 0
My move: e7e5
Score: 1000
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 59 0 23 - 4 0
2 -40 10 134 - 4 0
3 59 21 774 - 4 0
4 -38 73 3637 - 4 0
5 39 196 22082 - 4 0
My move: g8f6
Score: 59
elcabesa wrote:I think you should try letting the 2 engine play against each other some game and see the result.
some game= from 50 to 60000
I think the corrected engine will be stronger abd you'll see benefits with few hundreds of games
Xann wrote:
On 26 Apr 2014, at 16:01, Fabien Letouzey wrote:
> Leave it until you find the real bug.
Also, check first that you lose loads of Elo points. The two plies might be an illusion.
Testing is going to be a pain in the behind. (The engine has no concept of time management)
Oh well.
Matthew:out
As I suspected, no matter how hard I try, my engines lose on time, or have a connection stall.
Clearly, conventional testing is never going to work in this case.
I have a backup plan, though. The engine does have the capability of reading from EPDs and test positions, so I'm going to run both engines through test positions and see which is better.
ZirconiumX wrote:I have found a "good bug" in my modified version of Sunfish.
When the move is a TT move, I give it a bonus of 10 pawns to ensure it goes to the front of the move list. However, when doing this, I forgot that the function that gives move scores is also the eval.
When I fixed the bug, the engine ends up being 2 ply slower, and chooses a different move as a result, as well as searching almost 3 times more nodes (but does give a sane-looking evaluation score).
Do I fix the bug and take the 2 ply hit, or keep the bug and be forced to ignore the search score?
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 1059 0 25 - 4 0
2 642 9 154 - 4 0
3 300 17 336 - 4 0
4 601 64 2311 - 4 0
5 1059 133 7890 - 4 0
6 940 200 19122 - 4 0
7 0 322 56014 - 4 0
My move: e7e5
Score: 1000
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 59 0 23 - 4 0
2 -40 10 134 - 4 0
3 59 21 774 - 4 0
4 -38 73 3637 - 4 0
5 39 196 22082 - 4 0
My move: g8f6
Score: 59
Matthew:out
Better to use the fixed one. It is normal to see a somewhat weird engine behaviour after fixing some bugs. It could be that the engine had adapted to the bug. After the fix you have to re-check all features of the engine, especially those that relates to the bug.
ZirconiumX wrote:I have found a "good bug" in my modified version of Sunfish.
When the move is a TT move, I give it a bonus of 10 pawns to ensure it goes to the front of the move list. However, when doing this, I forgot that the function that gives move scores is also the eval.
When I fixed the bug, the engine ends up being 2 ply slower, and chooses a different move as a result, as well as searching almost 3 times more nodes (but does give a sane-looking evaluation score).
Do I fix the bug and take the 2 ply hit, or keep the bug and be forced to ignore the search score?
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 1059 0 25 - 4 0
2 642 9 154 - 4 0
3 300 17 336 - 4 0
4 601 64 2311 - 4 0
5 1059 133 7890 - 4 0
6 940 200 19122 - 4 0
7 0 322 56014 - 4 0
My move: e7e5
Score: 1000
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 59 0 23 - 4 0
2 -40 10 134 - 4 0
3 59 21 774 - 4 0
4 -38 73 3637 - 4 0
5 39 196 22082 - 4 0
My move: g8f6
Score: 59
Matthew:out
My suggestion is a bit more direct. I would not keep EITHER version until I understood exactly why the corrected version is 2 ply slower than the version with the bug. This is only affecting move ordering as I understand what you wrote? Or is this actually affecting the evaluation, which I don't quite follow if so.
Setting your eval to always return 0 will also gain a couple of ply. In general an eval that switches favorites less often will search deeper, but that doesn't mean that it searches better. I'd fix the bug.
ZirconiumX wrote:I have found a "good bug" in my modified version of Sunfish.
When the move is a TT move, I give it a bonus of 10 pawns to ensure it goes to the front of the move list. However, when doing this, I forgot that the function that gives move scores is also the eval.
When I fixed the bug, the engine ends up being 2 ply slower, and chooses a different move as a result, as well as searching almost 3 times more nodes (but does give a sane-looking evaluation score).
Do I fix the bug and take the 2 ply hit, or keep the bug and be forced to ignore the search score?
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 1059 0 25 - 4 0
2 642 9 154 - 4 0
3 300 17 336 - 4 0
4 601 64 2311 - 4 0
5 1059 133 7890 - 4 0
6 940 200 19122 - 4 0
7 0 322 56014 - 4 0
My move: e7e5
Score: 1000
r n b q k b n r
p p p p p p p p
. . . . . . . .
. . . . . . . .
. . . . P . . .
. . . . . . . .
P P P P . P P P
R N B Q K B N R
Depth Score Time Nodes PV
1 59 0 23 - 4 0
2 -40 10 134 - 4 0
3 59 21 774 - 4 0
4 -38 73 3637 - 4 0
5 39 196 22082 - 4 0
My move: g8f6
Score: 59
Matthew:out
My suggestion is a bit more direct. I would not keep EITHER version until I understood exactly why the corrected version is 2 ply slower than the version with the bug. This is only affecting move ordering as I understand what you wrote? Or is this actually affecting the evaluation, which I don't quite follow if so.
In Sunfish, which was originally a didactic program with small code, the function that updates the evaluation is also the function that does move valuing (it's only material + PST, it isn't expensive).
When I changed the move ordering to put the transposition table at the front, I had to make an ugly hack that involved checking for the presence of a TT move, and giving the move a ten pawn bonus if it was. Unfortunately, I forgot the above when doing this.
When I realised my mistake, I fixed the bug...and got a 2 ply loss.
I think maybe the high score from the moves makes the QS stand pat almost immediately, causing a reduction in nodes, and thus a speedup.
