Some xboard issues

[hgm]

OK, next round:

It seems that the change in blackTimeRemaining occurs during the handling of the move itself. So I plastered HandleMachineMove and some routines it calls with printfs in the next patch, so we can do bi-section. Please run that one.

[Evert]

Code: Select all

30178 >first : (10000) time 1000
30178 >first : (10000) otim 1000
book hit = (NULL)
30178 >first : (10000) go
nps: w=-1, b=-1
30183 <first &#58; &#40;10000&#41;   2    5     0        46 Ng1-f3 
30196 <first &#58; &#40;10000&#41;   3    9     1       501  e2-e4 Ng8-h6 
30202 <first &#58; &#40;10000&#41;   4    6     2       849  e2-e4 Ng8-f6 Qd1-f3 
30229 <first &#58; &#40;10000&#41;   5   11     5      3467  e2-e3 Ng8-h6 Nb1-c3 Nb8-c6 
30283 <first &#58; &#40;10000&#41;   6    4    10      9700 Nb1-c3 Ng8-f6  e2-e4 Nb8-c6 Bf1-e2 
30350 <first &#58; &#40;10000&#41;   7    3    17     19316 Ng1-f3 Nb8-c6  e2-e4  e7-e6 Bf1-e2 Bf8-d6 
30351 <first &#58; &#40;10000&#41; move g1f3 
machine move 0, castling = 7 0 4 7 0 4 &#40;10000&#41;
trial 6,1,5,3  type 2121
7 0 4 7 0 4 Legality test? g1f3
7 0 4 7 0 4 Legality test? g1f3
&#40;7,0&#41; &#40;0,0&#41; &#40;4,0&#41; &#40;7,7&#41; &#40;0,7&#41; &#40;4,7&#41; castling rights
H1 btr=188978571024
M1 btr=188978571024
M2 btr=188978571024
M3 btr=188978571024
CoordsToAlgebraic, piece=1 &#40;6,0&#41;-&#40;5,2&#41; - PARTIAL-1=ffffffff
M4 btr=188978571024
M5 btr=188978571024
M6 btr=188978571024
M7 btr=188978571024
S1 btr=188978571024
S3 btr=188978571024
S4 btr=188978571024
check B in&#58; last=&#40;0,0&#41; rem=&#40;9827,188978571024&#41;
TC string = '&#58;40/10'
mps=40 tc=10000 inc=0
check B out&#58; last=(-9827,188978571024&#41; rem=&#40;9827,188978571024&#41;
S5 btr=188978571024
M8 btr=188978571024
MateTest&#58; K=1, my=16, his=16
move&#58; g1f3
, parse&#58; Nf3 (
)
H2 btr=188978571024
MateTest&#58; K=1, my=16, his=16
repeat test fmm=1 bmm=0 ep=-4, reps=6
1 ep=-4
0 ep=-4
time odds&#58; 1.000000 1.000000 
30352 >second&#58; &#40;188978571024&#41; time 18897857102
30352 >second&#58; &#40;188978571024&#41; otim 982

I could try running through valgrind or gdb if you think that's helpful. It'll be slow though (valgrind, I mean). It may also produce loads of warnings that have nothing to do with the problem at hand.

[hgm]

Well, I have no experience with valgrind. As this is so nicely reproducible, I am sure the printfs are going to get us there, sooner or later. The damage seems to happen in the move parser. The next path is already waiting for you to test it.

[Evert]

Well, I have no experience with valgrind. As this is so nicely reproducible, I am sure the printfs are going to get us there, sooner or later. The damage seems to happen in the move parser. The next path is already waiting for you to test it.

We're probably getting closer:

Code: Select all

8307 >first : (10000) time 1000
8307 >first : (10000) otim 1000
book hit = (NULL)
8307 >first : (10000) go
nps: w=-1, b=-1
8309 <first &#58; &#40;10000&#41;   2   11     0         7 Nb1-c3 
8315 <first &#58; &#40;10000&#41;   3   20     0       135 Ng1-f3  f7-f5 
8316 <first &#58; &#40;10000&#41;   4   20     0       230 Ng1-f3  e7-e6 Nb1-c3 
8324 <first &#58; &#40;10000&#41;   5   15     1       963 Ng1-f3  f7-f6  e2-e4 Ng8-h6 
8398 <first &#58; &#40;10000&#41;   6    7     9      8581  e2-e4 Ng8-f6 Nb1-c3  e7-e5 Bf1-c4 
8423 <first &#58; &#40;10000&#41;   7   11    11     12040  e2-e4  e7-e5 Bf1-c4 Qd8-f6 Qd1-f3 Bf8-e7 
8700 <first &#58; &#40;10000&#41;   8    2    39     54492  d2-d4  e7-e6 Ng1-f3 Bf8-e7 Nb1-c3 Ng8-f6 
8701 <first &#58; &#40;10000&#41; move d2d4 
machine move 0, castling = 7 0 4 7 0 4 &#40;10000&#41;
P1 btr=10000
trial 3,2,3,4  type 2121
N1 btr=10000
N2 btr=10000
N3 btr=10000
7 0 4 7 0 4 Legality test? d2d4 &#40;10000&#41;
L1 btr=10000
L2 btr=188978571024

It's messed up from that point onward.

[hgm]

Hmm, it happens in GenLegal called for testing move legality. That is quite unexpected...

I pushed a new patch, which reports from the callbacks for each move. I hope this will be the last trial we need. (It could still be one level deeper, in CheckTest...)

[Evert]

Hmm, it happens in GenLegal called for testing move legality. That is quite unexpected...

I pushed a new patch, which reports from the callbacks for each move. I hope this will be the last trial we need. (It could still be one level deeper, in CheckTest...)

Code: Select all

7 0 4 7 0 4 Legality test? g1f3 (10000)
L1 btr=10000
G1 btr=10000
C1 btr=10000 0 6 2 5
C2 btr=10000 0 6 2 5
T1 btr=10000 0 6 2 5
G2 btr=10000
G3 btr=188978571024
G4 btr=188978571024
L2 btr=188978571024

By the way, there's an #include <malloc.h> in xhistory.c that is obsolete (for instance, http://hintsforums.macworld.com/showthread.php?t=2023). It exists on Linux but not on OS X, so I keep deleting that line because the compiler chokes on it.
A conservative fix would be to #ifdef it out on a Mac, but personally I'd just remove that line...

[hgm]

OK, it does apparently happen in CheckTest, so we need another round. (Sorry, I forgot to remove the malloc.h.)

[Evert]

Ok, so it is doing an out-of-memory write on the board array (indexing element [-1][-1]). I could patch that myself to work around the problem, but I don't know what the correct behaviour is supposed to be and I don't fully understand the code, so I might break something else.

Code: Select all

7 0 4 7 0 4 Legality test? d2d4 (10000)
L1 btr=10000
G1 btr=10000
C1 btr=10000 1 3 3 3
E1 btr=10000 1 3 3 3
E2 btr=10000
E3 btr=10000 0 0
E3 btr=10000 1 0
E3 btr=10000 2 0
E3 btr=10000 3 0
E3 btr=10000 4 0
E3 btr=10000 5 0
E3 btr=10000 6 0
E3 btr=10000 7 0
E3 btr=10000 0 1
E3 btr=10000 1 1
E3 btr=10000 2 1
E3 btr=10000 3 1
E3 btr=10000 4 1
E3 btr=10000 5 1
E3 btr=10000 6 1
E3 btr=10000 7 1
E3 btr=10000 0 2
E3 btr=10000 1 2
E3 btr=10000 2 2
E3 btr=10000 3 2
E3 btr=10000 4 2
E3 btr=10000 5 2
E3 btr=10000 6 2
E3 btr=10000 7 2
E3 btr=10000 0 3
E3 btr=10000 1 3
E3 btr=10000 2 3
E3 btr=10000 3 3
E3 btr=10000 4 3
E3 btr=10000 5 3
E3 btr=10000 6 3
E3 btr=10000 7 3
E3 btr=10000 0 4
E4 btr=10000
K1 btr=10000 6 0 5 0
K1 btr=10000 6 0 4 0
K1 btr=10000 6 1 5 1
K1 btr=10000 6 1 4 1
K1 btr=10000 6 2 5 2
K1 btr=10000 6 2 4 2
K1 btr=10000 6 3 5 3
K1 btr=10000 6 3 4 3
K1 btr=10000 6 4 5 4
K1 btr=10000 6 4 4 4
K1 btr=10000 6 5 5 5
K1 btr=10000 6 5 4 5
K1 btr=10000 6 6 5 6
K1 btr=10000 6 6 4 6
K1 btr=10000 6 7 5 7
K1 btr=10000 6 7 4 7
K1 btr=10000 7 1 5 0
K1 btr=10000 7 1 5 2
K1 btr=10000 7 6 5 5
K1 btr=10000 7 6 5 7
E5 btr=10000
E6 btr=10000
C2 btr=10000 1 3 3 3
T1 btr=10000 1 3 3 3
G2 btr=10000
E1 btr=10000 -1 -1 -1 -1
E2 btr=-4294957296
E3 btr=-4294957296 0 0
E3 btr=-4294957296 1 0
E3 btr=-4294957296 2 0
E3 btr=-4294957296 3 0
E3 btr=-4294957296 4 0
E3 btr=-4294957296 5 0
E3 btr=-4294957296 6 0
E3 btr=-4294957296 7 0
E3 btr=-4294957296 0 1
E3 btr=-4294957296 1 1
E3 btr=-4294957296 2 1
E3 btr=-4294957296 3 1
E3 btr=-4294957296 4 1
E3 btr=-4294957296 5 1
E3 btr=-4294957296 6 1
E3 btr=-4294957296 7 1
E3 btr=-4294957296 0 2
E3 btr=-4294957296 1 2
E3 btr=-4294957296 2 2
E3 btr=-4294957296 3 2
E3 btr=-4294957296 4 2
E3 btr=-4294957296 5 2
E3 btr=-4294957296 6 2
E3 btr=-4294957296 7 2
E3 btr=-4294957296 0 3
E3 btr=-4294957296 1 3
E3 btr=-4294957296 2 3
E3 btr=-4294957296 3 3
E3 btr=-4294957296 4 3
E3 btr=-4294957296 5 3
E3 btr=-4294957296 6 3
E3 btr=-4294957296 7 3
E3 btr=-4294957296 0 4
E4 btr=-4294957296
K1 btr=-4294957296 6 0 5 0
K1 btr=-4294957296 6 0 4 0
K1 btr=-4294957296 6 1 5 1
K1 btr=-4294957296 6 1 4 1
K1 btr=-4294957296 6 2 5 2
K1 btr=-4294957296 6 2 4 2
K1 btr=-4294957296 6 3 5 3
K1 btr=-4294957296 6 3 4 3
K1 btr=-4294957296 6 4 5 4
K1 btr=-4294957296 6 4 4 4
K1 btr=-4294957296 6 5 5 5
K1 btr=-4294957296 6 5 4 5
K1 btr=-4294957296 6 6 5 6
K1 btr=-4294957296 6 6 4 6
K1 btr=-4294957296 6 7 5 7
K1 btr=-4294957296 6 7 4 7
K1 btr=-4294957296 7 1 5 0
K1 btr=-4294957296 7 1 5 2
K1 btr=-4294957296 7 6 5 5
K1 btr=-4294957296 7 6 5 7
E5 btr=-4294957296
E6 btr=188978571024
G3 btr=188978571024
G4 btr=188978571024

The change in value is actually due to a change in the upper 32 bits, but that's just from accessing the same element in the board array again.

[Evert]

Actually, reading the comments at the top of the function and seeing that it's called with (-1,-1,-1,-1) deliberately at one point suggests that this is in fact the correct fix:

Code: Select all

--- moves_old.c	2011-12-22 23:33:48.000000000 +0100
+++ moves.c	2011-12-22 23:33:50.000000000 +0100
@@ -1016,7 +1016,7 @@
 	}
 	board[rt][ft] = board[rf][ff];
 	board[rf][ff] = EmptySquare;
-    } else board[rt][ft] = ff; // [HGM] drop
+    } else if (rt >= 0) board[rt][ft] = ff; // [HGM] drop
 if(appData.debugMode) fprintf(debugFP, "E2 btr=%ld\n",blackTimeRemaining);
 
     /* For compatibility with ICS wild 9, we scan the board in the
@@ -1056,7 +1056,7 @@
 	} else {
 	    board[rt][ft] = captured;
 	}
-    } else board[rt][ft] = EmptySquare; // [HGM] drop
+    } else if (rt >= 0) board[rt][ft] = EmptySquare; // [HGM] drop
 
 if(appData.debugMode) fprintf(debugFP, "E6 btr=%ld\n",blackTimeRemaining);
     return cl.check;

It certainly seems to fix the problem for me.

[hgm]

I will have a look at it, but it is getting a bit late here. Negative from-rank is used to encode drop moves (where the from-file then holds the piece type). But in move disambiguation -1 is used to indicate a coordinate that was not given in the SAN.

It should never happen that a check test is performed on a partially unknown move, though. The bug seems to be that the check test is performed in the first place. I might have messed something up here in my attempt to accelerate the SAN decoding, not realizing themove generator might be called recursively (first for moves, an then the next ply for checktesting).

The original SAN decoding was extreely inefficient, first generating all pseudo-legal moves (even those with a piece of type other than specified in the move), testing all of them for legality (even those to squares other than the given to-square), and only then comparing the legal ones to see if they matched the SAN move. When loading a PGN with 40,000 games that really started to hurt. So I tried to accelerate it a bit, but it seems I have broken something in the process.

Luckily it had a very obvious visible effect (in 64-bit).