Strange draw scores when searching Fine#70

[konsolas]

Things are only getting stranger - disabling killer moves within the singular search doesn't help: the draw scores only go away if killer moves are disabled for the entire search.

This is incredibly confusing.

[Dann Corbit]

Here is something interesting I saw using your program:

uci
id name Topple v0.3.5
id author Vincent Tang
option name Hash type spin default 128 min 1 max 1048576
option name Threads type spin default 1 min 1 max 128
option name Clear Hash type button
option name Ponder type check default false
uciok
ucinewgame
position fen 8/k7/3p4/p2P1p2/P2P1P2/8/8/K7 w - -
eval
32
position fen 8/7k/4p3/2p1P2p/2P1P2P/8/8/7K w - -
eval
46
position fen k7/8/8/p2p1p2/P2p1P2/3P4/K7/8 b - -
eval
32
position fen 7k/8/8/2p1p2p/2P1p2P/4P3/7K/8 b - -
eval
46


The thing that is interesting is that the positions are simply the four mirrors and so the evaluations should all be identical.

[odomobo]

The thing that is interesting is that the positions are simply the four mirrors and so the evaluations should all be identical.

Topple's PSTs aren't all left-right symmetrical. I'm assuming this is because they were automatically tuned.

[odomobo]

Things are only getting stranger - disabling killer moves within the singular search doesn't help: the draw scores only go away if killer moves are disabled for the entire search.

This is incredibly confusing.

You might not be actually resolving the issue, but just adding chaos to the TT that prevents it from happening. An idea might be to seed your zobrist hash random function with different seeds, and see how many times you can get the error to happen, with different seeds.

[xr_a_y]

at the end of search you count legal move to look for check mate but if excluded, this seems wrong.

[konsolas]

Things are only getting stranger - disabling killer moves within the singular search doesn't help: the draw scores only go away if killer moves are disabled for the entire search.

This is incredibly confusing.

You might not be actually resolving the issue, but just adding chaos to the TT that prevents it from happening. An idea might be to seed your zobrist hash random function with different seeds, and see how many times you can get the error to happen, with different seeds.

I've tried various different seeds for my zobrist hashing, but the zero score issue occurs for all of them. I've even tried completely disabling transposition table writes within the singular search, which doesn't fix the issue either. I'm also using 64 bit keys in each hash entry so collisions should be very unlikely in a simple position like Fine#70.

I have noticed that some seeds take much longer to reach a zero score than others, e.g. trying to seed with std::random_device didn't cause a zero score until depth 54:

Code: Select all

info depth 53 seldepth 68 score cp 1817 time 44955 nodes 60001575 nps 1334000 hashfull 466 beta 0.959522 null 0.980416 p
v a1b1 a7b7 b1c1 b7c7 c1d1 c7c8 d1d2 c8c7 d2d3 c7b6 d3e2 b6c7 e2f3 c7d7 f3g3 d7e7 g3h4 e7f6 h4h5 f6f7 h5g5 f7g7 g5f5 g7f
7 f5g5 f7g7 f4f5 g7f7 f5f6 f7f8 g5f4 f8g8 f4g4 g8f8 g4g5 f8f7 g5f5 f7f8 f5e6 f8e8 e6d6 e8f7 d6e5 f7e8 d5d6 e8d8 f6f7 d8d
7 f7f8q d7c6 f8a8 c6d7 a8a5 d7e8
info currmove a1b1 currmovenumber 1
info depth 54 seldepth 69 score cp 0 lowerbound time 76321 nodes 106106961 nps 1390000 hashfull 688 beta 0.959291 null 0
.981597 pv a1b1 a7a8 b1b2 a8a7 b2b3 a7b6 b3c4 b6a6 c4b3 a6a7 b3c2 a7b8 c2d1 b8c7 d1e2 c7d8 e2e3 d8d7 e3f2 d7e8 f2g3 e8f7
 g3g2 f7g6 g2f3 g6h5 f3g3 h5h6 g3f3 h6g6 f3f2 g6h6 f2g3 h6h5 g3h3 h5h6 h3g3
info currmove a1b2 currmovenumber 2
info currmove a1a2 currmovenumber 3
info depth 54 seldepth 69 score cp 0 time 76450 nodes 106253700 nps 1389000 hashfull 689 beta 0.959288 null 0.981551 pv
a1b1 a7a8 b1b2 a8a7 b2b3 a7b6 b3c4 b6a6 c4b3 a6a7 b3c2 a7b8 c2d1 b8c7 d1e2 c7d8 e2e3 d8d7 e3f2 d7e8 f2g3 e8f7 g3g2 f7g6
g2f3 g6h5 f3g3 h5h6 g3f3 h6g6 f3f2 g6h6 f2g3 h6h5 g3h3 h5h6 h3g3

Do you think that hash collisions are causing the issue?

at the end of search you count legal move to look for check mate but if excluded, this seems wrong.

This is intended - if there are no legal moves other than the singular move and we are in check, there is only one check evasion so the position should be extended - the singular search will fail low with a mate score.

[konsolas]

Disabling killers solves the issue, independent of:

• The seed for zobrist hashes

• Whether or not the hash table is read from in singular extension search

• Whether or not the hash table is written to in singular extension search

Disabling either killers or singular extensions removes the draw scores.

How could what is essentially just move ordering cause invalid scores to be returned?

[Ronald]

I experience the same effect with rofChade when using singular extension. It looks like the killer moves/move ordering are responsible for the strange behaviour, I also cannot explain yet why that is.

I don't initialize killer moves for ply+1/ply+2 so the singular extension probably leads to changed killer moves which lead to the draw scores sometimes. If I save the killermoves of ply+1 before doing SE I don't get the draw score, so that probably explains why it also happens with SE.

When I don't sort the quiet moves (with history scores) I even get more draw scores, I will have to find out why....

[konsolas]

I experience the same effect with rofChade when using singular extension. It looks like the killer moves/move ordering are responsible for the strange behaviour, I also cannot explain yet why that is.

I don't initialize killer moves for ply+1/ply+2 so the singular extension probably leads to changed killer moves which lead to the draw scores sometimes. If I save the killermoves of ply+1 before doing SE I don't get the draw score, so that probably explains why it also happens with SE.

When I don't sort the quiet moves (with history scores) I even get more draw scores, I will have to find out why....

It's incredibly strange that move ordering would cause invalid scores, when alpha-beta and PVS should be provably correct, regardless of the move ordering.

In Topple, I still get draw scores even if I completely disable killers within the singular extension search.

Maybe others will have experienced this in the past and found a solution?
Notably, the two top open source alpha/beta engines, Ethereal and Stockfish, both use singular extensions and the killer heuristic, although neither has issues with draw scores in the Lasker-Reichhelm position.

[Ronald]

I just saw that Laser 1.7 also has draw scores at certain depths.

You say that you still get the draw scores when disabling killers in SE, do you mean not using killers in SE or do you (also) mean not updating the killers in SE (which will result in different killers in regular search)? For rofChade it works when I save the killers of ply/ply+1 before I start SE and put them back right after SE. So I still use and update killers in SE like in the regular search.