Yet another tt issue

[Joost Buijs]

I think more and more this is something due to time control. If I use very small TC the problem seems to be worst while at longer TC it seems less...
I'll look at "stopflag" thing again probably ...

When the stop-flag has been set you probably unwind the stack and doing so you have to make sure that you don't store anything in the TT during that process because it may pollute the TT with garbage, at least this is a bug I encountered in the past. To avoid checking the stopflag everywhere I use a throw/catch mechanism in my latest engines. I always thought that this would be slow or unreliable but actually it works very well, even in a multi-threaded environment.

[xr_a_y]

When the stop-flag has been set you probably unwind the stack and doing so you have to make sure that you don't store anything in the TT during that process because it may pollute the TT with garbage, at least this is a bug I encountered in the past.

Indeed, I never store anything if stopFlag is set. I also do not update the bestMove if stopFlag is set. A good idea was given a little above in the thread : update the bestMove anyway if it's score is better than the TTmove's score but I didn't try it yet.

To avoid checking the stopflag everywhere I use a throw/catch mechanism in my latest engines. I always thought that this would be slow or unreliable but actually it works very well, even in a multi-threaded environment.

I agree that using exception might be both beautiful and not that slow, but for now I simply check for stopFlag just at the beginning of QSearch, NegaSearch, and at some point in WindowSearch.

[xr_a_y]

I am wondering if it can be a time management issue. Very often (when pondering even more), when using TT at root, the move that is used is the TT one, because the first iterative depth that is not skipped is to long to fit in the available search time.

Here you are mentioning a point that seems to apply to several engines. People sometimes tend to not playing a best move from an incomplete iteration. I see no convincing argument for that. The first move that you search in iteration N is the best move M1 from iteration N-1. The following can occur:

a) If you get a timeout while still searching the subtree of M1 then you play M1 since it is still the best move from previous iteration.

b) If you finish searching the M1 subtree, start searching M2, and get a timeout before finishing it, then you obviously play M1 as well (best move from previous iteration and also best move of current one so far).

c) The interesting situation starts after finishing the M2 subtree and starting M3. Now you have completely searched the subtrees of two root moves in the new iteration. If score(M2) <= score(M1) then M1 still remains your best move. So now let's assume score(M2) > score(M1), making M2 your new best move of current iteration. Why would you *not* play M2 in the game, then, in case of a timeout? You have sufficient information to state that M2 is better than M1 under your search conditions. There might be even better moves but if you don't have enough time to search them then why ignore the information you already have?

It is getting slightly more complex if the new iteration is failing low, and you want to react properly to it. But in the "normal" case mentioned above it should be obvious that it is a valid strategy to play the best move from an incomplete iteration.

Ok i've implement this (keep the bestmove even if all root moves haven't been explored), and this indeed worth some elo (around 25)

[Michael Sherwin]

I have long suspected that RomiChess has problems with its TT. Though when I go over the code it looks okay. I read in this thread that some might do TT cutoffs only on zero window nodes and since that was easy to do it was tried. RomiChessX lost about 1.5 ply in search depth compared to RomiChessP3n, however RomiChessX is +1 in a match against RomiChessP3n at the moment. Here is a nice game by RCX against P3n. There was some middle game wobbling while RCX was searching for a plan. I been watching the Sinquefield Cup and the game reminded me of a Carlson style grind, . (Disclaimer: I'm not saying that RCX is anywhere near as strong as Carlson, .) The book moves end with 4. Bg2.

[pgn][Event "Isa"] [Site "MASTER"] [Date "2018.08.24"] [Round "4"] [White "RomiChess64P3n"] [Black "RomiChessX"] [Result "0-1"] [BlackElo "2432"] [ECO "E11"] [Opening "Bogo-Indian"] [Time "11:49:06"] [Variation "Nimzowitsch, 5.g3 Nc6 Main Line, 8...O-O"] [WhiteElo "2424"] [TimeControl "30+1"] [Termination "normal"] [PlyCount "135"] [WhiteType "human"] [BlackType "human"] 1. d4 f5 2. c4 Nf6 3. g3 e6 4. Bg2 Bb4+ 5. Bd2 Bxd2+ 6. Nxd2 O-O 7. Ngf3 d6 8. Nb3 Qe7 9. h3 e5 10. d5 a5 11. a4 Bd7 12. Nfd2 e4 13. e3 Qe5 14. Nd4 c5 15. dxc6 Nxc6 16. Nxc6 bxc6 17. Qc2 Rab8 18. Rb1 Be6 19. O-O Rb4 20. b3 Nd7 21. Rfd1 Nc5 22. Qa2 Bf7 23. Bf1 Bh5 24. Rdc1 Nd3 25. Rc2 Rfb8 26. c5 d5 27. Qa1 Kh8 28. Qxe5 Nxe5 29. Kh2 Rf8 30. Ba6 Ra8 31. Bf1 Rg8 32. Kg1 Rf8 33. Ba6 Ra8 34. Bf1 Kg8 35. h4 Rab8 36. Kh2 Kh8 37. Kg1 Re8 38. Rc3 Rf8 39. Ba6 Ra8 40. Bf1 Bg4 41. Kh2 Rab8 42. Rc2 R4b7 43. Ba6 Ra7 44. Bf1 Nd3 45. Kg1 Rf7 46. Rc3 Ne5 47. Ba6 Ra7 48. Bf1 Rab7 49. Ba6 Rb4 50. Rc2 Ra8 51. Bf1 g6 52. Kh1 h6 53. Kh2 Bh5 54. Ra1 Rab8 55. Rb1 g5 56. hxg5 hxg5 57. Kg2 Bf3+ 58. Kg1 R4b7 59. Bg2 Be2 60. Ra1 Bd3 61. Rc3 g4 62. Rd1 Rh7 63. Re1 Rbb7 64. Rcc1 Kg8 65. Rc3 Rh5 66. Nf1 Rbh7 67. Nh2 Rxh2 68. Rxd3 {White resigns} 0-1 [\pgn] Ah, but what to do now? Maybe rip out all the TT code and start over using code from another open source engine as a guide?[/pgn]

[Joost Buijs]

Nightmare also plays somewhat better when I disable TT cutoffs in the PV, I have the impression that it helps a little bit to avoid the nasty GHI problem, especially during endgames, I can't find any other reason why it helps.