TalkChess.com

Code: Select all

Crafty 25.0 

*  This version contains a major rewrite of the parallel search code, now referred to as Generation II. It has a more lightweight split algorithm, that costs the parent MUCH less effort to split the search. The key is that now the individual "helper" threads do all the work, allocating a split block, copying the data from the parent, etc., rather than the parent doing it all. Gen II based on the DTS "late-join" idea so that a thread will try to join existing split points before it goes to the idle wait loop waiting for some other thread to split with it. In fact, this is now the basis for the new split method where the parent simply creates a split point by allocating a split block for itself, and then continuing the search, after the parent split block is marked as "joinable". Now any idle threads just "jump in" without the parent doing anything else, which means that currently idle threads will "join" this split block if they are in the "wait-for-work" spin loop, and threads that become idle also join in exactly the same way. This is MUCH more efficient, and also significantly reduces the number of split blocks needed during the search. We also now pre-split the search when we are reasonably close to the root of the tree, which is called a "gratuitous split. This leaves joinable split points laying around so that whenever a thread becomes idle, it can join in at these pre-existing split points immediately. We now use a much more conservative approach when dealing with fail highs at the root. Since LMR and such have introduced greater levels of search instability, we no longer trust a fail-high at the root if it fails low on the research. We maintain the last score returned for every root move, along with the PV. Either an exact score or the bound score that was returned. At the end of the iteration, we sort the root move list using the backed-up score as the sort key, and we play the move with the best score. This solves a particularly ugly issue where we get a score for the first move, then another move fails high, but then fails low and the re-search produces a score that is actually WORSE than the original best move. We still see that, but we always play the best move now. One other efficiency trick is that when the above happens, the search would tend to be less efficient since the best score for that fail-high/fail-low move is actually worse than the best move/score found so far. If this happens, the score is restored to the original best move score (in Search()) so that we continue searching with a good lower bound, not having to deal with moves that would fail high with this worse value, but not with the original best move's value. 

*  We also added a new method to automatically tune the new SMP parameters. The command is autotune and "help autotune" will explain how to run it. 

*  In addition , we did a complete re-factor of pawn evaluation code. There were too many overlapping terms that made tuning difficult. Now a pawn is classified as one specific class, there is no overlap between classes, which simplifies the code significantly. The code is now easier to understand and modify. In addition, the passed pawn evaluation was rewritten and consolidates all the passed pawn evaluation in one place. The evaluation used to add a bonus for rooks behind passed pawns in rook scoring, blockading somewhere else, etc. All of this was moved to the passed pawn code to make it easier to understand and modify. 

*  Added a limited version of late move pruning (LMP) for the last two plies. Once a set number of moves have been searched with no fail high, non-interesting moves are simply skipped in a way similar to futility pruning. 

*  We had a minor change to history counters that now rely on a "saturating counter" idea. I wanted to avoid the aging idea, and it seemed to not be so clear that preferring history moves by the depth at which they were good was the way to go. I returned to a history counter idea I tested around 2005 but discarded, namely using a saturating counter. The idea is that a center value (at present 1024) represents a zero score. Decrementing it makes it worse, incrementing it makes it better. But to make it saturate at either end, I only reduce the counter by a fraction of its distance from the saturation point so that once it gets to either extreme value, it will not be modified further avoiding wrap-around. This basic idea was originally reported by Mark Winands in 2005. It seems to provide better results (slightly) on very deep searches. One impetus for this was an intent to fold this into a move so that I could sort the moves rather than doing the selection approach I currently use. However, this had a bad effect on testing, since history information is dynamic and is constantly changing, between two moves at the same ply in fact. The sort fixed the history counters to the value at the start of that ply. This was discarded after testing, but the history counter based on the saturating counter idea seemed to be OK and was left in even though it produced minimal Elo gain during testing. 

*  We change to the way moves are counted, to add a little more consistency to LMR. Now Next*() returns an order number that starts with 1 and monotonically increases, this order number is used for LMR and such decisions that vary things based on how far down the move list something occurs. Root move counting was particularly problematic with parallel searching, now things are at least "more consistent". The only negative impact is that now the move counter gets incremented even for illegal moves, but testing showed this was a no-change change with one thread, and the consistency with multiple threads made it useful. 

*  Added the "counter-move" heuristic for move ordering (Jos Uiterwijk, JICCA) which simply remembers a fail high move and the move at the previous ply. If the hash, captures or killer moves don't result in a fail high, this move is tried next. No significant cost, seems to reduce tree size noticeably. Added a follow-up idea based on the same idea, except we pair a move that fails high with the move two plies back, introducing a sort of "connection" between them. This is a sort of "plan" idea where the first move of the pair enables the second move to fail high. The benefit is that this introduces yet another pair of good moves that get ordered before history moves, and is therefore not subject to reduction. I have been unable to come up with a reference for this idea, but I believe I first saw it somewhere around the time Fruit showed up, I am thinking perhaps in the JICCA/JICGA. Any reference would be appreciated. 

*  A minor change to the way the PV and fail-hi/fail-low moves are displayed when pondering. 

*  Crafty now adds the ponder move to the front of the PV enclosed in parentheses so that it is always visible in console mode. The depths are reaching such extreme numbers the ponder move scrolls off the top of the screen when running in console mode or when "tail -f" is used to watch the log file while a game is in progress. This is a bit trickier than you might think since Crafty displays the game move numbers in the PV. 

*  The penalty for pawns on same color as bishop now only applies when there is one bishop.