SF 1.9.1 question about search()

Discussion of chess software programming and technical issues.

Moderator: Ras

Ralph Stoesser
Posts: 408
Joined: Sat Mar 06, 2010 9:28 am

SF 1.9.1 question about search()

Post by Ralph Stoesser »

search.cpp, line 1303

Code: Select all

if (doFullDepthSearch)
{
    value = newDepth < ONE_PLY ? -qsearch<NonPV>(pos, ss+1, -(alpha+1), -alpha, DEPTH_ZERO, ply+1)
                               : - search<NonPV>(pos, ss+1, -(alpha+1), -alpha, newDepth, ply+1);

    // Step extra. pv search (only in PV nodes)
    // Search only for possible new PV nodes, if instead value >= beta then
    // parent node fails low with value <= alpha and tries another move.
    if (PvNode && value > alpha && value < beta)
        value = newDepth < ONE_PLY ? -qsearch<PV>(pos, ss+1, -beta, -alpha, DEPTH_ZERO, ply+1)
                                   : - search<PV>(pos, ss+1, -beta, -alpha, newDepth, ply+1);
}
I wonder why SF tests for

Code: Select all

value < beta 
within the condition

Code: Select all

if (PvNode && value > alpha && value < beta)
Though there are 3 lines of source comment I still don't get it. :(

value is the result of a null windows search around alpha.
How can you test this value against beta?

Thanks for any explanation.
Gerd Isenberg
Posts: 2251
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: SF 1.9.1 question about search()

Post by Gerd Isenberg »

The value < beta condition is not redundant. Beta is supposed to be greater than alpha+1, and you re-search only to get an exact score, but not if you are already >= beta failing soft from a zero window search around alpha. May be the redundancy is the boolean PvNode condition, which seems implicit if both other conditions were true. Otoh, since the leading PvNode pre-condition is rarely true inside the search, it is therefor a simple to predict branch and may pay off.
zamar
Posts: 613
Joined: Sun Jan 18, 2009 7:03 am

Re: SF 1.9.1 question about search()

Post by zamar »


I wonder why SF tests for

Code: Select all

value < beta 
within the condition

Code: Select all

if (PvNode && value > alpha && value < beta)
Though there are 3 lines of source comment I still don't get it. :(

value is the result of a null windows search around alpha.
How can you test this value against beta?

Thanks for any explanation.
(1) PvNode is a template parameter, so we want this code to be generated only for PV nodes. Speed optimization trick.

(2) Because this is PV-node then alpha +1 != beta.

(3) We want to make expensive PV-research only for likely PV. If value >= beta, this means that in parent node value <= alpha, so after all this variation is not going to be a new PV and we avoid expensive PV-research.

Hope this helps!
Joona Kiiski
Ralph Stoesser
Posts: 408
Joined: Sat Mar 06, 2010 9:28 am

Re: SF 1.9.1 question about search()

Post by Ralph Stoesser »

zamar wrote:...

If value >= beta, this means that in parent node value <= alpha, so after all this variation is not going to be a new PV and we avoid expensive PV-research.

Hope this helps!
No, my point is the following. Does value >= beta really mean a fail high?
To say it with code ...

Code: Select all

 if (PvNode && value > alpha)
 {
      if (value >= beta)
      {
          value = newDepth < ONE_PLY ? -qsearch<NonPV>(pos, ss+1, -beta, -(beta-1), DEPTH_ZERO, ply+1)
  	                                : - search<NonPV>(pos, ss+1, -beta, -(beta-1), newDepth, ply+1);
          if (value < beta)
  	        please_explain_why_this_may_happen(); // TODO
      }
      ...
 }
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: SF 1.9.1 question about search()

Post by mcostalba »

zamar wrote: (1) PvNode is a template parameter, so we want this code to be generated only for PV nodes. Speed optimization trick.
Not to be pedantic, but I would like to spend few words on this because templates are heavily used in SF and because it seems is one of the few engines with available sources that use this powerful C++ feature.

Just to be very very basic and up to the point, when you see something like

Code: Select all

v = my_func<SomeParameter>(a1, a2, a3);
it means that inside the function my_func() the parameter SomeParameter is evaluated at compile time, IOW suppose SomeParameter has boolean type, as is the case with PvNode in search().

A boolean type can have 2 values, 'true' and 'false'. What the compiler does is to create two copies of my_func(), the first that we can call my_func<true>() is a function built from my_func() where all the instances of SomeParameter are substituted at compile time by 'true', the second my_func<false>() is a function where all the instances of SomeParameter are substituted at compile time by 'false'.

Then, after this step, the compilers compiles the two obtained functions indipendently using the usual C/C++ rules.

So when in the code you see something like:

Code: Select all

void my_func<SomeParameter>(....) {

  .....

  if (SomeParameter && another_condition)
  {
       do_something();
  }

  .....
}
what happens is that the compiler creates two functions:

Code: Select all

void my_func<true>(....) {

  if (another_condition)
  {
       do_something();
  }

}

void my_func<false>(....) {

  // This will be removed during next compilations steps
  // because is unreachable code.
  if (false && another_condition)
  {
       do_something();
  }

}
And the two functions will then be compiled indipendently.

So the bottom line is that templates are a facility to generate a family of different functions starting from a single and shared...ehm..template ;-)

This greatly reduce source code redundancy with zero speed penalty overhead.
Rein Halbersma
Posts: 750
Joined: Tue May 22, 2007 11:13 am

Re: SF 1.9.1 question about search()

Post by Rein Halbersma »

Ralph Stoesser wrote:search.cpp, line 1303

Code: Select all

if (doFullDepthSearch)
{
    value = newDepth < ONE_PLY ? -qsearch<NonPV>(pos, ss+1, -(alpha+1), -alpha, DEPTH_ZERO, ply+1)
                               : - search<NonPV>(pos, ss+1, -(alpha+1), -alpha, newDepth, ply+1);

    // Step extra. pv search (only in PV nodes)
    // Search only for possible new PV nodes, if instead value >= beta then
    // parent node fails low with value <= alpha and tries another move.
    if (PvNode && value > alpha && value < beta)
        value = newDepth < ONE_PLY ? -qsearch<PV>(pos, ss+1, -beta, -alpha, DEPTH_ZERO, ply+1)
                                   : - search<PV>(pos, ss+1, -beta, -alpha, newDepth, ply+1);
}
I wonder why SF tests for

Code: Select all

value < beta 
within the condition

Code: Select all

if (PvNode && value > alpha && value < beta)
Though there are 3 lines of source comment I still don't get it. :(

value is the result of a null windows search around alpha.
How can you test this value against beta?

Thanks for any explanation.
In this thread last year http://www.talkchess.com/forum/viewtopi ... =&start=40, some of those issues were discussed. In particular, the reason why the re-search is with window [-beta, -alpha] rather than [-beta, -value] is because value-based prunings (null-move, futility, razoring etc.) can lead to search inconsistencies. As GCP pointed out, after a null-window search returning <value>, you cannot be sure that a re-search with [-beta, -value] will give at least <value>. The reason is that some moves could have fallen outside the pruning margins with respect to the partially widened window, but they may not be pruned during a fully widened window search.

So I think you are right that, for the same reasons as above, one also cannot fully trust value>=beta from a null-window search to imply that you will also get a fail-high from a widened window search.

A more conservative condition would be:

Code: Select all

if (PvNode && alpha < beta - 1 && value > alpha)
This will only do a re-search in PV-nodes (and compile the condition away in non-PV nodes), and only if alpha hasn't been raised to just below beta during the current iteration over moves. The costs of this change are more re-searches (the cases where the widened window search also gives value>=beta) and the benefits are less search inconsistencies (the cases where the widened window search gives value <= alpha) or better PVs (when the re-search raises alpha)
Ralph Stoesser
Posts: 408
Joined: Sat Mar 06, 2010 9:28 am

Re: SF 1.9.1 question about search()

Post by Ralph Stoesser »

Rein Halbersma wrote:
A more conservative condition would be:

Code: Select all

if (PvNode && alpha < beta - 1 && value > alpha)
Thanks, but the condition I would have expected is

Code: Select all

if (PvNode && value > alpha)
because alpha < beta - 1 should be always true in PV case.

The original condition may lead to slightly better ELO, I don't know, but I think it deserves a source comment about the "value < beta" test being of somewhat speculative nature...
Rein Halbersma
Posts: 750
Joined: Tue May 22, 2007 11:13 am

Re: SF 1.9.1 question about search()

Post by Rein Halbersma »

Ralph Stoesser wrote:
Rein Halbersma wrote:
A more conservative condition would be:

Code: Select all

if (PvNode && alpha < beta - 1 && value > alpha)
Thanks, but the condition I would have expected is

Code: Select all

if (PvNode && value > alpha)
because alpha < beta - 1 should be always true in PV case.

The original condition may lead to slightly better ELO, I don't know, but I think it deserves a source comment about the "value < beta" test being of somewhat speculative nature...
This is not true: value > alpha && value < beta automatically implies alpha < beta - 1 for PV nodes. If you remove that and replace it with value > alpha, this is no longer the case. If e.g. a previous move raised alpha to beta - 1, you no longer can re-search with a non-null window.
Ralph Stoesser
Posts: 408
Joined: Sat Mar 06, 2010 9:28 am

Re: SF 1.9.1 question about search()

Post by Ralph Stoesser »

Rein Halbersma wrote: This is not true: value > alpha && value < beta automatically implies alpha < beta - 1 for PV nodes. If you remove that and replace it with value > alpha, this is no longer the case. If e.g. a previous move raised alpha to beta - 1, you no longer can re-search with a non-null window.
Ok. You are right. Thanks for clarifying.
zamar
Posts: 613
Joined: Sun Jan 18, 2009 7:03 am

Re: SF 1.9.1 question about search()

Post by zamar »

Ralph Stoesser wrote:
zamar wrote:...

If value >= beta, this means that in parent node value <= alpha, so after all this variation is not going to be a new PV and we avoid expensive PV-research.

Hope this helps!
No, my point is the following. Does value >= beta really mean a fail high?
To say it with code ...

Code: Select all

 if (PvNode && value > alpha)
 {
      if (value >= beta)
      {
          value = newDepth < ONE_PLY ? -qsearch<NonPV>(pos, ss+1, -beta, -(beta-1), DEPTH_ZERO, ply+1)
  	                                : - search<NonPV>(pos, ss+1, -beta, -(beta-1), newDepth, ply+1);
          if (value < beta)
  	        please_explain_why_this_may_happen(); // TODO
      }
      ...
 }
Why could it not happen? It's called search instability. (Or maybe I missed your point again?)
Joona Kiiski