Stockfish 1.7

[gladius]

Awesome work, it's really cool to see such well commented and readable chess source code.

One thing that would be cool to consider in the future is perhaps releasing the version of SF with the tunable eval (not necessarily the eval tuner itself). I realize it's extra work for you, but it would be really fun to experiment on such a strong program.

Anyhow, congrats on another strong release .

Btw, I ported a slightly faster SEE over, which only gives boolean yes/no to Stockfish. The idea is from Strelka originally, but the code is original.

Code: Select all

// This version of SEE does not calculate the exact material imbalance, it just returns true = winning or equal, false = losing
int Position::see_sign(Move m) const {

  assert(move_is_ok(m));

  static const int seeValues[18] = {
    0, PawnValueMidgame, BishopValueMidgame, BishopValueMidgame,
    RookValueMidgame, QueenValueMidgame, QueenValueMidgame*10, 0,
    0, PawnValueMidgame, BishopValueMidgame, BishopValueMidgame,
    RookValueMidgame, QueenValueMidgame, QueenValueMidgame*10, 0,
    0, 0
  };

  const Square from = move_from(m);
  const Square to = move_to(m);

  const Color us = (from != SQ_NONE ? color_of_piece_on(from) : opposite_color(color_of_piece_on(to)));
  const Color them = opposite_color(us);

  assert(piece_on(from) != NO_PIECE);
  assert(color_of_piece_on(from) == us);
  assert(piece_on(to) == NO_PIECE ||
    (color_of_piece_on(from) != color_of_piece_on(to)) ||
    move_is_castle(m));

  const int fromValue = seeValues[piece_on(from)];
  const int toValue = seeValues[piece_on(to)];

  if &#40;fromValue <= toValue&#41;
  &#123;
    return 1;
  &#125;

  if &#40;st->epSquare == to && type_of_piece_on&#40;from&#41; == PAWN&#41;
  &#123;
    // e.p. captures are always pxp which is winning or equal.
    return 1;
  &#125;

  if &#40;move_is_castle&#40;m&#41;)
  &#123;
    // TODO&#58; castles are always "good".
    // We would have to change the destination square to be the final location for the rook.
    return 1;
  &#125;

  // We should only be making initially losing captures here
  //assert&#40;fromValue > seeValues&#91;PAWN&#93; || GetMoveType&#40;move&#41; == MoveTypePromotion || toValue == 0&#41;;

  const Bitboard enemyPieces = pieces_of_color&#40;them&#41;;

  // Pawn attacks
  // If any opponent pawns can capture back, this capture is probably not worthwhile &#40;as we must be using knight or above&#41;.
  if &#40;attacks_from<PAWN>&#40;to, us&#41; & pieces&#40;PAWN&#41; & enemyPieces&#41;
  &#123;
    return -1;
  &#125;

  // Knight attacks
  Bitboard attackingPieces = attacks_from<KNIGHT>&#40;to&#41; & pieces&#40;KNIGHT&#41;;
  const int captureDeficit = fromValue - toValue;
  // If any opponent knights can capture back, and the deficit we have to make up is greater than the knights value,
  // it's not worth it.  We can capture on this square again, and the opponent doesn't have to capture back.
  if &#40;captureDeficit > seeValues&#91;KNIGHT&#93; && &#40;attackingPieces & enemyPieces&#41;)
  &#123;
    return -1;
  &#125;

  // Bishop attacks
  Bitboard allPieces = pieces_of_color&#40;us&#41; | enemyPieces;
  clear_bit&#40;&allPieces, from&#41;;

  attackingPieces |= bishop_attacks_bb&#40;to, allPieces&#41; & &#40;pieces&#40;BISHOP&#41; | pieces&#40;QUEEN&#41;);
  if &#40;captureDeficit > seeValues&#91;BISHOP&#93; && &#40;attackingPieces & pieces&#40;BISHOP&#41; & enemyPieces&#41;)
  &#123;
    return -1;
  &#125;

  // Pawn defenses
  // At this point, we are sure we are making a "losing" capture.  The opponent can not capture back with a
  // pawn.  They cannot capture back with a bishop/knight and stand pat either.  So, if we can capture with
  // a pawn, it's got to be a winning or equal capture.
  if &#40;attacks_from<PAWN>&#40;to, them&#41; & pieces&#40;PAWN&#41; & pieces_of_color&#40;us&#41;)
  &#123;
    return 1;
  &#125;

  // Rook attacks
  attackingPieces |= rook_attacks_bb&#40;to, allPieces&#41; & &#40;pieces&#40;ROOK&#41; | pieces&#40;QUEEN&#41;);
  if &#40;captureDeficit > seeValues&#91;ROOK&#93; && &#40;attackingPieces & pieces&#40;ROOK&#41; & enemyPieces&#41;)
  &#123;
    return -1;
  &#125;

  // King attacks
  attackingPieces |= attacks_from<KING>&#40;to&#41; & pieces&#40;KING&#41;;

  // Make sure our original attacking piece is not included in the set of attacking pieces
  attackingPieces &= allPieces;

  if (!attackingPieces&#41;
  &#123;
    // No pieces attacking us, we have won
    return 1;
  &#125;

  // At this point, we have built a bitboard of all the pieces attacking the "to" square.  This includes
  // potential x-ray attacks from removing the "from" square piece, which we used to do the capture.

  // We are currently winning the amount of material of the captured piece, time to see if the opponent
  // can get it back somehow.  We assume the opponent can capture our current piece in this score, which
  // simplifies the later code considerably.
  int seeValue = toValue - fromValue;

  for (;;)
  &#123;
    int capturingPieceValue = -1;

    // Find the least valuable piece of the opponent that can attack the square
    for &#40;PieceType capturingPiece = KNIGHT; capturingPiece <= KING; capturingPiece++)
    &#123;
      const Bitboard attacks = pieces&#40;capturingPiece&#41; & attackingPieces & pieces_of_color&#40;them&#41;;
      if &#40;attacks&#41;
      &#123;
        // They can capture with a piece
        capturingPieceValue = seeValues&#91;capturingPiece&#93;;

        const Square attackingSquare = first_1&#40;attacks&#41;;
        clear_bit&#40;&attackingPieces, attackingSquare&#41;;
        clear_bit&#40;&allPieces, attackingSquare&#41;;
        break;
      &#125;
    &#125;

    if &#40;capturingPieceValue == -1&#41;
    &#123;
      // The opponent can't capture back - we win
      return 1;
    &#125;

    // Now, if seeValue < 0, the opponent is winning.  If even after we take their piece,
    // we can't bring it back to 0, then we have lost this battle.
    seeValue += capturingPieceValue;
    if &#40;seeValue < 0&#41;
    &#123;
      return -1;
    &#125;

    // Add any x-ray attackers
    attackingPieces |= 
      (&#40;bishop_attacks_bb&#40;to, allPieces&#41; & &#40;pieces&#40;BISHOP&#41; | pieces&#40;QUEEN&#41;)) |
      &#40;rook_attacks_bb&#40;to, allPieces&#41; & &#40;pieces&#40;ROOK&#41; | pieces&#40;QUEEN&#41;))) & allPieces;

    // Our turn to capture
    capturingPieceValue = -1;

    // Find the least valuable piece of the opponent that can attack the square
    for &#40;PieceType capturingPiece = KNIGHT; capturingPiece <= KING; capturingPiece++)
    &#123;
      const Bitboard attacks = pieces&#40;capturingPiece&#41; & attackingPieces & pieces_of_color&#40;us&#41;;
      if &#40;attacks&#41;
      &#123;
        // We can capture with a piece
        capturingPieceValue = seeValues&#91;capturingPiece&#93;;

        const Square attackingSquare = first_1&#40;attacks&#41;;
        clear_bit&#40;&attackingPieces, attackingSquare&#41;;
        clear_bit&#40;&allPieces, attackingSquare&#41;;
        break;
      &#125;
    &#125;

    if &#40;capturingPieceValue == -1&#41;
    &#123;
      // We can't capture back, we lose &#58;(
      return -1;
    &#125;

    // Assume our opponent can capture us back, and if we are still winning, we can stand-pat
    // here, and assume we've won.
    seeValue -= capturingPieceValue;
    if &#40;seeValue >= 0&#41;
    &#123;
      return 1;
    &#125;

    // Add any x-ray attackers
    attackingPieces |= 
      (&#40;bishop_attacks_bb&#40;to, allPieces&#41; & &#40;pieces&#40;BISHOP&#41; | pieces&#40;QUEEN&#41;)) |
      &#40;rook_attacks_bb&#40;to, allPieces&#41; & &#40;pieces&#40;ROOK&#41; | pieces&#40;QUEEN&#41;))) & allPieces;
  &#125;
&#125;

With some unit tests here:

Code: Select all

#include <cassert>

void CheckSee&#40;const std&#58;&#58;string &fen, const std&#58;&#58;string &move, int expected&#41;
&#123;
  Position position&#40;fen&#41;;

  Move captureMove = move_from_string&#40;position, move&#41;;
  assert&#40;move_is_ok&#40;captureMove&#41;);
  assert&#40;position.see_sign&#40;captureMove&#41; == expected&#41;;

  Position flipped;
  flipped.flipped_copy&#40;position&#41;;

  std&#58;&#58;string flippedMove&#40;move&#41;;
  flippedMove&#91;1&#93; = '1' + ('8' - flippedMove&#91;1&#93;);
  flippedMove&#91;3&#93; = '1' + ('8' - flippedMove&#91;3&#93;);
  captureMove = move_from_string&#40;flipped, flippedMove&#41;;
  assert&#40;move_is_ok&#40;captureMove&#41;);
  assert&#40;flipped.see_sign&#40;captureMove&#41; == expected&#41;;
&#125;

void SeeTests&#40;)
&#123;
  // Winning pawn capture on rook
  CheckSee&#40;"2r3k1/2r4p/1PNqb1p1/3p1p2/4p3/2Q1P2P/5PP1/1R4K1 w - - 0 37", "b6c7", 1&#41;;

  // Winning rook/queen capture on pawn
  CheckSee&#40;"2r3k1/2P4p/2Nqb1p1/3p1p2/4p3/2Q1P2P/5PP1/1R4K1 b - - 0 37", "c8c7", 1&#41;;
  CheckSee&#40;"2r3k1/2P4p/2Nqb1p1/3p1p2/4p3/2Q1P2P/5PP1/1R4K1 b - - 0 37", "d6c7", 1&#41;;

  // Winning rook/queen capture on knight
  CheckSee&#40;"6k1/2r4p/2Nqb1p1/3p1p2/4p3/2Q1P2P/5PP1/1R4K1 b - - 0 38", "c7c6", 1&#41;;
  CheckSee&#40;"6k1/2r4p/2Nqb1p1/3p1p2/4p3/2Q1P2P/5PP1/1R4K1 b - - 0 38", "d6c6", 1&#41;;

  // Losing rook/queen capture on knight &#40;revealed rook attack&#41;
  CheckSee&#40;"6k1/2r4p/2Nqb1p1/3p1p2/4p3/2Q1P2P/5PP1/2R3K1 b - - 0 38", "c7c6", -1&#41;;
  CheckSee&#40;"6k1/2r4p/2Nqb1p1/3p1p2/4p3/2Q1P2P/5PP1/2R3K1 b - - 0 38", "d6c6", -1&#41;;

  // Winning rook/queen capture on knight &#40;revealed bishop attack&#41;
  CheckSee&#40;"4b1k1/2rq3p/2N3p1/3p1p2/4p3/2Q1P2P/5PP1/2R3K1 b - - 0 38", "c7c6", 1&#41;;
  CheckSee&#40;"4b1k1/2rq3p/2N3p1/3p1p2/4p3/2Q1P2P/5PP1/2R3K1 b - - 0 38", "d7c6", 1&#41;;

  // Winning pawn capture on pawn
  CheckSee&#40;"2r3k1/2pq3p/3P2p1/b4p2/4p3/2R1P2P/5PP1/2R3K1 w - - 0 38", "d6c7", 1&#41;;

  // Losing rook capture on pawn
  CheckSee&#40;"2r3k1/2pq3p/3P2p1/b4p2/4p3/2R1P2P/5PP1/2R3K1 w - - 0 38", "c3c7", -1&#41;;

  // Losing queen capture on rook
  CheckSee&#40;"2r3k1/2p4p/3P2p1/q4p2/4p3/2R1P2P/5PP1/2R3K1 b - - 0 38", "a5c3", -1&#41;;

  // Losing rook capture on pawn
  CheckSee&#40;"1br3k1/2p4p/3P2p1/q4p2/4p3/2R1P2P/5PP1/2R3K1 w - - 0 38", "c3c7", -1&#41;;

  // Winning Q promotion &#40;non-capture&#41;
  CheckSee&#40;"4rrk1/2P4p/6p1/5p2/4p3/2R1P2P/5PP1/2R3K1 w - - 0 38", "c7c8q", 1&#41;;

  // Losing Q promotion &#40;non-capture&#41;
  CheckSee&#40;"r3rrk1/2P4p/6p1/5p2/4p3/2R1P2P/5PP1/2R3K1 w - - 0 38", "c7c8q", -1&#41;;

  // Knight capturing pawn defended by pawn
  CheckSee&#40;"K7/8/2p5/3p4/8/4N3/8/7k w - - 0 1", "e3d5", -1&#41;;

  // Knight capturing undefended pawn
  CheckSee&#40;"K7/8/8/3p4/8/4N3/8/7k w - - 0 1", "e3d5", 1&#41;;

  // Rook capturing pawn defended by knight
  CheckSee&#40;"K7/4n3/8/3p4/8/3R4/3R4/7k w - - 0 1", "d3d5", -1&#41;;

  // Rook capturing pawn defended by bishop
  CheckSee&#40;"K7/5b2/8/3p4/8/3R4/3R4/7k w - - 0 1", "d3d5", -1&#41;;

  // Rook capturing knight defended by bishop
  CheckSee&#40;"K7/5b2/8/3n4/8/3R4/3R4/7k w - - 0 1", "d3d5", 1&#41;;

  // Rook capturing rook defended by bishop
  CheckSee&#40;"K7/5b2/8/3r4/8/3R4/3R4/7k w - - 0 1", "d3d5", 1&#41;;
&#125;

[Tord Romstad]

What is the philosophical idea behind auto detecting then number of cores? Based on the things I read before, in the spirit of SF it looked like it should leave this task to the interface. Right?

Perhaps it would have been, if the UCI protocol had offered a standard parameter for setting the number of search threads, like it does for some other features shared by all or most chess programs (i.e. transposition table size). Unfortunately, it doesn't, and different MP capable chess programs use different parameter names for adjusting the number of threads.

It is therefore not possible to leave this task to the interface. Leaving the task to the user would be an option, but I don't like it, because the vast majority of computer users have no idea how many CPU cores they have (or even what a CPU core is).

[mcostalba]

Awesome work, it's really cool to see such well commented and readable chess source code.

One thing that would be cool to consider in the future is perhaps releasing the version of SF with the tunable eval (not necessarily the eval tuner itself). I realize it's extra work for you, but it would be really fun to experiment on such a strong program.

Anyhow, congrats on another strong release .

Btw, I ported a slightly faster SEE over, which only gives boolean yes/no to Stockfish. The idea is from Strelka originally, but the code is original.

Thanks Gary, I will gladly give it a try.

Can I ask how did you measured it is faster ? in SF see code (actually see_sign() is the function you should look at) has been tuned to death so I am very interesting in knowing the speed difference with your version.

[gladius]

Awesome work, it's really cool to see such well commented and readable chess source code.

One thing that would be cool to consider in the future is perhaps releasing the version of SF with the tunable eval (not necessarily the eval tuner itself). I realize it's extra work for you, but it would be really fun to experiment on such a strong program.

Anyhow, congrats on another strong release .

Btw, I ported a slightly faster SEE over, which only gives boolean yes/no to Stockfish. The idea is from Strelka originally, but the code is original.

Thanks Gary, I will gladly give it a try.

Can I ask how did you measured it is faster ? in SF see code (actually see_sign() is the function you should look at) has been tuned to death so I am very interesting in knowing the speed difference with your version.

I measured it using the benchmark command. Sadly, I did this a few months back, and have lost the measurements. It was a very slight speed win.

Now that I think of it, a more consistent way to do measure would be running a huge set of random positions/moves over both of them. Micro-benchmarks can be unreliable, but should give a good indication at least. I will give this a quick shot.

[zullil]

can i get some rough NPS estimates for stockfish? even though it's reporting 99% CPU usage its getting slaughtered by rybka. doesn't seem to gel... (-4 =2 +0)

i'm running it on a remote 2.93 ghz nehalem and it looks like it gets around ~6.5M nps out of the openings.

On a remote dual quad-core 2.26GHz Nehalem MacPro, I'm seeing nps=6733108 on a depth 28 search of the opening position (with OwnBook=false). This is with no GUI and the following UCI settings:

Code: Select all

Stockfish 1.7 64bit. By Tord Romstad, Marco Costalba, Joona Kiiski.
Good! CPU has hardware POPCNT. We will use it.
uci
id name Stockfish 1.7 64bit
id author Tord Romstad, Marco Costalba, Joona Kiiski

option name Use Search Log type check default true
option name Search Log Filename type string default SearchLog.txt
option name Book File type string default book.bin
option name Mobility (Middle Game) type spin default 100 min 0 max 200
option name Mobility (Endgame) type spin default 100 min 0 max 200
option name Pawn Structure (Middle Game) type spin default 100 min 0 max 200
option name Pawn Structure (Endgame) type spin default 100 min 0 max 200
option name Passed Pawns (Middle Game) type spin default 100 min 0 max 200
option name Passed Pawns (Endgame) type spin default 100 min 0 max 200
option name Space type spin default 100 min 0 max 200
option name Aggressiveness type spin default 100 min 0 max 200
option name Cowardice type spin default 100 min 0 max 200
option name Check Extension (PV nodes) type spin default 2 min 0 max 2
option name Check Extension (non-PV nodes) type spin default 1 min 0 max 2
option name Single Evasion Extension (PV nodes) type spin default 2 min 0 max 2
option name Single Evasion Extension (non-PV nodes) type spin default 2 min 0 max 2
option name Mate Threat Extension (PV nodes) type spin default 0 min 0 max 2
option name Mate Threat Extension (non-PV nodes) type spin default 0 min 0 max 2
option name Pawn Push to 7th Extension (PV nodes) type spin default 1 min 0 max 2
option name Pawn Push to 7th Extension (non-PV nodes) type spin default 1 min 0 max 2
option name Passed Pawn Extension (PV nodes) type spin default 1 min 0 max 2
option name Passed Pawn Extension (non-PV nodes) type spin default 0 min 0 max 2
option name Pawn Endgame Extension (PV nodes) type spin default 2 min 0 max 2
option name Pawn Endgame Extension (non-PV nodes) type spin default 2 min 0 max 2
option name Randomness type spin default 0 min 0 max 10
option name Minimum Split Depth type spin default 7 min 4 max 7
option name Maximum Number of Threads per Split Point type spin default 5 min 4 max 8
option name Threads type spin default 8 min 1 max 8
option name Hash type spin default 2048 min 4 max 8192
option name Clear Hash type button
option name New Game type button
option name Ponder type check default true
option name OwnBook type check default true
option name MultiPV type spin default 1 min 1 max 500
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
uciok
setoption name OwnBook value false
ucinewgame
isready
readyok
go depth 28

One thing to check is your Minimum Split Depth setting. If Stockfish detects 8 cores, it sets this to 7 by default. Otherwise it defaults to 4. Since there seems to be a bug in core detection, you may be running with Minimum Split Depth equal to 4, which would lead to much weaker play than with Minimum Split Depth equal to 6 or 7.

[zullil]

The problem that is arising is that function HT_enabled() is broken at least for some i7 CPU models.

So we have two options:

1) Fix HT_enabled() to works always: but this is very difficult because, reading Intel documentation, the proper way to detect HT is brain damaged complex (this time Intel made a real idiotic decision with HT detection design). Current routine works always on older types of CPU, but newer ones require a much more complex approach.

2) Remove HT_enabled() entirely and revert to 1.6.3 behaviour. In this case people should remember of disable HT or, when HT is enabled to manually set the correct number of cores through UCI options "threads" to the number of real cores (that normally is the half of what is reported with HT enabled).

Code: Select all

  // Detect number of cores
  __cpuid(CPUInfo, 4);
  nCores = 1 + ((CPUInfo[0] >> 26) & 0x3F);

worked correctly on my machine, with HT enabled.

[gladius]

Now that I think of it, a more consistent way to do measure would be running a huge set of random positions/moves over both of them. Micro-benchmarks can be unreliable, but should give a good indication at least. I will give this a quick shot.

Okay, basic results are in:

Total time (ms) : 154103 (my version, 32 bit)
Total time (ms) : 159646 (original see_sign, 32 bit)

The super simple testing harness was:

Code: Select all

int seePerft(Position &pos, int ply) {
  if (ply == 0) return 0;
  MoveStack moves[256];

  int result = 0;
  MoveStack *lastMove = generate_moves(pos, moves);
  MoveStack *at = &moves[0];
  while (at != lastMove) {
    StateInfo si;
    if (pos.move_is_capture(at->move)) {
      const int see_sign = pos.see_sign(at->move);
      result += see_sign < 0 ? -1 &#58; &#40;see_sign ? 1 &#58; 0&#41;;
    &#125;
    pos.do_move&#40;at->move, si&#41;;
    result += seePerft&#40;pos, ply - 1&#41;;
    pos.undo_move&#40;at->move&#41;;
    at++;
  &#125;
  return result;
&#125;

And I ran it at ply = 5, for each of the benchmark positions.

However, the result count did not match, so I'm going to do a little double checking there.

[JManion]

I tried to make the stockfish a uci engine. I did this under Fritz 9. When I try to sue the engine it does not show up. However if I open up chessbase 9 I see the engine.

[gladius]

Figured out what was going on with the different results, slight mistake in seePerft. Any captures >= 0 should be +1, otherwise - 1. Now the numbers match up .

Also, I pasted a slightly old version of the code in mistakenly. This chunk of code should be below the rook captures.

Code: Select all

  // Pawn defenses
  // At this point, we are sure we are making a "losing" capture.  The opponent can not capture back with a
  // pawn.  They cannot capture back with a bishop/knight/rook and stand pat either.  So, if we can capture with
  // a pawn, it's got to be a winning or equal capture.
  if &#40;attacks_from<PAWN>&#40;to, them&#41; & pieces&#40;PAWN&#41; & pieces_of_color&#40;us&#41;)
  &#123;
    return 1;
  &#125;

[Alex Lobanov]

great engine!

whether implemented a similar mode of analysis "preserve analyse"?
evaluation of the hash memory quickly lost.

thanks!

будет ли реализован режим анализа аналогичный "preserve analyse"?
оценки из хэш-памяти быстро теряются.