extensions for quick solutions?

[outAtime]

I was running some test positions and came across a position for which
the solution is was quickly found by engine A at ply 2, yet was not found
untill ply 10 on engine B.
I changed the move ordering on engine B to place captures at a higher
priority, since the solution to the puzzle was to make a capture. (Knight for Pawn leading to +2.00 or more advantage.)
After that, the move was considered at ply 2, but not searched deeply enough to show the advantage, and skipped again untill later plys.
so:
engine B looks at the move and thinks its bad :

Code: Select all

2     -50     1       77  f5g7 ??
2     -17     1       81  f1f2 !
2      31     3      153  f1f2 a6a5
2      31     6      493  f1f2 a6a5
3     -19     6      505  f5g7 ??
3     -18     8      806  f1f2 !
3      45     9     1202  f1f2 a6a5 g2h3
3      45    15     1601  f1f2 a6a5 g2h3

Until later plys (10):

Code: Select all

10     111  1998  5542512  f5g7 !!
stat01: 2088 5738496 10 0 0
stat01: 2187 6070272 10 0 0
stat01: 2289 6397952 10 0 0
stat01: 2390 6758400 10 0 0
stat01: 2489 7098368 10 0 0
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2550  7245303  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7

Whereas engine A looks at the solution and sees it is good right away :

Code: Select all

info multipv 1 depth 1 score cp 45 time 0 nodes 1112 pv f5e3 b8b7
info multipv 1 depth 2 score cp 27 time 94 nodes 1628 pv f5e3 b8b7 e3c4
info multipv 1 depth 2 score cp 149 time 94 nodes 2000 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 3 score cp 149 time 94 nodes 2798 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 4 score cp 74 time 109 nodes 7733 pv f5g7 e5g7 f3f7 g8h8 f7h5 h8g8 f1f7 e8e4
info multipv 1 depth 5 score cp 163 time 203 nodes 14783 pv f5g7 e8f8 g7f5 b8b5 f5h6 g8h7 f3f5 h7g7 h6f7
info multipv 1 depth 6 score cp 187 time 203 nodes 19283 pv f5g7 e8f8 g7f5 b8b7 h4h5 e5b2 

I was thinking that perhaps adding a capture extension could solve the problem of the correct solution being skipped so early? Engine B surely
seems to not look at this move in much detail before going to another move, where engine A seems to be looking at it deeper and sees an advantage.

Any ideas? Ultimately I suppose I'd like the engine to consider captures very early, and to extend them.
I dont know how to write a capture extension. (in c, I am writing in)
Even on ply 6 engine B gives only : f5g7??
where engine A gives : f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8 - as early as ply 2.
I'd like engine B to extend this line as A does, and not give just 1 move without any search.
How can I get it to extend the line?

[Edsel Apostol]

I was running some test positions and came across a position for which
the solution is was quickly found by engine A at ply 2, yet was not found
untill ply 10 on engine B.
I changed the move ordering on engine B to place captures at a higher
priority, since the solution to the puzzle was to make a capture. (Knight for Pawn leading to +2.00 or more advantage.)
After that, the move was considered at ply 2, but not searched deeply enough to show the advantage, and skipped again untill later plys.
so:
engine B looks at the move and thinks its bad :

Code: Select all

2     -50     1       77  f5g7 ??
2     -17     1       81  f1f2 !
2      31     3      153  f1f2 a6a5
2      31     6      493  f1f2 a6a5
3     -19     6      505  f5g7 ??
3     -18     8      806  f1f2 !
3      45     9     1202  f1f2 a6a5 g2h3
3      45    15     1601  f1f2 a6a5 g2h3

Until later plys (10):

Code: Select all

10     111  1998  5542512  f5g7 !!
stat01: 2088 5738496 10 0 0
stat01: 2187 6070272 10 0 0
stat01: 2289 6397952 10 0 0
stat01: 2390 6758400 10 0 0
stat01: 2489 7098368 10 0 0
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2550  7245303  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7

Whereas engine A looks at the solution and sees it is good right away :

Code: Select all

info multipv 1 depth 1 score cp 45 time 0 nodes 1112 pv f5e3 b8b7
info multipv 1 depth 2 score cp 27 time 94 nodes 1628 pv f5e3 b8b7 e3c4
info multipv 1 depth 2 score cp 149 time 94 nodes 2000 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 3 score cp 149 time 94 nodes 2798 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 4 score cp 74 time 109 nodes 7733 pv f5g7 e5g7 f3f7 g8h8 f7h5 h8g8 f1f7 e8e4
info multipv 1 depth 5 score cp 163 time 203 nodes 14783 pv f5g7 e8f8 g7f5 b8b5 f5h6 g8h7 f3f5 h7g7 h6f7
info multipv 1 depth 6 score cp 187 time 203 nodes 19283 pv f5g7 e8f8 g7f5 b8b7 h4h5 e5b2 

I was thinking that perhaps adding a capture extension could solve the problem of the correct solution being skipped so early? Engine B surely
seems to not look at this move in much detail before going to another move, where engine A seems to be looking at it deeper and sees an advantage.

Any ideas? Ultimately I suppose I'd like the engine to consider captures very early, and to extend them.
I dont know how to write a capture extension. (in c, I am writing in)
Even on ply 6 engine B gives only : f5g7??
where engine A gives : f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8 - as early as ply 2.
I'd like engine B to extend this line as A does, and not give just 1 move without any search.
How can I get it to extend the line?

Please post also the details of your search. It seems to me that your doing null move in PV nodes, or there might me something broken in your search.

[MattieShoes]

Is the engine really calculating multiple principle variations? That'll change results by itself - It will end up searching shallower but more fully in order to calculate several principle variations, which should make its play weaker if this were an actual game. . .

Edit:
A simple capture extension is easy to write -- somewhere early in the search function:

Code: Select all

if(lastMoveWasCapture())
    depth++;

Edit Edit:
Capture extensions don't necessarily make an engine stronger though -- You have to weigh the cost with the benefit. The benefit is obviously that it'll search deeper for moves with captures, perhaps finding tactics it would have otherwise missed, but the cost is it'll spend a LOT more time on lines with capture sequences, which means a lot LESS time on lines without capture sequences. A chess engine spends nearly 100% of its time looking at bad moves. I think a blanket capture extension generally comes out bad.

[Tony]

I was running some test positions and came across a position for which
the solution is was quickly found by engine A at ply 2, yet was not found
untill ply 10 on engine B.
I changed the move ordering on engine B to place captures at a higher
priority, since the solution to the puzzle was to make a capture. (Knight for Pawn leading to +2.00 or more advantage.)
After that, the move was considered at ply 2, but not searched deeply enough to show the advantage, and skipped again untill later plys.
so:
engine B looks at the move and thinks its bad :

Code: Select all

2     -50     1       77  f5g7 ??
2     -17     1       81  f1f2 !
2      31     3      153  f1f2 a6a5
2      31     6      493  f1f2 a6a5
3     -19     6      505  f5g7 ??
3     -18     8      806  f1f2 !
3      45     9     1202  f1f2 a6a5 g2h3
3      45    15     1601  f1f2 a6a5 g2h3

Until later plys (10):

Code: Select all

10     111  1998  5542512  f5g7 !!
stat01: 2088 5738496 10 0 0
stat01: 2187 6070272 10 0 0
stat01: 2289 6397952 10 0 0
stat01: 2390 6758400 10 0 0
stat01: 2489 7098368 10 0 0
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2550  7245303  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7

Whereas engine A looks at the solution and sees it is good right away :

Code: Select all

info multipv 1 depth 1 score cp 45 time 0 nodes 1112 pv f5e3 b8b7
info multipv 1 depth 2 score cp 27 time 94 nodes 1628 pv f5e3 b8b7 e3c4
info multipv 1 depth 2 score cp 149 time 94 nodes 2000 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 3 score cp 149 time 94 nodes 2798 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 4 score cp 74 time 109 nodes 7733 pv f5g7 e5g7 f3f7 g8h8 f7h5 h8g8 f1f7 e8e4
info multipv 1 depth 5 score cp 163 time 203 nodes 14783 pv f5g7 e8f8 g7f5 b8b5 f5h6 g8h7 f3f5 h7g7 h6f7
info multipv 1 depth 6 score cp 187 time 203 nodes 19283 pv f5g7 e8f8 g7f5 b8b7 h4h5 e5b2 

I was thinking that perhaps adding a capture extension could solve the problem of the correct solution being skipped so early? Engine B surely
seems to not look at this move in much detail before going to another move, where engine A seems to be looking at it deeper and sees an advantage.

Any ideas? Ultimately I suppose I'd like the engine to consider captures very early, and to extend them.
I dont know how to write a capture extension. (in c, I am writing in)
Even on ply 6 engine B gives only : f5g7??
where engine A gives : f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8 - as early as ply 2.
I'd like engine B to extend this line as A does, and not give just 1 move without any search.
How can I get it to extend the line?

At first glance this looks like a position (2 ?) from BT2630. iirc generating check giving moves in quiescence makes the difference.

Tony

[outAtime]

Yes, this is position 1 from BT2630, where the correct answer is Nxg7.
Fruit 2.3.1 is the engine I randomly selected to compare (engine A).
Engine B is an old version of Sjeng. In fact all versions (free) of Sjeng I have tested fail to examine the correct solution in early plys, or even to consider it unless I make a change to the move ordering. It is very possible that some versions do use null in pv search :

Code: Select all

 if (phase != Endgame && (is_null == FALSE) && !incheck && donull && (threat == FALSE) && (Variant != Suicide))
    {

so in this version I would say pv uses null.
Whereas Sjeng 11.2 does not :

Code: Select all

 if ((is_null == NONE)
      && ((phase != Endgame) || ((phase == Endgame) && (depth <= 6)))
      && !incheck && donull && !searching_pv
      && (threat == FALSE)
      && (((Variant != Suicide) && (Variant != Losers)) 
	  || (Variant == Losers && moves[0].captured == npiece)))

And although 11.2 considers Nxg7 early, the score for this move does not get past about +0.40 cp.

Code: Select all

2      40     5      142  Nxg7 Bxg7
2      40     5      142  Nxg7 Bxg7
2      40     8      241  Nxg7 Bxg7
3      46    10      476  Nxg7 Kxg7 <Qxf7+>
3      46    10      476  Nxg7 Kxg7 <Qxf7+>
3      46    13      718  Nxg7 Kxg7 <Qxf7+>
4      31    16     1665  Nxg7 Bxg7 Qxf7+ Kh8 Qh5+ <Kg8>
4      31    16     1665  Nxg7 Bxg7 Qxf7+ Kh8 Qh5+ <Kg8>
4      32    22     3812  Kh3 !
4      34    26     5236  Kh3 Ra7 Rf2 Rd7
4      34    27     6338  Kh3 Ra7 Rf2 Rd7
5      43    46    13523  Kh3 Qb7 Rf2
5      43    46    13523  Kh3 Qb7 Rf2
5      43    61    25883  Kh3 Qb7 Rf2
6      43    76    48607  Kh3 Ra7 Rf2 Rd7 Rd1 Red8 Rxd7
6      43    76    48607  Kh3 Ra7 Rf2 Rd7 Rd1 Red8 Rxd7
6      44    99    62750  Qb3 !
6      44   109    74658  Qb3 Qb6 Nxg7 Bxg7 Qxf7+ Kh8 Rf6
6      44   120    84584  Qb3 Qb6 Nxg7 Bxg7 Qxf7+ Kh8 Rf6

so it seems the pv null search may have some effect, but is not the complete answer? Generating check giving moves in quiesence makes the
difference says Tony, but I thought it was good to keep check out of q..?
I could try it though. Any other ideas here? Im going to try the capture extension and see if that helps any.

[outAtime]

Isn't that a re-capture extension? Ive looked for examples of c coded capture extensions without any luck. I was just wondering because the code says :

Code: Select all

if(lastMoveWasCapture()) 
    depth++;

I was wondering how to extend captures while seaching for a move
to make for the side to move, and not in resonse to a move just made... Just wondering how it works.
Interesting to see that Crafty is never tested by any of these types of positions. I guess Im not really trying correctly to improve anything here, but more so to learn what does what. I really appreciate the help and advice here, thanks all.

[bob]

Isn't that a re-capture extension? Ive looked for examples of c coded capture extensions without any luck. I was just wondering because the code says :

Code: Select all

if(lastMoveWasCapture()) 
    depth++;

I was wondering how to extend captures while seaching for a move
to make for the side to move, and not in resonse to a move just made... Just wondering how it works.
Interesting to see that Crafty is never tested by any of these types of positions. I guess Im not really trying correctly to improve anything here, but more so to learn what does what. I really appreciate the help and advice here, thanks all.

Not sure what you mean by "is never tested..." In general, the extension you are looking for is not just a capture extension, that will weaken your engine significantly. The "recapture" extension has been used, and was developed to reduce horizon-effect moves back in the 5-8 ply search depth days where a simple capture/recapture would burn two plies and allow one side to "hide" a loss by pushing it over the horizon. This is hardly an issue with today's 20+ ply searches. I've removed the recapture extension because it shows a loss in testing over real games.

BTW Crafty finds the above with a 1 ply search....

[bob]

I was running some test positions and came across a position for which
the solution is was quickly found by engine A at ply 2, yet was not found
untill ply 10 on engine B.
I changed the move ordering on engine B to place captures at a higher
priority, since the solution to the puzzle was to make a capture. (Knight for Pawn leading to +2.00 or more advantage.)
After that, the move was considered at ply 2, but not searched deeply enough to show the advantage, and skipped again untill later plys.
so:
engine B looks at the move and thinks its bad :

Code: Select all

2     -50     1       77  f5g7 ??
2     -17     1       81  f1f2 !
2      31     3      153  f1f2 a6a5
2      31     6      493  f1f2 a6a5
3     -19     6      505  f5g7 ??
3     -18     8      806  f1f2 !
3      45     9     1202  f1f2 a6a5 g2h3
3      45    15     1601  f1f2 a6a5 g2h3

Until later plys (10):

Code: Select all

10     111  1998  5542512  f5g7 !!
stat01: 2088 5738496 10 0 0
stat01: 2187 6070272 10 0 0
stat01: 2289 6397952 10 0 0
stat01: 2390 6758400 10 0 0
stat01: 2489 7098368 10 0 0
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2529  7212816  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7
10     235  2550  7245303  f5g7 e8f8 g7f5 a8a7 f1f2 a7d7

Whereas engine A looks at the solution and sees it is good right away :

Code: Select all

info multipv 1 depth 1 score cp 45 time 0 nodes 1112 pv f5e3 b8b7
info multipv 1 depth 2 score cp 27 time 94 nodes 1628 pv f5e3 b8b7 e3c4
info multipv 1 depth 2 score cp 149 time 94 nodes 2000 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 3 score cp 149 time 94 nodes 2798 pv f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8
info multipv 1 depth 4 score cp 74 time 109 nodes 7733 pv f5g7 e5g7 f3f7 g8h8 f7h5 h8g8 f1f7 e8e4
info multipv 1 depth 5 score cp 163 time 203 nodes 14783 pv f5g7 e8f8 g7f5 b8b5 f5h6 g8h7 f3f5 h7g7 h6f7
info multipv 1 depth 6 score cp 187 time 203 nodes 19283 pv f5g7 e8f8 g7f5 b8b7 h4h5 e5b2 

I was thinking that perhaps adding a capture extension could solve the problem of the correct solution being skipped so early? Engine B surely
seems to not look at this move in much detail before going to another move, where engine A seems to be looking at it deeper and sees an advantage.

Any ideas? Ultimately I suppose I'd like the engine to consider captures very early, and to extend them.
I dont know how to write a capture extension. (in c, I am writing in)
Even on ply 6 engine B gives only : f5g7??
where engine A gives : f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8 - as early as ply 2.
I'd like engine B to extend this line as A does, and not give just 1 move without any search.
How can I get it to extend the line?

Please post also the details of your search. It seems to me that your doing null move in PV nodes, or there might me something broken in your search.

Nothing wrong with doing null-move in pv nodes. I do null-move _everywhere_ and have no problem with this position, finding the correct move at ply 1.

[outAtime]

The engine Im testing looks at the move at ply 2 but just gives it a bad score and moves on to another move, not revisiting it until 10 plys or so

Code: Select all

2     -50     1       77  f5g7 ?? 
2     -17     1       81  f1f2 !

I agree with you that a null move is ok in the pv, but I cant figure out why this move is looked at and trashed so quickly...at -50 cp and given a
??, Id like to see instead output which looks like this at ply 2 :

Code: Select all

2    +204           f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8

or somthing similar, this is an e.g. of course...
it seems the engine is perhaps not generating enough nodes at early ply?
caputure extension may not be the answer, especially if adding one will
only serve to weaken things overall.
This position seems simple enough to me as a human player and that is why Im also curious why an engine even low rated at 2300+ or somthing
e.g. would take 10plys. This depth might not be reached during a quick time control match...Im just trying to find where in the search I could maybe get that move to look more promising to the engine.
I dont think its here the first part of the move ordering :

Code: Select all

 /* generate the move list: */
  gen (&moves[0]);
  num_moves = numb_moves;

  /* loop through the moves at the current depth: */
  for (i = 0; i < num_moves; i++) {
    make (&moves[0], i);

    /* check to see if our move is legal: */
    if (check_legal (&moves[0], i)) {
      raw_nodes++;
      /* go deeper into the tree recursively, increasing the indent to
	 create the "tree" effect: */
      perft (depth-1);
    }

    /* unmake the move to go onto the next: */
    unmake (&moves[0], i);

The next part performs a quiscense search on the current node using alpha-beta with negamax search, so maybe its in there...
Any ideas?

[bob]

The engine Im testing looks at the move at ply 2 but just gives it a bad score and moves on to another move, not revisiting it until 10 plys or so

Code: Select all

2     -50     1       77  f5g7 ?? 
2     -17     1       81  f1f2 !

I agree with you that a null move is ok in the pv, but I cant figure out why this move is looked at and trashed so quickly...at -50 cp and given a
??, Id like to see instead output which looks like this at ply 2 :

Code: Select all

2    +204           f5g7 g8g7 f3f7 g7h8 f7h5 h8g8 h5g6 g8h8 g6h6 h8g8

or somthing similar, this is an e.g. of course...
it seems the engine is perhaps not generating enough nodes at early ply?
caputure extension may not be the answer, especially if adding one will
only serve to weaken things overall.

This is an eval issue at shallow depths. Nxg7 followed by Qxf7 is generally bad because you gave up a piece (knight) for two pawns. However, you now have an attack on white's king, two connected passed pawns that are both advanced and mobile to help with the attack, so it is easily worth it even though you don't see recovering the material with a very shallow search.

So this isn't tactical at this point, it is positional, and if your test engine does not evaluate king attacks strongly, in addition to the help provided by those two advanced/mobile passed pawns, then it isn't going to find the solution until it goes deep enough to see winning back the material you have given up.

This position seems simple enough to me as a human player and that is why Im also curious why an engine even low rated at 2300+ or somthing
e.g. would take 10plys. This depth might not be reached during a quick time control match...Im just trying to find where in the search I could maybe get that move to look more promising to the engine.
I dont think its here the first part of the move ordering :

Code: Select all

 /* generate the move list: */
  gen (&moves[0]);
  num_moves = numb_moves;

  /* loop through the moves at the current depth: */
  for (i = 0; i < num_moves; i++) {
    make (&moves[0], i);

    /* check to see if our move is legal: */
    if (check_legal (&moves[0], i)) {
      raw_nodes++;
      /* go deeper into the tree recursively, increasing the indent to
	 create the "tree" effect: */
      perft (depth-1);
    }

    /* unmake the move to go onto the next: */
    unmake (&moves[0], i);

The next part performs a quiscense search on the current node using alpha-beta with negamax search, so maybe its in there...
Any ideas?