Discocheck 5.01: Bishop related endgame problems

[Mike S.]

Two observations from a singlecore blitz tournament (only):

DeepSaros 4.1.0 - Discocheck 5.01
[d]6k1/8/1b6/P7/6KP/8/8/4B3 b - - 0 68
It was already lost for Black, but here Discocheck played 68...Bxa5 with an eval of -0.25/d47 from engine viewpoint, IOW zero minus contempt. It looks like being related to blind bishop detection, but White's bishop is not blind but winning, being on the color of the promotion corner square.

Murka 3 - Discocheck 5.01
[d]8/7P/8/5k2/8/3ppBP1/8/b2K4 b - - 0 99
I am almost sure it was a draw anyway, but at least the eval shows a problem. Discocheck played 99...e2+ with +2.46/d22 from it's viewpoint. It exepcted the upcoming g-pawn vs. bishop position, but evaluated up to +2.59 until the end (Murka foreited on time, resulting in a draw in this case).

(I seem to remember that I may have reported something similar like the 2nd example sime time ago, for an earlier Disco- oder Doublecheck version. If yes then I apologize for "more of the same" )

P.S. On the positive side, in the small tournament the examples are from, Discocheck scored wins against R232a and twice against Shredder 12 SE, amongst others.
[pgn]
[Event "A_new_Saros"]
[Site "Wien"]
[Date "2013.11.24"]
[Round "2"]
[White "Discocheck 5.01"]
[Black "Rybka 2.3.2a"]
[Result "1-0"]
[ECO "D02"]
[WhiteElo "2655"]
[BlackElo "2826"]
[PlyCount "114"]
[EventDate "2013.??.??"]
[TimeControl "180+1"]

1. Nf3 d5 2. g3 c6 3. Bg2 Bg4 4. O-O Nd7 5. d4 e6 6. Nbd2 Ngf6 7. Re1 Be7 {-0.
12/13 14} 8. e4 O-O 9. c3 dxe4 {-0.06/13 7} 10. Nxe4 {0.00/15 3} Nxe4 {-0.05/
14 6} 11. Rxe4 {-0.05/18 3} Bf5 {-0.07/15 15} 12. Re1 {0.00/17 7} h6 {-0.07/14
14} 13. Bf4 {0.07/18 8} Nb6 {-0.13/14 33} 14. Ne5 {0.12/16 6} Nd5 {-0.13/13 6}
15. Bd2 {0.10/16 6} f6 {-0.21/12 23} 16. Nf3 {0.32/17 6} g5 {-0.15/11 1} 17. h3
{0.35/16 6} Qd7 {-0.11/11 3} 18. Nh2 {0.27/15 5} Rfe8 {-0.09/11 4} 19. Rc1 {0.
27/14 5} Kg7 {0.04/10 3} 20. Qe2 {0.20/14 5} Rad8 {0.06/9 1} 21. Ng4 {0.01/13 5
} h5 {0.13/10 1} 22. Nh2 {0.22/15 5} Rh8 {0.05/11 5} 23. Nf1 {0.09/14 5} Kf7 {
0.00/12 8} 24. a3 {0.17/15 5} h4 {-0.03/13 5} 25. c4 {0.26/16 4} Nb6 {-0.05/13
3} 26. g4 {0.18/15 4} Bg6 {-0.04/12 1} 27. Ba5 {0.20/15 4} Rde8 {-0.05/11 3}
28. Rcd1 {0.32/15 4} Bd8 {-0.17/11 5} 29. b3 {0.33/16 4} Bc7 {-0.13/11 6} 30.
Bb4 {0.38/16 4} Kg7 {-0.14/10 3} 31. Ne3 {0.39/15 4} Bf4 {-0.01/10 0} 32. a4 {
0.38/14 6} Qc7 {-0.07/10 0} 33. Qb2 {0.26/14 3} Nd7 {-0.17/10 6} 34. c5 {0.49/
14 6} Bh2+ {-0.18/10 2} 35. Kh1 {0.65/15 3} Bf4 {-0.30/11 1} 36. Nc4 {0.58/14 3
} Re7 {-0.31/11 2} 37. Qe2 {0.49/14 3} Rg8 {-0.44/10 3} 38. Nd6 {0.55/13 3} a5
{-0.46/11 1} 39. Ba3 {0.74/14 3} Rd8 {-0.46/11 0} 40. d5 {0.53/14 3} Ne5 {-0.
44/11 1} 41. Nb5 {0.87/16 3} Qb8 {-0.43/13 1} 42. d6 {0.85/17 3} Red7 {-0.69/
13 4} 43. Nd4 {0.97/18 2} Re8 {-0.73/13 2} 44. Be4 {0.95/17 2} Bxe4+ {-0.81/12
2} 45. Qxe4 {1.11/16 1} Qa7 {-0.83/13 1} 46. Kg2 {1.11/15 2} Rdd8 {-0.62/10 2}
47. Kg1 {1.17/16 2} Qa6 {-0.60/11 1} 48. Bb2 {1.12/15 2} Kh6 {-0.59/10 1} 49.
Ne2 {3.44/15 2} Nd7 {-2.69/10 1} 50. Nxf4 {4.61/15 1} gxf4 {-2.90/11 1} 51.
Qxf4+ {5.94/14 1} Kg6 {-5.23/11 1} 52. Rxe6 {5.52/16 2} Rxe6 {-2.77/8 0} 53.
Qf5+ {7.53/17 2} Kf7 {-2.81/7 0} 54. Qh7+ {7.73/16 2} Kf8 {-4.92/7 0} 55. g5 {
7.73/12 3} Ne5 {-8.00/7 1} 56. gxf6 {27.91/13 3} Nf3+ {-7.64/5 1} 57. Kh1 {#7/
14 2} Qf1+ {-#11/5 1 Black resigns} 1-0
[/pgn]

[lucasart]

IOW zero minus contempt. It looks like being related to blind bishop detection, but White's bishop is not blind but winning, being on the color of the promotion corner square.

Thank you for the useful feedback. I found the bug: it's exactly what you say, the KBPK draw recognizer is buggy. One of the conditions is that the bishop and the promotion squares are of different color:

Code: Select all

color_of(bishop_sq) != color_of(prom_sq)

The problem is that the color_of() function is buggy:

Code: Select all

int color_of(int sq) { assert(square_ok(sq)); return (sq & 1) ^ 1; }

That will teach me to commit untested code

[lucasart]

Murka 3 - Discocheck 5.01
[d]8/7P/8/5k2/8/3ppBP1/8/b2K4 b - - 0 99
I am almost sure it was a draw anyway, but at least the eval shows a problem. Discocheck played 99...e2+ with +2.46/d22 from it's viewpoint. It exepcted the upcoming g-pawn vs. bishop position, but evaluated up to +2.59 until the end (Murka foreited on time, resulting in a draw in this case).

Actually this is not a bug, but rather lack of knowledge. Let's have a look:

Code: Select all

position fen 8/7P/8/5k2/8/3ppBP1/8/b2K4 b - - 0 99
go depth 4
...
info score cp 176 depth 4 nodes 883 time 0 pv e3e2 f3e2 d3e2 d1e2 f5g6 e2e3 g6h7

Now let's play that PV and see
[d]8/7k/8/8/8/6P1/8/b7 w - - 0 105
In this position DiscoCheck evaluates sees a material advantage for black (among other things). It simply lacks the knowledge to understand that it is a draw.

[hgm]

It doesn't value the King!

(OK, cheap shot... But seriously, this is very elementary, and I think it really hurts Elo if an engine does not recognize positions without mating potential for the leading side. Just recognizing KBK and KNK as draws isn't enough. End-games like KNPKN where the Pawn can be trivially blocked by the threat of an NxP sacrifice are quite common, and end-games of minor + Pawn vs minor + Pawn in which you cannot avoid trading into minor vs Pawn are not rare either. A leading side without mating potential is as good as dead.)

[lucasart]

It doesn't value the King!

Oops. Wrong FEN. I forgot the king on e3. Anyway, you got the idea.

But seriously, this is very elementary, and I think it really hurts Elo if an engine does not recognize positions without mating potential for the leading side. Just recognizing KBK and KNK as draws isn't enough. End-games like KNPKN where the Pawn can be trivially blocked by the threat of an NxP sacrifice are quite common, and end-games of minor + Pawn vs minor + Pawn in which you cannot avoid trading into minor vs Pawn are not rare either. A leading side without mating potential is as good as dead.

I will never add such knowledge to DiscoCheck, because:
* It does not improve elo measurably, contrary to your claim. Not with my testing resolution anyway.
* It adds a ton of code and complexity. Not to mention the risk of buggy transitions between the various specific endgame evaluation functions associated to various material configurations.
* The only "advantage" is of a cosmetic nature: it shows zero scores in a few more drawn positions, that would not be recognized otherwise.

I don't know why people like to have bulky endgame code in their engine. But for me it's a no brainer: if it does not improve elo it has no place in DiscoCheck.

[PK]

IMHO there is no added complexity in changing "KBK and KNK are draws" into "KBK(p)(p) and KNK(p)(p) are draws". You need just a clause "if stronger side has just a minor piece, then it is a draw" somewhere at the end of eval function.

Of course adding such knowledge has to stop somewhere (and Rodent knows much less about specific endings than Stockfish), but I think the good rule of thumb is as follows: if a (relatively) complex endgame rule is added, then You have to add all the simpler rules, else the engine will behave inconsistently.

By "simpler" I mean "positions with material removed". Thus, if You do "KBPK, edge pawn, wrong bishop", engine should know about "KBK" and "KPK, edge pawn, king in the corner". And if You do "KBPK, edge pawn, wrong bishop, additional pawn(s) for weaker side", then this configuration can be reduced to KBKP(P).

As for Elo gain or lack thereof, it should probably be measured for a whole package (complex rule + all simpler rules), not for the complex rule alone.

[lucasart]

IMHO there is no added complexity in changing "KBK and KNK are draws" into "KBK(p)(p) and KNK(p)(p) are draws". You need just a clause "if stronger side has just a minor piece, then it is a draw" somewhere at the end of eval function.

I hate adding specific knowledge. Here's the kind of code that I do not want to see in DiscoCheck:

Code: Select all

if (mat_key == KBKP || mat_key == KPKB)
  return kbkp_eval(position);
[... and a few more dozens of lines like that...]

It adds complexity:
- many eval functions for each possible config
- risk of errors made by such functions, as well as the transition problem I mentionned already.

It slows down the program
- at every single node you are wasting time to do these useless checks.
- even if the play is improved near certain endings, the net result is most likely zero or negative if only for that reason.

Basicaly to make a design decision in DiscoCheck, I ask myself the following question: What would Symbolic NOT do ?

On the other hand, I like general rules:
- when stronger side has no pawns, half the eval (unless in KXK situation where X is mating material)
- when the stronger side has no mating material, return a zero eval: I will try to add this one, and see if it is measurably positive (I doubt it but maybe).

[hgm]

I will never add such knowledge to DiscoCheck, because:
* It does not improve elo measurably, contrary to your claim. Not with my testing resolution anyway.

How can you know that, if you haven't even implemented it?

* It adds a ton of code and complexity. Not to mention the risk of buggy transitions between the various specific endgame evaluation functions associated to various material configurations.

I agree with Pawel: when properly implemented, this doesn't need a 'ton of code'. Even in Pair-o-Max, which did not even maintain a material key or a material table, it only took a routine of 22 lines to calculate the appropriate multiplier. When you have a material table, as I have in Spartacus, it becomes totally trivial. Just put a code that represents an index in a table with multipliers in the material table, for those material combinations that need to be discounted.

* The only "advantage" is of a cosmetic nature: it shows zero scores in a few more drawn positions, that would not be recognized otherwise.

I doubt that. If it is unaware that these positions are drawn, it will naively trade into them from positions that are won. Especially against opponents that know they are drawn. KBKP versus a not too advanced passer can be an appreciable lead, > 100 cP. So in KBPPKNP, with only a single Pawn ahead (which might not be a passer), it will start to seek N vs 2P trades. Which the opponent will happily offer. While the KBPPKNP would very likely be won.

I don't know why people like to have bulky endgame code in their engine. But for me it's a no brainer: if it does not improve elo it has no place in DiscoCheck.

Bulky?

Code: Select all

Fac(int k)
{/* if mating potential in jeopardy, indicate score reduction (as shift)   */
 int m,h,i,j,e,f=0,r=0,n=pl[k+1]+pl[k+2];      /* k indicates strong side  */
 if(n<2){                                      /* less than 2 pawns        */
  r=1-n;                                       /* reduce by 2 if no pawns  */
  m=pl[16-k]-n;                                /* my pieces (incl. King)   */
  if(m<4){                                     /* we have at most 2 pieces */
   h=pl[k]-pl[17-k]-pl[18-k];                  /* his pieces (incl. King)  */
   if(h<2)r=0;                                 /* bare K easy even w.o. P  */
   j=h-n;                                      /* defenders after sac for P*/
   if(j<3&&j-->0){                             /* can sac, <= 1 piece left */
    i=18-k;while(!pl[++i]);                    /* get lowest piece         */
    e=abs(w[i--])*n;                           /* sac for Pawn (if any)    */
    while(h>1)h-=pl[++i],e-=pl[i]*w[i];        /* total value his remaining*/
    if(!h)e+=3*w[i];                           /* dual King, correct w[]<0 */
    j&=pb[i]==-1;                              /* tough defenders left     */
    for(i=k+3,h=m;h>1;h-=pl[i++])e+=pl[i]*w[i],/* lead in piece material   */
     f|=pl[i]&pb[i];                           /* detect mating minors     */
    if(!f&&e<350                               /* non-mating minor ahead   */
     ||m-1&(pb[--i]>3|j)                 /* single color-bound or vs tough */
     ||pl[i]==2&pb[i]<-1)r=3-n;                /* non-mating pair          */
 }}}
 return r;
}

        re=v>>Fac(v>>15&16^k);  /* Reduce eval if drawish for leading side */

Another incentive could be that they don't want to look stupid. There are many things that make you look stupid, without lowering the Elo more than your testing can measure. As I calculated here once before, even something that happens in 1% of the games, would already take more than 40,000 games to be significant at the 95% level. And if your engine does something monumentously stupid in 1% of the games, you can be very sure people will notice and remember it. OTOH, no one cares a hoot whether your engine is 2 Elo stronger or weaker.

[PK]

in my case, this "overly complicated" code consists of simple clauses like:

Code: Select all

       

       // no win if stronger side has just one minor piece and no pawns
       if (p->pcCount[stronger][P] == 0) {
       if ( p->pieceMat[stronger] < 400 ) return 0;

       if (p->pieceMat[stronger] == Data.matValue[R]
       && p->pieceMat[weaker] > 0 ) return 16; // multiply eval score by 1/4
        }

No need for special evaluators etc, and almost no speed loss, since I call it only when pieceMat[stronger] is low enough.

[hgm]

Indeed, the run-time overhead can be kept extremely low. In Spartacus i do this through the material table, which contains 1 byte for each material combination. Eval() accesses the value for the current material key anyway. if it is <= 200, it adds materialTable[materialKey] - 100 to the (white POV) evaluation. This is the usual non-additive material correction, which is used for Bishop pair, trading gradients, etc. The only overhead is the <= 200 test.

Only when it is >200 more is done. End even that is fast:

Code: Select all

if(materialCorrection <= 240) {
  int ahead = (eval < 0); // 0 if white is ahead, 1 if black is ahead
  eval >>= multipliers[materialCorrection-200][ahead];
} else {
}

The else part is not implemented yet, but is intended to call special evaluation functions (e.g. for KQKP, or KBPK), through a table of pointers. Something like

Code: Select all

eval = (*specialEvals[materialCorrection-240])(eval)