Crafty Resign threshold

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Spock

Re: Crafty Resign threshold

Post by Spock »

Resurrecting this...

Just because Crafty "picked it up" doesn't mean it actually followed it

despite resign=0 crafty still resigns here under Arena


Game here
[Event "Crafty 23.0 x64 4CPU vs Bright 0.4a 1CPU"]
[Site "Q6600"]
[Date "2009.04.28"]
[Round "2"]
[White "Bright-0.4a"]
[Black "Crafty-23.0-win64"]
[Result "1-0"]
[BlackElo "2200"]
[ECO "A28"]
[Opening "English"]
[Time "22:22:43"]
[Variation "Four Knights, Nenarokov Variation"]
[WhiteElo "2200"]
[TimeControl "40/180:40/180:40/180"]
[Termination "normal"]
[PlyCount "181"]
[WhiteType "program"]
[BlackType "program"]

1. e4 e5 2. Nf3 Nf6 3. Nc3 Nc6 4. d4 exd4 5. Nxd4 Bb4 6. Bg5 O-O 7. Be2 h6
8. Bh4 Re8 9. Nxc6 bxc6 10. Qd4 Bxc3+ 11. bxc3 {-1.50/16 16} g5 {-1.55/17
5} 12. e5 {-1.50/14 7} c5 {-1.38/17 4} 13. Qd3 {-1.59/14 11} Rxe5 {-1.42/18
5} 14. Bg3 {-1.29/15 9} Re6 {-1.47/17 4} 15. O-O {-1.38/14 4} d5 {-1.50/16
4} 16. Qd1 {-1.11/13 7} Ne4 {-1.44/16 4} 17. Bf3 {-1.05/12 3} Nxc3
{-1.54/17 5} 18. Qd2 {-1.16/13 3} Ne4 {-1.50/18 4} 19. Bxe4 {-1.20/13 7}
Rxe4 {-1.50/17 4} 20. f4 {-1.20/13 7} c6 {-1.65/15 4} 21. fxg5 {-1.02/12 5}
hxg5 {-1.59/16 5} 22. Rae1 {-0.72/12 2} Ba6 {-1.57/15 5} 23. Rf5 {-0.73/12
3} f6 {-1.29/14 5} 24. Qc3 {-0.83/12 7} Rxe1+ {-1.00/16 5} 25. Qxe1
{-0.55/14 4} Bc8 {-0.91/16 5} 26. Rf1 {-0.44/13 6} Kf7 {-0.84/16 6} 27. h4
{-0.22/12 5} Qe7 {-0.67/16 5} 28. Be5 {-0.22/12 4} f5 {-0.01/17 5} 29. hxg5
{+0.38/12 3} Bd7 {+0.52/16 13} 30. Rf3 {+0.41/13 5} Rg8 {+0.61/16 7} 31.
Rh3 {+0.48/13 3} Qe6 {+0.77/15 4} 32. Qg3 {+0.85/12 4} f4 {+1.08/15 5} 33.
Qxf4+ {+0.95/12 5} Qf5 {+1.16/15 5} 34. Qe3 {+1.21/12 5} Qe4 {+1.12/15 5}
35. Rf3+ {+1.29/12 6} Ke8 36. Qf2 {+1.41/12 6} d4 {+1.42/15 5} 37. Bd6
{+1.44/11 3} Be6 {+1.23/16 6} 38. Bxc5 {+1.41/11 4} Rxg5 {+1.08/14 6} 39.
Bxd4 {+1.29/11 5} Rb5 {+1.07/14 10} 40. Rf4 {+1.26/10 2} Rb1+ {+0.94/13 4}
41. Kh2 {+1.50/12 2} Qh7+ {+1.35/15 3} 42. Rh4 {+1.87/13 6} Qc7+ {+1.35/15
3} 43. g3 {+1.69/13 3} Qf7 {+1.46/15 4} 44. Bf6 {+1.54/12 6} c5 {+1.46/13
3} 45. c4 {+1.81/11 4} Rb6 {+1.40/13 3} 46. Rh6 {+1.30/10 5} Bxc4 {+1.60/13
4} 47. Qxc5 {+1.30/10 4} Kd7 {+2.00/13 9} 48. g4 {+1.47/10 6} Rxf6
{+1.96/13 4} 49. Qd4+ {+1.83/11 3} Ke7 {+2.18/14 3} 50. Qxa7+ {+1.84/11 4}
Ke6 {+2.20/14 3} 51. Qe3+ {+1.89/11 2} Kd5 52. Qg5+ {+1.91/12 6} Ke6
{+2.38/16 4} 53. Qf5+ {+1.93/13 6} Ke7 54. Qc5+ {+1.93/13 6} Kd7 {+2.48/15
4} 55. Rxf6 {+1.98/13 6} Qxf6 56. Qxc4 {+1.97/13 6} Qf2+ {+2.66/14 3} 57.
Kh3 {+2.08/13 3} Qe3+ {+2.68/14 4} 58. Kg2 {+2.10/13 3} Qd2+ {+2.85/14 7}
59. Kf3 {+2.11/14 6} Qd1+ {+2.85/13 3} 60. Kf4 {+2.21/13 4} Qd6+ {+2.81/13
4} 61. Kf5 {+2.22/13 3} Qf8+ {+2.84/13 3} 62. Ke4 {+2.22/13 5} Qe7+
{+2.98/14 3} 63. Kd3 {+2.32/12 2} Qa3+ {+3.14/14 4} 64. Kd4 {+2.39/12 2}
Qb2+ 65. Ke4 {+2.63/13 4} Qg2+ {+3.28/13 3} 66. Kf5 {+2.64/13 4} Qf2+
{+3.42/14 8} 67. Qf4 {+2.63/12 5} Qb6 {+3.46/14 3} 68. Qe4 {+2.77/12 4}
Qa5+ {+3.70/13 4} 69. Kg6 {+2.95/13 5} Qa3 {+3.82/13 4} 70. Kf7 {+2.99/12
3} Qg3 {+4.43/14 6} 71. Qd5+ {+3.13/12 2} Kc7 {+4.73/14 4} 72. g5 {+3.37/13
2} Qf4+ {+5.31/14 7} 73. Kg7 {+3.97/13 3} Kb6 {+5.31/14 3} 74. g6 {+9.30/13
4} Qh2 {+6.34/14 7} 75. Kf6 {+9.35/12 3} Qh6 {+12.39/14 11} 76. Qe5
{+11.69/12 4} Qf8+ {+12.81/14 6} 77. Kg5 {+12.34/14 5} Ka7 {+12.81/13 1}
78. g7 {+21.01/13 4} Qd8+ {+12.85/13 2} 79. Kg6 {+21.18/12 2} Qd3+ 80. Kf7
{+22.04/11 1} Qf3+ {+13.57/13 1} 81. Qf6 {+58.77/11 6} Qd5+ {+M17/12 3} 82.
Qe6 {+58.73/11 5} Qf3+ 83. Ke7 {+58.98/12 6} Qa3+ {+M10/12 4} 84. Ke8
{+59.02/12 6} Qg3 {+M9/13 3} 85. g8=Q {+59.20/10 2} Qb8+ {+M8/12 4} 86. Ke7
{+M8/14 2} Qb4+ 87. Qd6 {+M7/12 0} Qe4+ {+M6/11 3} 88. Kd7 {+M6/10 0} Qf5+
89. Kd8 {+M5/9 0} Qf3 {+M4/11 3} 90. Qc5+ {+M4/6 0} Kb7 {+M3/11 3} 91. Qc7+
{+M3/4 0} ... {Black resigns} 1-0
And the log, notice the "resignations disabled" bit at the beginning, yet it resigned, admittedly very late though, note "black resigns" at the end:

Unless I'm missing something extremely obvious and am being stupid, Crafty did resign despite receiving and acknowledging the command not to. And hopefully it is the right log file...
Last edited by Spock on Wed Apr 29, 2009 12:19 am, edited 1 time in total.
Spock

Re: Crafty Resign threshold

Post by Spock »

I can't post the entire log, but the beginning says:
book file disabled.
show book statistics
SMP terminate extra threads when idle.
disabled resignations.
Crafty v23.0 (4 cpus)
And the end


SMP-> splits=1427 aborts=0 data=23/512 elap=3.70
terminating SMP processes.

mated in 2 moves.

resign
1-0 {Black resigns}
Black(91): Ka8
time used: 3.70

+---+---+---+---+---+---+---+---+
8 |<K>| . | |-K-| | . |-Q-| . |
+---+---+---+---+---+---+---+---+
7 | . | |-Q-| | . | | . | |
+---+---+---+---+---+---+---+---+
6 | | . | | . | | . | | . |
+---+---+---+---+---+---+---+---+
5 | . | | . | | . | | . | |
+---+---+---+---+---+---+---+---+
4 | | . | | . | | . | | . |
+---+---+---+---+---+---+---+---+
3 | . | | . | | . |<Q>| . | |
+---+---+---+---+---+---+---+---+
2 |-P-| . | | . | | . | | . |
+---+---+---+---+---+---+---+---+
1 | . | | . | | . | | . | |
+---+---+---+---+---+---+---+---+
a b c d e f g h

White(92): ?
White(92): result 1-0
White(92): ?
White(92): force
White(92): quit
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Crafty Resign threshold

Post by bob »

Spock wrote:Resurrecting this...

Just because Crafty "picked it up" doesn't mean it actually followed it

despite resign=0 crafty still resigns here under Arena


Game here
[Event "Crafty 23.0 x64 4CPU vs Bright 0.4a 1CPU"]
[Site "Q6600"]
[Date "2009.04.28"]
[Round "2"]
[White "Bright-0.4a"]
[Black "Crafty-23.0-win64"]
[Result "1-0"]
[BlackElo "2200"]
[ECO "A28"]
[Opening "English"]
[Time "22:22:43"]
[Variation "Four Knights, Nenarokov Variation"]
[WhiteElo "2200"]
[TimeControl "40/180:40/180:40/180"]
[Termination "normal"]
[PlyCount "181"]
[WhiteType "program"]
[BlackType "program"]

1. e4 e5 2. Nf3 Nf6 3. Nc3 Nc6 4. d4 exd4 5. Nxd4 Bb4 6. Bg5 O-O 7. Be2 h6
8. Bh4 Re8 9. Nxc6 bxc6 10. Qd4 Bxc3+ 11. bxc3 {-1.50/16 16} g5 {-1.55/17
5} 12. e5 {-1.50/14 7} c5 {-1.38/17 4} 13. Qd3 {-1.59/14 11} Rxe5 {-1.42/18
5} 14. Bg3 {-1.29/15 9} Re6 {-1.47/17 4} 15. O-O {-1.38/14 4} d5 {-1.50/16
4} 16. Qd1 {-1.11/13 7} Ne4 {-1.44/16 4} 17. Bf3 {-1.05/12 3} Nxc3
{-1.54/17 5} 18. Qd2 {-1.16/13 3} Ne4 {-1.50/18 4} 19. Bxe4 {-1.20/13 7}
Rxe4 {-1.50/17 4} 20. f4 {-1.20/13 7} c6 {-1.65/15 4} 21. fxg5 {-1.02/12 5}
hxg5 {-1.59/16 5} 22. Rae1 {-0.72/12 2} Ba6 {-1.57/15 5} 23. Rf5 {-0.73/12
3} f6 {-1.29/14 5} 24. Qc3 {-0.83/12 7} Rxe1+ {-1.00/16 5} 25. Qxe1
{-0.55/14 4} Bc8 {-0.91/16 5} 26. Rf1 {-0.44/13 6} Kf7 {-0.84/16 6} 27. h4
{-0.22/12 5} Qe7 {-0.67/16 5} 28. Be5 {-0.22/12 4} f5 {-0.01/17 5} 29. hxg5
{+0.38/12 3} Bd7 {+0.52/16 13} 30. Rf3 {+0.41/13 5} Rg8 {+0.61/16 7} 31.
Rh3 {+0.48/13 3} Qe6 {+0.77/15 4} 32. Qg3 {+0.85/12 4} f4 {+1.08/15 5} 33.
Qxf4+ {+0.95/12 5} Qf5 {+1.16/15 5} 34. Qe3 {+1.21/12 5} Qe4 {+1.12/15 5}
35. Rf3+ {+1.29/12 6} Ke8 36. Qf2 {+1.41/12 6} d4 {+1.42/15 5} 37. Bd6
{+1.44/11 3} Be6 {+1.23/16 6} 38. Bxc5 {+1.41/11 4} Rxg5 {+1.08/14 6} 39.
Bxd4 {+1.29/11 5} Rb5 {+1.07/14 10} 40. Rf4 {+1.26/10 2} Rb1+ {+0.94/13 4}
41. Kh2 {+1.50/12 2} Qh7+ {+1.35/15 3} 42. Rh4 {+1.87/13 6} Qc7+ {+1.35/15
3} 43. g3 {+1.69/13 3} Qf7 {+1.46/15 4} 44. Bf6 {+1.54/12 6} c5 {+1.46/13
3} 45. c4 {+1.81/11 4} Rb6 {+1.40/13 3} 46. Rh6 {+1.30/10 5} Bxc4 {+1.60/13
4} 47. Qxc5 {+1.30/10 4} Kd7 {+2.00/13 9} 48. g4 {+1.47/10 6} Rxf6
{+1.96/13 4} 49. Qd4+ {+1.83/11 3} Ke7 {+2.18/14 3} 50. Qxa7+ {+1.84/11 4}
Ke6 {+2.20/14 3} 51. Qe3+ {+1.89/11 2} Kd5 52. Qg5+ {+1.91/12 6} Ke6
{+2.38/16 4} 53. Qf5+ {+1.93/13 6} Ke7 54. Qc5+ {+1.93/13 6} Kd7 {+2.48/15
4} 55. Rxf6 {+1.98/13 6} Qxf6 56. Qxc4 {+1.97/13 6} Qf2+ {+2.66/14 3} 57.
Kh3 {+2.08/13 3} Qe3+ {+2.68/14 4} 58. Kg2 {+2.10/13 3} Qd2+ {+2.85/14 7}
59. Kf3 {+2.11/14 6} Qd1+ {+2.85/13 3} 60. Kf4 {+2.21/13 4} Qd6+ {+2.81/13
4} 61. Kf5 {+2.22/13 3} Qf8+ {+2.84/13 3} 62. Ke4 {+2.22/13 5} Qe7+
{+2.98/14 3} 63. Kd3 {+2.32/12 2} Qa3+ {+3.14/14 4} 64. Kd4 {+2.39/12 2}
Qb2+ 65. Ke4 {+2.63/13 4} Qg2+ {+3.28/13 3} 66. Kf5 {+2.64/13 4} Qf2+
{+3.42/14 8} 67. Qf4 {+2.63/12 5} Qb6 {+3.46/14 3} 68. Qe4 {+2.77/12 4}
Qa5+ {+3.70/13 4} 69. Kg6 {+2.95/13 5} Qa3 {+3.82/13 4} 70. Kf7 {+2.99/12
3} Qg3 {+4.43/14 6} 71. Qd5+ {+3.13/12 2} Kc7 {+4.73/14 4} 72. g5 {+3.37/13
2} Qf4+ {+5.31/14 7} 73. Kg7 {+3.97/13 3} Kb6 {+5.31/14 3} 74. g6 {+9.30/13
4} Qh2 {+6.34/14 7} 75. Kf6 {+9.35/12 3} Qh6 {+12.39/14 11} 76. Qe5
{+11.69/12 4} Qf8+ {+12.81/14 6} 77. Kg5 {+12.34/14 5} Ka7 {+12.81/13 1}
78. g7 {+21.01/13 4} Qd8+ {+12.85/13 2} 79. Kg6 {+21.18/12 2} Qd3+ 80. Kf7
{+22.04/11 1} Qf3+ {+13.57/13 1} 81. Qf6 {+58.77/11 6} Qd5+ {+M17/12 3} 82.
Qe6 {+58.73/11 5} Qf3+ 83. Ke7 {+58.98/12 6} Qa3+ {+M10/12 4} 84. Ke8
{+59.02/12 6} Qg3 {+M9/13 3} 85. g8=Q {+59.20/10 2} Qb8+ {+M8/12 4} 86. Ke7
{+M8/14 2} Qb4+ 87. Qd6 {+M7/12 0} Qe4+ {+M6/11 3} 88. Kd7 {+M6/10 0} Qf5+
89. Kd8 {+M5/9 0} Qf3 {+M4/11 3} 90. Qc5+ {+M4/6 0} Kb7 {+M3/11 3} 91. Qc7+
{+M3/4 0} ... {Black resigns} 1-0
And the log, notice the "resignations disabled" bit at the beginning, yet it resigned, admittedly very late though, note "black resigns" at the end:

Unless I'm missing something extremely obvious and am being stupid, Crafty did resign despite receiving and acknowledging the command not to. And hopefully it is the right log file...
That is a different animal. Crafty has a normal "resignation" facility that looks at the search scores over time. But in this case, once an opponent finds a fourced mate, and the mate distance is steadily reduced, it will resign a couple of moves before getting mated, to be more human-like. Whether this is a good idea is debatable, but this has been in the code for 15 years now, at the request of humans that played it.

I don't see a problem resigning 2-3 moves before checkmate is played, it won't resign after the normal 5 moves in a row with the score < -9.0, which is the default action without resign=0.
Spock

Re: Crafty Resign threshold

Post by Spock »

bob wrote: I don't see a problem resigning 2-3 moves before checkmate is played, it won't resign after the normal 5 moves in a row with the score < -9.0, which is the default action without resign=0.
Agreed, it isn't a problem in the slightest. It just differs from what you said earlier

"it will never resign and will play the game out to the bitter end"

so I thought I would point it out in case it was a bug and you weren't aware of it.
User avatar
Matthias Gemuh
Posts: 3245
Joined: Thu Mar 09, 2006 9:10 am

Re: Crafty Resign threshold

Post by Matthias Gemuh »

bob wrote: Crafty ... it will resign a couple of moves before getting mated, ...

Some call that the Capablanca feature (the GM was allegedly never checkmated).

Matthias.
My engine was quite strong till I added knowledge to it.
http://www.chess.hylogic.de
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Crafty Resign threshold

Post by bob »

Spock wrote:
bob wrote: I don't see a problem resigning 2-3 moves before checkmate is played, it won't resign after the normal 5 moves in a row with the score < -9.0, which is the default action without resign=0.
Agreed, it isn't a problem in the slightest. It just differs from what you said earlier

"it will never resign and will play the game out to the bitter end"

so I thought I would point it out in case it was a bug and you weren't aware of it.
I had not considered that, to be honest. Resigning in Crafty is primarily related to scores returned from the search, the resign threshold, and the number of searches required with a score <= resign threshold before Crafty will resign. Some resign way too soon. Thanks to EGTBs I added the extra resign feature as I have seen Crafty see a mate against itself, such as in a KQKR or KRBKR or whatever, but the opponent didn't know how to play the ending, and didn't have EGTBs. So Crafty notices that the mate score is not steadily dropping, and will not resign. But if the mate score drops steadily, down to a mate in 2 I believe, it will resign at that point as it figures any program or human will see that.

In the previous discussion, I was not considering resigning when getting mated to be the issue being discussed. You have _definitely_ lost by the time you get to a mate-in-2, where being a queen down is not a certain loss. I've seen several over the years where a +9 score turned into a draw. And in rare cases even worse, a loss.
Spock

Re: Crafty Resign threshold

Post by Spock »

bob wrote:
In the previous discussion, I was not considering resigning when getting mated to be the issue being discussed. You have _definitely_ lost by the time you get to a mate-in-2
Yes, I'm absolutely fine with this, I just needed clarification that it wasn't me doing something wrong