Mortimer trap test bug in Stockfish 10

[ailin]

As you may know Mortimer trap is a famous trap in Ruy lupez openining .(google it and you’ll find it easily).
Also it’s a valid test for new chess engines!

According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently. No problem till here .

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish? Please answer everyone.

[MikeB]

As you may know Mortimer trap is a famous trap in Ruy lupez openining .(google it and you’ll find it easily).
Also it’s a valid test for new chess engines!

According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently. No problem till here .

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish? Please answer everyone.

Game is already over with Nc4 as well At move 8 below , I changed the scoring eval to scoring % for McCain ( this is unreleased version and close to version X4) with the Lc0 scoring % evaluation output option. I'm not finding any stockfish derived engine that would play 5.Nxe5 - it's a clear loser. Not sure as to the bug you are claiming.

[pgn][Event "Mac Pro x5690 3.46 Ghz "] [Site "Mac-Pro.local"] [Date "2019.06.07"] [Round "-"] [White "asmFishX_2019-06-07_popc"] [Black "McCain 060419"] [Result "0-1"] [TimeControl "60+1"] [Annotator "7. -1.84 6... +1.45"] 1. e4 e5 2. Nf3 Nc6 3. Bb5 Nf6 4. d3 Ne7 5. Nxe5 c6 6. Nc4 Ng6 {+1.45/24 +53} 7. e5 {-1.84/24 +56} Nd5 {+1.53/24 +53} 8. Nd6+ {-1.99/25 +52} Bxd6 {+69.37/23 +53} 9. exd6 {-2.19/25 +52} cxb5 {+68.95/22 +52} 10. Qf3 {-2.30/27 +45} Nf6 {+69.79/21 +52} 11. Qe3+ {-2.65/25 +35} Kf8 {+69.65/24 +52} 12. a4 {-2.51/24 +31} bxa4 {+71.60/22 +49} 13. Rxa4 {-2.46/23 +26} a6 {+71.06/22 +49} 14. Nc3 {-2.87/24 +25} b6 {+71.23/22 +47} 15. O-O {-2.74/21 +23} Bb7 {+72.95/22 +47} 16. Ra1 {-2.75/22 +22} h5 {+73.79/22 +44} 17. Na4 {-2.47/21 +22} b5 {+73.24/22 +42} 18. Qb6 {-2.69/24 +18} Qc8 {+75.77/20 +42} 19. Nc5 {-3.04/21 +17} Bc6 {+77.94/22 +42} 20. Rxa6 {-3.75/23 +10} h4 {+78.75/22 +41} 21. h3 {-3.49/23 +9} Rh5 {+78.83/24 +39} 22. Rxa8 {-4.02/24 +6} Qxa8 {+79.85/23 +39} 23. f3 {-3.83/21 +6} Nd5 {+80.01/23 +38} 24. Qa6 {-4.13/21 +6} Qd8 {+81.40/23 +36} 25. Bd2 {-3.98/23 +3} Ndf4 {+81.89/21 +35} 26. Ne4 {-4.21/21 +3} Re5 {+83.05/22 +35} 27. b4 {-4.48/23 +1} Kg8 {+83.24/22 +34} 28. Qa5 {-4.44/23 +1} Qf8 {+83.89/25 +33} 29. Qb6 {-4.54/23 +1} f5 {+84.70/26 +33} 30. Nc3 {-4.62/24 +1} Qxd6 {+84.87/25 +32} 31. Qf2 {-4.57/25 +1} Qxb4 {+84.81/25 +31} 32. Nd5 {-4.80/26 +1} Qa4 {+85.51/30 +31} 33. Nxf4 {-4.74/25 +1} Nxf4 {+85.51/30 +30} 34. Re1 {-4.77/25 +1} Re6 {+85.92/24 +29} 35. Rxe6 {-4.91/25 +1} Nxe6 {+87.20/24 +29} 36. Be1 {-4.99/25 +1} g5 {+87.20/27 +29} 37. Qd2 {-5.11/23 +1} Kf7 {+86.96/25 +28} 38. Kh2 {-5.15/24 +1} Qd4 {+88.81/27 +26} 39. Bf2 {-5.32/24 +1} Qb2 {+88.81/31 +26} 40. Kh1 {-5.36/22 +1} g4 {+94.25/27 +21} 41. hxg4 {-7.21/21 +1} fxg4 {+95.58/21 +21} 42. fxg4 {-8.45/20 +1} h3 {+96.65/21 +21} 43. Kh2 {-9.83/22 +1} hxg2 {+98.16/24 +18} 44. d4 {-11.14/21 +1} b4 {+98.66/23 +18} 45. Qd3 {-9.04/21 +1} Qc1 {+99.07/23 +18} 46. Qf5+ {-9.92/21 +1} Ke8 {+99.40/24 +17} 47. Bg1 {-10.97/21 +1} Qd2 {+99.65/22 +16} 48. Qe5 {-12.03/22 +1} Qxc2 {+99.89/21 +14} 49. d5 {-7.38/20 +1} Bxd5 {+99.93/22 +14} 50. Qh8+ {-15.02/22 +1} Ke7 {+99.99/23 +11} 51. Qh4+ {-16.52/22 +1} Kd6 {+99.94/22 +8} 52. Qg3+ {-9.56/20 +1} Kc6 {+100.00/22 +7} 53. Qb8 {-18.37/23 +1} Qh7+ {+100.00/22 +7} 54. Kg3 {-6.21/1 +2} Qd3+ {+100.00/25 +7} 55. Kh4 {-20.05/19 +1} Qb5 {+100.00/27 +5} 56. Qa8+ {-22.24/17 +1} Kd6 {+100.00/25 +5} 57. Qe8 {-30.18/21 +1} b3 {+100.00/27 +3} 58. Bh2+ {-24.86/17 +1} Kc5 {+100.00/23 +3} 59. Bg1+ {-32.78/19 +1} Kc4 {+1000.17/29 +2} 60. Qg6 {-153.87/19 +1} Qb4 {+1000.14/33 +2} 61. Qb1 {-1000.12/36 +1} Qc3 {+1000.13/29 +1} 62. g5 {-1000.10/36 +1} Qc2 {+1000.09/30 +1} 63. Qe1 {-1000.08/33 +2} b2 {+1000.08/31 +1} 64. Be3 {-1000.07/39 +2} b1=Q {+1000.07/35 +2} 65. Qxb1 {-1000.06/49 +3} Qxb1 {+1000.06/48 +2} 66. Kg4 {-1000.05/61 +3} Qe4+ {+1000.05/76 +2} 67. Kh5 {-1000.04/127 +4} Qxe3 {+1000.04/245 +2} 68. Kg6 {-1000.03/127 +5} Qxg5+ {+1000.03/245 +3} 69. Kf7 {-1000.02/127 +6} Nc7+ {+1000.02/17 +4} 70. Kf8 {-1000.01/1 +7} Qf6# {+1000.01/245 +5} {Black mates} 0-1[/pgn]

[jp]

According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently.

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish?

SF hates the position, so it thinks all 6th moves are bad. I don't know why you'd call that a bug.

5. Nxe5? is never in SF's first 5 PVs.

[elpapa]

As you may know Mortimer trap is a famous trap in Ruy lupez openining .(google it and you’ll find it easily).
Also it’s a valid test for new chess engines!

According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently. No problem till here .

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish? Please answer everyone.

It is black who sets a trap and white who falls into it, right? So why are you surprised black is better? Am I missing something here?

[ailin]

Mike B,
thanks for your great analysis. so your system prefers 6.Nc4 strictly over 6.Nxf7 ,right ?
But you didn’t give us the ply or plies calculated for selecting this move . What means +50 ??
Please clarify.


Jp and elpapa , I didn’t say Stockfish plays or prefers 5.Nxe5. My problem is on” best move” and” variation” on sixth move.
Stockfish 10 prefers 6.Nxf7 for WHITE , but when you follow the line it shows you that this line is in favor of BLACk!

About the trap itself. There’s 2 trap here. First :mortimer trap (when black plays...Ne7), and second white’s trap when he leaves white bishop en praise . )
Please notice.
Any further evaluation of move 6. For white ?

[jp]

Jp and elpapa , I didn’t say Stockfish plays or prefers 5.Nxe5. My problem is on” best move” and” variation” on sixth move.
Stockfish 10 prefers 6.Nxf7 for WHITE , but when you follow the line it shows you that this line is in favor of BLACk!

What all three of us were trying to tell you is that all 6th moves are bad for White according to SF (which is why it won't play 5.Nxe5 and why this line is just a Black trap). You are arguing about which of a group of bad moves is "least bad". It's often the case that engines order a group of all very good or all very bad moves differently from a human looking for practical chances.

There is no sign of a bug here.

[ailin]

Jp
Coming close to the hot point.
When you continue 6.Nc4 (which is inferior line according to Stockfish) after 10 moves , Stockfish evaluates -3.54 for black.
But when you continue 6.Nxf7 (Stockfish’s best choice for white ), then 10 moves afterward, Stockfish evaluates -4.57 for black !
Is this logical ?!
I don’t think so.

[elpapa]

Jp
Coming close to the hot point.
When you continue 6.Nc4 (which is inferior line according to Stockfish) after 10 moves , Stockfish evaluates -3.54 for black.
But when you continue 6.Nxf7 (Stockfish’s best choice for white ), then 10 moves afterward, Stockfish evaluates -4.57 for black !
Is this logical ?!
I don’t think so.

I see what you mean now, but you can't expect Stockfish to know as much about a certain position when you search it from 10 moves away as it does using that position as the root of your search. This is not a bug, it's just an observation that Stockfish doesn't play perfect chess.

[ailin]

Mike B ,
Is your mac pro a 12 cores or a 6 cores ?
Has you upgraded it ?
Does your system chooses 6.Nc4 ? Or 6. Nxf7 ? at how much plies ?
Thanks very much

[jp]

Jp
Coming close to the hot point.
When you continue 6.Nc4 (which is inferior line according to Stockfish) after 10 moves , Stockfish evaluates -3.54 for black.
But when you continue 6.Nxf7 (Stockfish’s best choice for white ), then 10 moves afterward, Stockfish evaluates -4.57 for black !
Is this logical ?!
I don’t think so.

That's because it's gone deeper into the line. Of course, the eval. can change when it goes deeper or there would be no point searching deeper.

And what do you mean by "10 moves afterward"? How many plies was the original eval for?