Mortimer trap test bug in Stockfish 10

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
ailin
Posts: 13
Joined: Sun Sep 24, 2017 11:25 am
Location: Iran

Mortimer trap test bug in Stockfish 10

Post by ailin » Fri Jun 07, 2019 4:19 am

As you may know Mortimer trap is a famous trap in Ruy lupez openining .(google it and you’ll find it easily).
Also it’s a valid test for new chess engines!

According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently. No problem till here .

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish? Please answer everyone.

User avatar
MikeB
Posts: 4041
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Mortimer trap test bug in Stockfish 10

Post by MikeB » Fri Jun 07, 2019 5:48 am

ailin wrote:
Fri Jun 07, 2019 4:19 am
As you may know Mortimer trap is a famous trap in Ruy lupez openining .(google it and you’ll find it easily).
Also it’s a valid test for new chess engines!

According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently. No problem till here .

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish? Please answer everyone.
Game is already over with Nc4 as well At move 8 below , I changed the scoring eval to scoring % for McCain ( this is unreleased version and close to version X4) with the Lc0 scoring % evaluation output option. I'm not finding any stockfish derived engine that would play 5.Nxe5 - it's a clear loser. Not sure as to the bug you are claiming.

Image

jp
Posts: 1345
Joined: Mon Apr 23, 2018 5:54 am

Re: Mortimer trap test bug in Stockfish 10

Post by jp » Fri Jun 07, 2019 5:54 am

ailin wrote:
Fri Jun 07, 2019 4:19 am
According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently.

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish?
SF hates the position, so it thinks all 6th moves are bad. I don't know why you'd call that a bug.

5. Nxe5? is never in SF's first 5 PVs.

elpapa
Posts: 203
Joined: Sun Jan 18, 2009 10:27 pm
Location: Sweden
Full name: Patrik Karlsson

Re: Mortimer trap test bug in Stockfish 10

Post by elpapa » Fri Jun 07, 2019 3:32 pm

ailin wrote:
Fri Jun 07, 2019 4:19 am
As you may know Mortimer trap is a famous trap in Ruy lupez openining .(google it and you’ll find it easily).
Also it’s a valid test for new chess engines!

According to all chess literature and famous GMs the best continuation in move 6 for white is 6.Nc4 ?! But when you test the position by Stockfish it will tell you something different! Stockfish prefers 6.Nxf7 confidently. No problem till here .

but when you follow the suggested variation, finally it turns out that final evaluation is completely in favor of black !! How could this happen? Is it a bug in Stockfish? Please answer everyone.

It is black who sets a trap and white who falls into it, right? So why are you surprised black is better? Am I missing something here?

ailin
Posts: 13
Joined: Sun Sep 24, 2017 11:25 am
Location: Iran

Re: Mortimer trap test bug in Stockfish 10

Post by ailin » Fri Jun 07, 2019 5:05 pm

Mike B,
thanks for your great analysis. so your system prefers 6.Nc4 strictly over 6.Nxf7 ,right ?
But you didn’t give us the ply or plies calculated for selecting this move . What means +50 ??
Please clarify.


Jp and elpapa , I didn’t say Stockfish plays or prefers 5.Nxe5. My problem is on” best move” and” variation” on sixth move.
Stockfish 10 prefers 6.Nxf7 for WHITE , but when you follow the line it shows you that this line is in favor of BLACk!

About the trap itself. There’s 2 trap here. First :mortimer trap (when black plays...Ne7), and second white’s trap when he leaves white bishop en praise . )
Please notice.
Any further evaluation of move 6. For white ?

jp
Posts: 1345
Joined: Mon Apr 23, 2018 5:54 am

Re: Mortimer trap test bug in Stockfish 10

Post by jp » Fri Jun 07, 2019 5:11 pm

ailin wrote:
Fri Jun 07, 2019 5:05 pm
Jp and elpapa , I didn’t say Stockfish plays or prefers 5.Nxe5. My problem is on” best move” and” variation” on sixth move.
Stockfish 10 prefers 6.Nxf7 for WHITE , but when you follow the line it shows you that this line is in favor of BLACk!
What all three of us were trying to tell you is that all 6th moves are bad for White according to SF (which is why it won't play 5.Nxe5 and why this line is just a Black trap). You are arguing about which of a group of bad moves is "least bad". It's often the case that engines order a group of all very good or all very bad moves differently from a human looking for practical chances.

There is no sign of a bug here.

ailin
Posts: 13
Joined: Sun Sep 24, 2017 11:25 am
Location: Iran

Re: Mortimer trap test bug in Stockfish 10

Post by ailin » Fri Jun 07, 2019 5:31 pm

Jp
Coming close to the hot point.
When you continue 6.Nc4 (which is inferior line according to Stockfish) after 10 moves , Stockfish evaluates -3.54 for black.
But when you continue 6.Nxf7 (Stockfish’s best choice for white ), then 10 moves afterward, Stockfish evaluates -4.57 for black !
Is this logical ?!
I don’t think so.

elpapa
Posts: 203
Joined: Sun Jan 18, 2009 10:27 pm
Location: Sweden
Full name: Patrik Karlsson

Re: Mortimer trap test bug in Stockfish 10

Post by elpapa » Fri Jun 07, 2019 6:37 pm

ailin wrote:
Fri Jun 07, 2019 5:31 pm
Jp
Coming close to the hot point.
When you continue 6.Nc4 (which is inferior line according to Stockfish) after 10 moves , Stockfish evaluates -3.54 for black.
But when you continue 6.Nxf7 (Stockfish’s best choice for white ), then 10 moves afterward, Stockfish evaluates -4.57 for black !
Is this logical ?!
I don’t think so.
I see what you mean now, but you can't expect Stockfish to know as much about a certain position when you search it from 10 moves away as it does using that position as the root of your search. This is not a bug, it's just an observation that Stockfish doesn't play perfect chess.

ailin
Posts: 13
Joined: Sun Sep 24, 2017 11:25 am
Location: Iran

Re: Mortimer trap test bug in Stockfish 10

Post by ailin » Fri Jun 07, 2019 6:41 pm

Mike B ,
Is your mac pro a 12 cores or a 6 cores ?
Has you upgraded it ?
Does your system chooses 6.Nc4 ? Or 6. Nxf7 ? at how much plies ?
Thanks very much

jp
Posts: 1345
Joined: Mon Apr 23, 2018 5:54 am

Re: Mortimer trap test bug in Stockfish 10

Post by jp » Fri Jun 07, 2019 6:48 pm

ailin wrote:
Fri Jun 07, 2019 5:31 pm
Jp
Coming close to the hot point.
When you continue 6.Nc4 (which is inferior line according to Stockfish) after 10 moves , Stockfish evaluates -3.54 for black.
But when you continue 6.Nxf7 (Stockfish’s best choice for white ), then 10 moves afterward, Stockfish evaluates -4.57 for black !
Is this logical ?!
I don’t think so.
That's because it's gone deeper into the line. Of course, the eval. can change when it goes deeper or there would be no point searching deeper.

And what do you mean by "10 moves afterward"? How many plies was the original eval for?

Post Reply