Horrendous evaluation by Fire 1.4 Xtreme. Test Position

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Horrendous evaluation by Fire 1.4 Xtreme. Test Position

Post by mwyoung »

Trapping the clergy

[d] 4n3/R2bq1k1/3p2n1/1p1Pp2p/2p1Ppp1/2P3PP/2PN1PBB/4R1K1 b - - 0 33

Best Move 33..f3

Houdini finds the move and wins the game.

This position is from a game today played between Fire 1.4 x64P Vs. Houdini 1.5a

Fire's evaluation of the position after f3 is =, and given the above position fire will not play 33.f3.

Fire is not the only strong program that is clueless in this position.
MrEdCollins
Posts: 59
Joined: Tue May 03, 2011 12:12 am
Location: Southern California

Re: Horrendous evaluation by Fire 1.4 Xtreme. Test Position

Post by MrEdCollins »

Nice observation.

You're correct. Every other engine that I test with that position does not like 33... f3, other than Houdini.

Is it possible f3 is not as strong as Houdini thinks it is?

In an one-game, unofficial test, Houdini against itself in that position, in a 5 0 blitz game, drew as White, after Black played 33... f3.

Scratch that. I decided to play a few more games, before I posted this. In all other games, Black easily won.
MrEdCollins
Posts: 59
Joined: Tue May 03, 2011 12:12 am
Location: Southern California

Re: Horrendous evaluation by Fire 1.4 Xtreme. Test Position

Post by MrEdCollins »

Note: Crafty (23.4) also likes f3 for Black.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Horrendous evaluation by Fire 1.4 Xtreme. Test Position

Post by zullil »

MrEdCollins wrote:Note: Crafty (23.4) also likes f3 for Black.
Indeed:

Code: Select all

Crafty v23.4 (8 cpus)

White(1): setboard 4n3/R2bq1k1/3p2n1/1p1Pp2p/2p1Ppp1/2P3PP/2PN1PBB/4R1K1 b - - 0 33
Black(1): st 900
search time set to 900.00.
Black(1): go
              time limit 15:00 (+0.00) (15:00)
              depth   time  score   variation (1)
starting thread 1 2 3 4 5 6 7 <done>
               11     0.06  -0.07   1. ... Nf6 2. hxg4 hxg4 3. Rea1 f3
                                    4. Bf1 Qf7 5. R1a6 Qf8 6. Ra1 Ne7
               11->   0.07  -0.07   1. ... Nf6 2. hxg4 hxg4 3. Rea1 f3   
                                    4. Bf1 Qf7 5. R1a6 Qf8 6. Ra1 Ne7 &#40;s=2&#41;
               12     0.14  -0.12   1. ... Nf6 2. Rea1 f3 3. Bf1 Qf7 4.  
                                    R1a6 Qf8 5. Ra8 Qe7 6. hxg4 hxg4 7.
                                    R6a7
               12->   0.15  -0.12   1. ... Nf6 2. Rea1 f3 3. Bf1 Qf7 4.  
                                    R1a6 Qf8 5. Ra8 Qe7 6. hxg4 hxg4 7.
                                    R6a7
               13     0.22     +1   1. ... Nf6?                          
               13     0.30   0.49   1. ... Nf6 2. Rb1 Qd8 3. hxg4 fxg3   
                                    4. Bxg3 h4 5. Bh2 Nxg4 6. Rxb5 Nxh2
                                    7. Rbb7 Nf3+ 8. Nxf3
               13     0.37  -0.06   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    R1a6 Qe7 5. hxg4 hxg4 6. Kh1 Qf8 7.
                                    Ra1 Ne7
               13->   0.38  -0.06   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    R1a6 Qe7 5. hxg4 hxg4 6. Kh1 Qf8 7.
                                    Ra1 Ne7 &#40;s=2&#41;
               14     0.41  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    R1a6 Qe7 5. Ra1 Qf7 6. R1a6 Ne7 7.
                                    hxg4 hxg4 8. Kh1 Nc8 9. Rb7 Qh5
               14->   0.45  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    R1a6 Qe7 5. Ra1 Qf7 6. R1a6 Ne7 7.
                                    hxg4 hxg4 8. Kh1 Nc8 9. Rb7 Qh5 &#40;s=2&#41;
               15     0.62  -0.01   1. ... Qd8 2. Rea1 Nf6 3. R1a6 Qf8    
                                    4. Rb6 f3 5. Bf1 Kg8 6. Raa6 Ne8 7.
                                    Ra7 Nf6
               15->   0.72  -0.01   1. ... Qd8 2. Rea1 Nf6 3. R1a6 Qf8    
                                    4. Rb6 f3 5. Bf1 Kg8 6. Raa6 Ne8 7.
                                    Ra7 Nf6
               16     1.11  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. hxg4
                                    hxg4 8. Rc7 Kg6 9. Ra1
               16->   1.18  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. hxg4
                                    hxg4 8. Rc7 Kg6 9. Ra1
               17     1.30  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. hxg4
                                    hxg4 8. Kh1 Qh8 9. Ra6 Qh5
               17->   1.41  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. hxg4
                                    hxg4 8. Kh1 Qh8 9. Ra6 Qh5 &#40;s=2&#41;
               18     1.61  -0.07   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. hxg4
                                    hxg4 8. Kh1 Kf7 9. Rc7 Kg6 10. Ra1
               18->   1.91  -0.07   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. hxg4
                                    hxg4 8. Kh1 Kf7 9. Rc7 Kg6 10. Ra1
                                    &#40;s=2&#41;
               19     2.17  -0.07   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. Kh1
                                    Kf7 8. Kg1 Kg7 9. hxg4 hxg4 10. Kh1
                                    Kf7 11. Rc7 Kg6 12. Ra1
               19->   2.71  -0.07   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Qe8 7. Kh1
                                    Kf7 8. Kg1 Kg7 9. hxg4 hxg4 10. Kh1
                                    Kf7 11. Rc7 Kg6 12. Ra1 &#40;s=3&#41;
               20     3.26  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Kf8 7. hxg4
                                    hxg4 8. Ra1 Ke7 9. Ra6 Kf7 10. Ra8
                                    Qf8 11. Ra1 &#40;s=2&#41;
               20->   4.01  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Kf8 7. hxg4
                                    hxg4 8. Ra1 Ke7 9. Ra6 Kf7 10. Ra8
                                    Qf8 11. Ra1 &#40;s=2&#41;
               21     4.72  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Kf8 7. hxg4
                                    hxg4 8. Ra1 Ke7 9. Ra6 Kf7 10. Ra8
                                    Qg8 11. Rba7 Nxa7 12. Rxg8 Kxg8
               21->   6.21  -0.08   1. ... Qd8 2. Rea1 f3 3. Bf1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra8 Kf8 7. hxg4
                                    hxg4 8. Ra1 Ke7 9. Ra6 Kf7 10. Ra8
                                    Qg8 11. Rba7 Nxa7 12. Rxg8 Kxg8 &#40;s=2&#41;
               22    10.63  -0.08   1. ... Qd8 2. Rea1 f3 3. Bh1 Nf6 4.   
                                    Rb7 Nh8 5. Kf1 Nf7 6. Raa7 Ng5 7. h4
                                    Nf7 8. Ke1 Qc8 9. Kd1 Qe8 10. Rc7 Qd8
                                    11. Rcb7 Kg6 12. Rc7
               22    16.04  -0.20   1. ... f3 2. Bf1 Qd8 3. Rea1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra1 Qf8 7. Nb1
                                    Kg6 8. hxg4 hxg4 9. Na3 Qe8 10. Rb1
                                    Nxe4 11. Nxb5 Bxb5 12. R1xb5 Nxc3
               22->  16.04  -0.20   1. ... f3 2. Bf1 Qd8 3. Rea1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra1 Qf8 7. Nb1
                                    Kg6 8. hxg4 hxg4 9. Na3 Qe8 10. Rb1
                                    Nxe4 11. Nxb5 Bxb5 12. R1xb5 Nxc3 &#40;s=2&#41;
               23    16.87  -0.20   1. ... f3 2. Bf1 Qd8 3. Rea1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra1 Qf8 7. Nb1
                                    Kg6 8. hxg4 hxg4 9. Na3 Qe8 10. Rb1
                                    Nxe4 11. Nxb5 Bxb5 12. R1xb5 Nxc3
               23->  20.76  -0.20   1. ... f3 2. Bf1 Qd8 3. Rea1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra1 Qf8 7. Nb1
                                    Kg6 8. hxg4 hxg4 9. Na3 Qe8 10. Rb1
                                    Nxe4 11. Nxb5 Bxb5 12. R1xb5 Nxc3 &#40;s=5&#41;
               24    26.09  -0.26   1. ... f3 2. Bf1 Qd8 3. Rea1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra1 Qf8 7. Nb1
                                    Kg6 8. hxg4 hxg4 9. Ra8 Qh8 10. Rba7
                                    Qe8 11. Nd2 Nxa7 12. Rxe8 Bxe8 13.
                                    Kh1 &#40;s=4&#41;
               24->  32.35  -0.26   1. ... f3 2. Bf1 Qd8 3. Rea1 Nf6 4.   
                                    Rb7 Ne7 5. Raa7 Nc8 6. Ra1 Qf8 7. Nb1
                                    Kg6 8. hxg4 hxg4 9. Ra8 Qh8 10. Rba7
                                    Qe8 11. Nd2 Nxa7 12. Rxe8 Bxe8 13.
                                    Kh1 &#40;s=2&#41;
               25    41.40  -0.30   1. ... f3 2. Bf1 Qd8 3. Rea1 Ne7 4.   
                                    Ra8 Nc8 5. hxg4 hxg4 6. R1a7 Nf6 7.
                                    Rb7 Qh8 8. Nb1 Kg6 9. Rba7 Qe8 10.
                                    Nd2 Nxa7 11. Rxe8 Bxe8 12. Kh1 Bd7
                                    13. Kg1 Kg5
               25->  56.34  -0.30   1. ... f3 2. Bf1 Qd8 3. Rea1 Ne7 4.   
                                    Ra8 Nc8 5. hxg4 hxg4 6. R1a7 Nf6 7.
                                    Rb7 Qh8 8. Nb1 Kg6 9. Rba7 Qe8 10.
                                    Nd2 Nxa7 11. Rxe8 Bxe8 12. Kh1 Bd7
                                    13. Kg1 Kg5 &#40;s=3&#41;
               26     1&#58;33  -0.30   1. ... f3 2. Bf1 Qd8 3. Rea1 Ne7 4.   
                                    Ra8 Nc8 5. hxg4 hxg4 6. R1a7 Nf6 7.
                                    Rb7 Qh8 8. Nb1 Kg6 9. Rab8 Qe8 10.
                                    Nd2 Qd8 11. Ra8 Qf8 12. Nb1 Qh8 13.
                                    Rba7 Qe8 14. Nd2 Nxa7 15. Rxe8 Bxe8
                                    16. Kh1 Bd7 &#40;s=2&#41;
               26->   1&#58;54  -0.30   1. ... f3 2. Bf1 Qd8 3. Rea1 Ne7 4.   
                                    Ra8 Nc8 5. hxg4 hxg4 6. R1a7 Nf6 7.
                                    Rb7 Qh8 8. Nb1 Kg6 9. Rab8 Qe8 10.
                                    Nd2 Qd8 11. Ra8 Qf8 12. Nb1 Qh8 13.
                                    Rba7 Qe8 14. Nd2 Nxa7 15. Rxe8 Bxe8
                                    16. Kh1 Bd7
               27     1&#58;57  -0.30   1. ... f3 2. Bf1 Qd8 3. Rea1 Ne7 4.   
                                    Ra8 Nc8 5. hxg4 hxg4 6. R1a7 Nf6 7.
                                    Rb7 Qh8 8. Nb1 Kg6 9. Rba7 Qe8 10.
                                    Nd2 Nxa7 11. Rxe8 Bxe8 12. Kh1 Bd7
                                    13. Kg1 Kh5 14. Kh1 Kg5
               27->   2&#58;36  -0.30   1. ... f3 2. Bf1 Qd8 3. Rea1 Ne7 4.   
                                    Ra8 Nc8 5. hxg4 hxg4 6. R1a7 Nf6 7.
                                    Rb7 Qh8 8. Nb1 Kg6 9. Rba7 Qe8 10.
                                    Nd2 Nxa7 11. Rxe8 Bxe8 12. Kh1 Bd7
                                    13. Kg1 Kh5 14. Kh1 Kg5 &#40;s=9&#41;
               28     3&#58;46  -0.34   1. ... f3 2. Bh1 Nf8 3. Rb1 Qg5 4.    
                                    Rd1 Qf6 5. Kf1 Qg6 6. h4 Nf6 7. Re1
                                    Qh6 8. Rd1 Qh7 9. Re1 Kf7 10. Ra6 Qh6
                                    11. Rd1 Ke7 12. Ra8 Qg6 13. Re1 N8h7
                                    14. Ra7 Qh6 15. Rd1 Qg6 16. Re1 &#40;s=8&#41;
               28->   7&#58;54  -0.34   1. ... f3 2. Bh1 Nf8 3. Rb1 Qg5 4.    
                                    Rd1 Qf6 5. Kf1 Qg6 6. h4 Nf6 7. Re1
                                    Qh6 8. Rd1 Qh7 9. Re1 Kf7 10. Ra6 Qh6
                                    11. Rd1 Ke7 12. Ra8 Qg6 13. Re1 N8h7
                                    14. Ra7 Qh6 15. Rd1 Qg6 16. Re1 &#40;s=3&#41;
               29     8&#58;20  -0.36   1. ... f3 2. Bh1 Nf8 3. Rb1 Qg5 4.    
                                    Rd1 Kf7 5. Kf1 Qg6 6. h4 Nf6 7. Re1
                                    Qh6 8. Rd1 Ng6 9. Ra6 Ke7 10. Ra5 Nh8
                                    11. Ra7 Nf7 12. Kg1 Qg6 13. Re1 Qh7
                                    14. Re3 Kf8 15. Re1 Qh6 16. Re3 &#40;s=2&#41;
              time=15&#58;00  mat=0  n=20063090725  fh=88%  nps=22.3M         
              extensions=414.1M qchecks=890.8M reduced=2063.3M pruned=63.2M
              predicted=0  evals=3123.1M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=333488  aborts=50001  data=55/512  elap=15&#58;00
terminating SMP processes.
Black&#40;1&#41;&#58; f3
              time used&#58;  15&#58;00
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Horrendous evaluation by Fire 1.4 Xtreme. Test Position

Post by mwyoung »

MrEdCollins wrote:Nice observation.

You're correct. Every other engine that I test with that position does not like 33... f3, other than Houdini.

Is it possible f3 is not as strong as Houdini thinks it is?

In an one-game, unofficial test, Houdini against itself in that position, in a 5 0 blitz game, drew as White, after Black played 33... f3.

Scratch that. I decided to play a few more games, before I posted this. In all other games, Black easily won.
A few programs that have problems with this position:

Critter 1.01
Stockfish 2.1.1
Rybka 4.1
Spike 1.4
Deep Fritz 12

Then you have a few programs that play f3 but think white is better or = after the move f3.

As a human player I would play f3 in a heartbeat.

Here is the full game between Fire and houdini.

[Event "5m/40+5m/40+5m/40"]
[Site ""]
[Date "2011.05.28"]
[Round "2"]
[White "Fire 1.4 xTreme x64p"]
[Black "Houdini 1.5a x64"]
[Result "0-1"]
[ECO "C49"]
[WhiteElo "2500"]
[BlackElo "2500"]
[Annotator "0.13;0.16"]
[PlyCount "134"]
[TimeControl "40/300:40/300:40/300"]

1. e4 {B 0} e5{B 0} 2. Nf3 {B 0} Nc6 {B 0} 3. Bb5 {B 0} Nf6 {B 0} 4. Nc3 {B 0} Bb4 {B 0} 5.
O-O {B 0} O-O {B 0} 6. d3 {B 0} d6 {B 0} 7. Bg5 {B 0} Bxc3 {B 0} 8. bxc3 {B 0}
Qe7 {B 0} 9. Re1 {B 0} Nd8 {B 0} 10. d4 {B 0} Ne6 {B 0} 11. Bc1 {B 0} c5 {B 0
Both last book move} 12. h3 {0.13/18 10} a6 {0.16/18 29 (Rd8)} 13. Bf1 {0.25/
18 3 (Bc4)} b5 {0.13/20 8 (Rd8)} 14. a4 {0.35/21 11} Bb7 {0.11/19 7} 15. d5 {
0.31/20 3} Nc7 {0.36/20 10} 16. Bg5 {0.38/20 4 (c4)} h6 {0.13/20 8} 17. Bh4 {
0.28/19 2} Rfb8 {0.16/19 14} 18. Nd2 {0.29/18 4 (Qd3)} c4 {0.09/20 7} 19. Qb1 {
0.41/19 11 (Qf3)} g5 {0.16/19 11} 20. Bg3 {0.41/17 0} Bc8 {0.15/20 7} 21. Qb4 {
0.30/19 13 (f3)} Nh5 {0.20/18 12} 22. Bh2 {0.28/18 2} Nf4 {0.13/19 7} 23. Ra3 {
0.26/19 28 (Bxf4)} f5 {-0.01/18 8} 24. Raa1 {0.22/19 15} Kg7 {-0.01/17 10} 25.
Qa3 {0.17/18 21 (Bxf4)} Bd7 {-0.04/19 8} 26. g3 {0.05/18 17 (Bxf4)} Ng6 {-0.27/
18 12} 27. Bg2 {-0.07/18 13} f4 {-0.32/18 16} 28. Qb4 {-0.08/19 17 (Rab1)} h5 {
-0.34/18 11 (a5)} 29. Qa5 {-0.08/17 2} Ne8 {-0.48/19 21} 30. axb5 {-0.06/20 13
(Qb4)} axb5 {-0.66/18 11 (fxg3)} 31. Qxa8 {0.00/21 8 (Qb4)} Rxa8 {-0.83/17 2}
32. Rxa8 {0.00/20 0} g4 {-0.95/18 7 (fxg3)} 33. Ra7 {0.00/21 20} f3 {-0.95/16
0 (Qd8)} 34. Bh1 {0.08/23 18} Kh6 {-1.07/22 10 (Nf8)} 35. Rea1 {0.00/21 13
(Rb1)} Qd8 {-1.07/22 11} 36. h4 {0.00/21 14 (Rb7)} Nf6 {-0.92/23 16 (Nf8)} 37.
Rb7 {0.00/24 10 (R1a6)} Ne7 {-1.09/23 7} 38. Ra6 {0.00/24 7 (Raa7)} Nc8 {-1.09/
22 7} 39. Ra8 {0.00/24 2 (Ra1)} Kg6 {-1.35/21 15} 40. Ra6 {-0.08/25 17 (Nxf3)}
Qg8 {-1.26/20 6 (Kh6)} 41. Ra1 {-0.26/20 3 (Nxf3)} Nxe4 {-2.49/19 10 (Ne7)} 42.
Nxe4 {-1.36/20 11 (Nxf3)} Bf5 {-2.64/17 0} 43. Rxb5 {-2.12/21 40 (Ng5)} Bxe4 {
-2.90/16 0} 44. Ra2 {-2.38/22 18} Kf5 {-4.42/20 12 (Bxd5)} 45. Ra8 {-3.35/23 59
} Bxc2 {-4.42/18 0} 46. Rba5 {-3.75/22 33 (Rab8)} Be4 {-5.26/20 13 (Kg6)} 47.
Rb8 {-3.69/22 35 (Rb5)} Bxd5 {-5.30/19 5} 48. Ra1 {-3.86/22 22 (Rab5)} Qe6 {-5.
84/20 20} 49. Rb5 {-3.86/23 9 (Kf1)} Be4 {-5.96/19 7 (Bc6)} 50. Rb8 {-4.27/20
18} d5 {-5.96/17 0 (Bc6)} 51. Rb7 {-3.87/16 3 (Re1)} d4 {-6.96/16 1} 52. Rg7 {
-4.37/18 2} dxc3 {-8.62/15 2 (Kf6)} 53. Rg5+ {-7.25/18 12} Kf6 {-8.62/1 0} 54.
Rf1 {-6.52/16 0 (Rxh5)} c2 {-8.40/8 0 (Bg6)} 55. Rxh5 {-11.60/16 3} Bg6 {-7.89/
8 0} 56. Rg5 {-13.63/18 8 (Rh8)} Nd6 {-8.39/8 0} 57. Rc1 {-14.20/18 6} Ne4 {-8.
75/8 0} 58. Rxg4 {-13.74/16 0 (Rxc2)} Qxg4 {-12.07/8 0} 59. Rxc2 {-15.49/17 1
(Kf1)} Nxg3 {-13.27/8 0 (c3)} 60. Ra2 {-26.68/14 1 (fxg3)} Nxh1+ {-16.95/8 0}
61. Kf1 {-26.68/13 0} Bd3+ {-#7/23 2 (Qh3+)} 62. Ke1 {-#10/16 3} c3 {-#6/21 0
(Qxh4)} 63. Kd1 {-#5/15 1 (Ra6+)} Qb4 {-#5/19 0} 64. Ra6+ {-#4/15 1} Bxa6 {-#4/
17 0} 65. Bxe5+ {-#3/15 3 (Bf4)} Kxe5 {-#3/15 0} 66. Kc2 {-#2/15 2 (Kc1)} Qb2+
{-#2/13 0 (Nxf2)} 67. Kd1 {-#3/15 1} Qb1# {-#1/1 0} 0-1