Leela outplays SF Dev in a position that SF evaluates as draw

[Damir]

Not only that , this SF derivative (with Lc0 scoring) , at the critical moment, readily sacs the pawn to get in the position and it sees the Ne4 Bxe4 dxe4 Rxe4 f5 maneuver almost instantly:

[d]r2q1rk1/pp3p2/1bn2n1p/3p1Bpb/8/1NP2NBP/PP3PP1/R2QR1K1 b - - 1 16

Code: Select all

dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 44	+51.15 	5.52G	3:59.17	Ne4 Bxe4 dxe4 Rxe4 f5 Re2 Qxd1+ Rxd1 Rfe8 Rxe8+ Rxe8 Bd6 Re2 Rd2 Bxf2+ Kf1 Rxd2 Nfxd2 Bb6 Nc5 Bxc5 Bxc5 b6 Bd6 f4 Kf2 Bg6 Nf3 Kf7 g3 fxg3+ Kxg3 Be4 Ne5+ Nxe5 Bxe5 b5 a3 Ke6 Bd4 a5 h4 gxh4+ Kxh4 Kf5 Bg1 Bd5 Kh5 Ke4 Kxh6 Kd3 Kg5 

"this SF derivative"

Good plan. If you give it a name, someone might be triggered.

Have you abandoned the name "Burnzie" for some reason? Who could object to that one?

haha - no , but I haven't released it yet - but I will at some point ...

How about Lightning Volt for a name ?

[Ovyron]

Have you abandoned the name "Burnzie" for some reason? Who could object to that one?

haha - no , but I haven't released it yet - but I will at some point ...

How about Lightning Volt for a name ?

Are we suggesting names now? What about:

Mega Drain

?

It'd follow the pattern of previous names (*Brain -> *Cain -> Drain) while keeping the M from Mc.

Drain stands for "deprive of strength or vitality", and Mega for doing it in an extreme way, which is what MG would do to her opponents...

[jp]

SF (W) & Leela (B) play to this position:

[d]3r4/pp4k1/1bn1Rr1p/6p1/5p1P/1NP2P2/PP1R1P1B/6K1 w - - 1 11

SF thinks this is not Black's best line.

Can someone tell me what Leela's evaluation in the diagram position is?

1.Re4 Rxd2 2.Nxd2 Rd6 3.Nc4 Rd1+ 4.Kg2 Bc7 5.Re8 Rd7 6.a4 Kf6 7.Rh8 Kg6 8.Rg8+ Kf5 9.Rf8+ Kg6 10.Rg8+ Kf6 11.Rh8 Kg7 12.Re8 Ne7 13.Kf1 Kf7 14.Rh8 Kg7 15.Re8 a6 16.Ke2 Ng6 17.hxg5 hxg5 18.Bg1 Kf7 19.Re4 Kf6 20.Nd2 Kf5 21.Re8 Bd6 22.Nb3
-1.17
depth: 28/73
214MN
tb=747
LC0 42783

Thanks, Hai. In the diagram position, SF's analysis, depth=44, is

-0.12 11.Ree2 Rfd6 12.Rxd6 Rxd6 13.Nd2 Rd3 14.Nc4 Kf6 15.hxg5+ hxg5 16.Rd2 Rxd2 17.Nxd2 Bc7 18.Kf1 Ne5 19.Ke2 b5 20.Bg1 Kf5 21.Bh2 Bd8 22.Nb3 Bb6 23.Nd2 Bc7 24.Bg1 Nc6 25.Kd3 Bb6 26.Ke2 Bd8 27.Nb3 Be7 28.Kd3 Ke5 29.Nd2 Kd5 30.b3 Ne5+ 31.Ke2 Bd8 32.Ne4 a5 33.Nd2 Be7 34.Bh2 b4 35.cxb4 axb4 36.Ne4 Kd4 37.Nd2
-0.19 11.Re4 Rxd2 12.Nxd2 Rd6 13.Re2 Rd3 14.Nc4 Kf6 15.hxg5+ hxg5 16.Rd2 Rxd2 17.Nxd2 Bc7 18.Kf1 Ne5 19.Ke2 b5 20.Bg1 Kf5 21.Bh2 Bd8 22.Nb3 Bb6 23.Nd2 Ke6 24.Bg1 Bc5 25.Nb3 Kd5 26.Nd2 Be7 27.b3 Bd8 28.Bh2 Ke6 29.Bg1 Kf5 30.Bh2 Nc6 31.Kd3 Bf6 32.b4 Ne5+ 33.Ke2 Be7 34.Bg1 Bd8 35.Bh2

But we cannot say that Leela & SF have a genuine difference in evaluating positions here.

In Leela's PV after Black's a6 ---

[d]4R3/1pbrn1k1/p6p/6p1/P1N2p1P/2P2P2/1P3P1B/5K2 w - - 0 26

--- SF's eval (depth=44) is 0.00:

0.00 26.hxg5 hxg5 27.Ke2 Nf5 28.Nd2 Kf7 29.Rc8 Nh4 30.Rh8 Kg7 31.Re8 Kf7
-0.13 26.Ra8 g4 27.Ra7 Bb8 28.Ra8 gxf3 29.Ke1 Nc6 30.Nd2 Kg6 31.Nxf3 Kh5 32.b4 Kg4 33.Nd2 Be5 34.Ke2 Bxc3 35.f3+ Kh3 36.Bxf4 Nd4+ 37.Kf2 Bxb4 38.Re8 Be7 39.Bxh6 Bxh4+ 40.Kf1 Rc7 41.Ne4 Rc2 42.Be3 Nxf3 43.Nf2+ Bxf2 44.Bxf2 Kg4 45.Re4+ Kf5 46.Rb4 Nd2+ 47.Ke1 Nc4 48.Rxb7 Ke4 49.Bh4 Rh2 50.Re7+ Kd3 51.Bf6 Nd2 52.Rd7+ Kc2 53.Rb7 a5 54.Re7 Kb3

But SF thinks Leela's next PV White move, Ke2, is bad (depth=45):

-1.10 26...gxh4 27.Nd2 Nf5 28.Re4 h3 29.Rc4 Be5 30.Ne4 Kg6 31.Rb4 Re7 32.Rb6+ Kh5 33.Kf1 Nh4 34.Ke2 Bc7 35.Rb4 Bb8 36.b3 Be5 37.Rb6 Bc7 38.Rb4 Bb8 39.Rd4 Nf5 40.Rd5 Kg6 41.Kf1 Nh4 42.Rd3 Bc7 43.b4 b6 44.a5 bxa5 45.bxa5 Nxf3 46.Rxf3 Rxe4 47.Rxh3 h5 48.Rf3 Rc4 49.Kg2 Kf6 50.Rh3
...

So they agree about that line ( -1.17 vs -1.10), but SF thinks White should play hxg5 instead (0.00).

And we already saw (in the post http://talkchess.com/forum3/viewtopic.p ... 27#p805327 ) that in the OP position

[d]r4rk1/pp6/1bn1R2p/6pb/5p2/1NP2N1P/PP3PPB/3R2K1 b - - 0 7

SF & Leela both gave Black an advantage of 0.5-1.0, and there isn't evidence that it's not a draw.


It looks like the claim some make in this thread that SF is evaluating the position differently from Leela, and SF is the engine misevaluating, is incorrect. From what I've seen in general, if you give SF and Leela time, they mostly agree, and when they look like they disagree, it's just because somewhere down Leela's PV it forks to a line that SF thinks is not the PV.

[Nay Lin Tun]

SF (W) & Leela (B) play to this position:

[d]3r4/pp4k1/1bn1Rr1p/6p1/5p1P/1NP2P2/PP1R1P1B/6K1 w - - 1 11

SF thinks this is not Black's best line.

Can someone tell me what Leela's evaluation in the diagram position is?

1.Re4 Rxd2 2.Nxd2 Rd6 3.Nc4 Rd1+ 4.Kg2 Bc7 5.Re8 Rd7 6.a4 Kf6 7.Rh8 Kg6 8.Rg8+ Kf5 9.Rf8+ Kg6 10.Rg8+ Kf6 11.Rh8 Kg7 12.Re8 Ne7 13.Kf1 Kf7 14.Rh8 Kg7 15.Re8 a6 16.Ke2 Ng6 17.hxg5 hxg5 18.Bg1 Kf7 19.Re4 Kf6 20.Nd2 Kf5 21.Re8 Bd6 22.Nb3
-1.17
depth: 28/73
214MN
tb=747
LC0 42783

Thanks, Hai. In the diagram position, SF's analysis, depth=44, is

-0.12 11.Ree2 Rfd6 12.Rxd6 Rxd6 13.Nd2 Rd3 14.Nc4 Kf6 15.hxg5+ hxg5 16.Rd2 Rxd2 17.Nxd2 Bc7 18.Kf1 Ne5 19.Ke2 b5 20.Bg1 Kf5 21.Bh2 Bd8 22.Nb3 Bb6 23.Nd2 Bc7 24.Bg1 Nc6 25.Kd3 Bb6 26.Ke2 Bd8 27.Nb3 Be7 28.Kd3 Ke5 29.Nd2 Kd5 30.b3 Ne5+ 31.Ke2 Bd8 32.Ne4 a5 33.Nd2 Be7 34.Bh2 b4 35.cxb4 axb4 36.Ne4 Kd4 37.Nd2
-0.19 11.Re4 Rxd2 12.Nxd2 Rd6 13.Re2 Rd3 14.Nc4 Kf6 15.hxg5+ hxg5 16.Rd2 Rxd2 17.Nxd2 Bc7 18.Kf1 Ne5 19.Ke2 b5 20.Bg1 Kf5 21.Bh2 Bd8 22.Nb3 Bb6 23.Nd2 Ke6 24.Bg1 Bc5 25.Nb3 Kd5 26.Nd2 Be7 27.b3 Bd8 28.Bh2 Ke6 29.Bg1 Kf5 30.Bh2 Nc6 31.Kd3 Bf6 32.b4 Ne5+ 33.Ke2 Be7 34.Bg1 Bd8 35.Bh2

But we cannot say that Leela & SF have a genuine difference in evaluating positions here.

In Leela's PV after Black's a6 ---

[d]4R3/1pbrn1k1/p6p/6p1/P1N2p1P/2P2P2/1P3P1B/5K2 w - - 0 26

--- SF's eval (depth=44) is 0.00:

0.00 26.hxg5 hxg5 27.Ke2 Nf5 28.Nd2 Kf7 29.Rc8 Nh4 30.Rh8 Kg7 31.Re8 Kf7
-0.13 26.Ra8 g4 27.Ra7 Bb8 28.Ra8 gxf3 29.Ke1 Nc6 30.Nd2 Kg6 31.Nxf3 Kh5 32.b4 Kg4 33.Nd2 Be5 34.Ke2 Bxc3 35.f3+ Kh3 36.Bxf4 Nd4+ 37.Kf2 Bxb4 38.Re8 Be7 39.Bxh6 Bxh4+ 40.Kf1 Rc7 41.Ne4 Rc2 42.Be3 Nxf3 43.Nf2+ Bxf2 44.Bxf2 Kg4 45.Re4+ Kf5 46.Rb4 Nd2+ 47.Ke1 Nc4 48.Rxb7 Ke4 49.Bh4 Rh2 50.Re7+ Kd3 51.Bf6 Nd2 52.Rd7+ Kc2 53.Rb7 a5 54.Re7 Kb3

But SF thinks Leela's next PV White move, Ke2, is bad (depth=45):

-1.10 26...gxh4 27.Nd2 Nf5 28.Re4 h3 29.Rc4 Be5 30.Ne4 Kg6 31.Rb4 Re7 32.Rb6+ Kh5 33.Kf1 Nh4 34.Ke2 Bc7 35.Rb4 Bb8 36.b3 Be5 37.Rb6 Bc7 38.Rb4 Bb8 39.Rd4 Nf5 40.Rd5 Kg6 41.Kf1 Nh4 42.Rd3 Bc7 43.b4 b6 44.a5 bxa5 45.bxa5 Nxf3 46.Rxf3 Rxe4 47.Rxh3 h5 48.Rf3 Rc4 49.Kg2 Kf6 50.Rh3
...

So they agree about that line ( -1.17 vs -1.10), but SF thinks White should play hxg5 instead (0.00).

And we already saw (in the post http://talkchess.com/forum3/viewtopic.p ... 27#p805327 ) that in the OP position

[d]r4rk1/pp6/1bn1R2p/6pb/5p2/1NP2N1P/PP3PPB/3R2K1 b - - 0 7

SF & Leela both gave Black an advantage of 0.5-1.0, and there isn't evidence that it's not a draw.


It looks like the claim some make in this thread that SF is evaluating the position differently from Leela, and SF is the engine misevaluating, is incorrect. From what I've seen in general, if you give SF and Leela time, they mostly agree, and when they look like they disagree, it's just because somewhere down Leela's PV it forks to a line that SF thinks is not the PV.

Ok, after your long analysis, what do you think about this position?

Isn't it a forced loss that SF dont see it due to her limit in search horizon??

[jp]

Ok, after your long analysis, what do you think about this position?

Isn't it a forced loss that SF dont see it due to her limit in search horizon??

I don't think we can be sure, but it looks most likely to be a draw with best play.
Of course, White would probably have to be crazy to torture himself like this willingly.

Did you see the post on the previous page? Have a look if you didn't. It looks like if you jump off from Leela's long PV (the one George got starting from the OP diagram position) at different moves you end up in tablebase draws. When you keep finding tablebase draws wherever you look, it feels like it probably is a theoretical draw.

In the Leela PV, White does free his bishop, at a small cost (and Leela does not think that is decisive).

In the SF PV, White accepts doubled pawns on f2 & f3, which lock in the bishop even more, but maybe the claimed drawn positions it gets from there are what humans would understand as fortresses.

[jp]

... Which leaves the BrainFish-Leela (Nordlandia) game finish, which deviates from the SF & Leela analysis on this page around move 11.

[pgn]
[Date "2019.07.14"]
[White "BrainFish_190713_x64_bmi2"]
[Black "lc0"]
[Result "0-1"]
[TimeControl "900+30"]
[FEN "r4rk1/pp6/1bn1R2p/6pb/5p2/1NP2N1P/PP3PPB/3R2K1 b - - 0 7"]
[SetUp "1"]

7... Rad8 8. Rd2 Bxf3 9. gxf3 Kg7 10. h4 Rf6 11. Re4 Rfd6 12. Rde2 Rd1+ 13. Kg2 R8d5 14. Re8 Bd8 15. h5 Bf6 16. Rc8 Kf7 17. Rc7+ Rd7 18. Rxd7+ Rxd7 19. Nc5 Rc7 20. Ne4 Be7 21. Rc2 Rd7 22. Nd2 b5 23. Kf1 a5 24. Ke2 Rc7 25. Rc1 a4 26. a3 Ne5 27. Bg1 Ke6 28. Kd1 Rd7 29. Ke2 Kd5 30. Rc2 Rd6 31. Bh2 Re6 32. Kf1 Nc4 33. Ne4 Re5 34. Bg1 Bf8 35. Nf6+ Kc6 36. Ne4 Kd5 37. Bh2 Bg7 38. Bg1 Re7 39. Bh2 Rd7 40. Ke1 Kc6 41. Bg1 Rd3 42. Ke2 Rd8 43. Ke1 Be5 44. Ke2 Bc7 45. Nf6 Bd6 46. Ng4 Be5 47. Nxh6 Rh8 48. Ng4 Rxh5 49. Nh2 Rh6 50. Kd1 Rd6+ 51. Ke2 Re6 52. Kd1 Bg7 53. Nf1 Rh6 54. Nh2 Rd6+ 55. Ke2 Re6+ 56. Kd1 Kc5 57. Nf1 Rh6 58. Bh2 Ne5 59. Ke2 Kc6 60. Bxf4 gxf4 61. Nd2 Re6 62. Ne4 Nc4 63. Kd1 Bf8 64. Re2 Be7 65. Kc2 Re5 66. Re1 Nd6 67. Re2 Nxe4 68. fxe4 Rh5 69. f3 Rh3 70. Rf2 Bc5 { Black wins. } 71. Rf1 0-1[/pgn]

The question is whether BrainFish just played badly to lose. The pgn says: "TimeControl 900+30".

After move 14, SF thinks it's 0.00 (depth=39):
0.00 15.a4 Kf7 16.Rh8 Bf6 17.Rc8 Rd8 18.Rc7+ R1d7 19.Rxd7+ Rxd7 20.Nc5 Rd5 21.Nxb7 Ne5 22.b4 g4 23.Rxe5 Bxe5 24.fxg4 f3+ 25.Kh3 Bxc3 26.Nd6+ Ke7 27.Nf5+ Ke6 28.h5 Rd2 29.Bg3 Ra2 30.a5 Bd2 31.Bb8 Bxb4 32.Bxa7 Bxa5 33.Be3 Bd2 34.Bxd2 Rxd2 35.Kg3 Ra2
...

SF doesn't like 15. h5. It's not a huge negative score, but it looks like SF thinks the position gets gradually worse and worse from that position.

[d]3bR3/pp4k1/2n4p/3r2pP/5p2/1NP2P2/PP2RPKB/3r4 b - - 0 15

SF's eval at depth=45:
-0.56 15...Bf6 16.Rc8 Kf7 17.Rc7+ Rd7 18.Rxd7+ Rxd7 19.Nc5 Rc7 20.Ne4 Be7 21.Nd2 Rd7 22.Nc4 a6 23.a4 Rd5 24.Bg1 Rd3 25.Re1 Kf6 26.Ra1 Ne5 27.Nxe5 Kxe5 28.Rc1 Ke6 29.Rc2 Kf6 30.Rc1 Rd2 31.b4 Bd6 32.Re1 Be5 33.c4 Rb2 34.b5 Rb4 35.bxa6 bxa6 36.c5 Rc4 37.Rb1 Bc7 38.Re1 Rxc5 39.Re8 Rc6 40.Rh8
...

[zullil]

Lc0 v21.2 JH T8-swa-610000(the TCEC sufi15 winner net):

Code: Select all

 22/55	00:42	 1.448.327	33.759	-0,92	Ra8-d8 Rd1xd8 Rf8xd8 Nb3-d2 Bh5-f7 Re6-e2 Bf7xa2 g2-g3 Bb6-c7 g3xf4 g5xf4 Nf3-h4 Ba2-d5 f2-f3 Kg8-h7 Kg1-f1 Rd8-f8 Bh2-g1 a7-a5 Bg1-c5 Rf8-d8 Nh4-f5 Rd8-d7 Nd2-e4 Bc7-d8 Kf1-g2 Kh7-g6 Nf5-d6 b7-b6 Bc5-f2 Nc6-e5 Bf2-d4 Ne5-c4 Nd6xc4 Bd5xc4 Re2-e1 a5-a4 Ne4-d2 Bc4-f7 Re1-e4 Bd8-c7 Nd2-b1
 22/55	00:43	 1.457.324	33.764	-0,92	Ra8-d8 Rd1xd8 Rf8xd8 Nb3-d2 Bh5-f7 Re6-e2 Bf7xa2 g2-g3 Bb6-c7 g3xf4 g5xf4 Nf3-h4 Kg8-h7 Nh4-g2 Rd8-f8 b2-b3 f4-f3 Nd2xf3 Rf8xf3 Bh2xc7 Ba2xb3 Re2-d2 Rf3xc3 Rd2-d7+ Kh7-g8 Bc7-d6 b7-b5 Rd7-b7 b5-b4 Bd6xb4 Nc6xb4 Rb7xb4 a7-a5 Rb4-b6 a5-a4 Rb6-a6 Rc3-c1+ Kg1-h2 Rc1-a1 Ra6-b6 Ra1-b1 Rb6-a6 h6-h5 Ng2-e3 Rb1-a1 Ra6-b6 Bb3-f7 Rb6-b8+ Kg8-g7 Rb8-b7 a4-a3

Since this thread seems to still be living, here's a Lc0 search of the original position, this time with network 42850 and more than 30,000,000 nodes:

Code: Select all

$ ./lc0
       _
|   _ | |
|_ |_ |_| v0.22.0 built Aug  5 2019
setoption name Backend value cudnn-fp16
setoption name NNCacheSize value 40000000
ucinewgame
Found pb network file: ./network42850
Creating backend [cudnn-fp16]...
GPU: GeForce RTX 2080 Ti
GPU memory: 10.7534 Gb
GPU clock frequency: 1635 MHz
GPU compute capability: 7.5
CUDA Runtime version: 10.1.0
Cudnn version: 7.6.2
Latest version of CUDA supported by the driver: 10.1.0
position fen r4rk1/pp6/1bn1R2p/6pb/5p2/1NP2N1P/PP3PPB/3R2K1 b - - 0 7
go infinite

...

info depth 32 seldepth 73 time 895750 nodes 31677849 score cp 93 hashfull 408 nps 35364 tbhits 0 pv a8d8 d1d8 f8d8 b3d2 h5f7 e6e2 f7a2 g2g3 b6c7 g3f4 g5f4 f3h4 a2d5 f2f3 g8h7 g1f1 d8f8 h2g1 c7d6 e2e1 b7b5 b2b4 d6e5 d2e4 f8d8 e1c1 a7a6 g1b6 d8d7 f1g2 d5e4 f3e4 d7d3 c3c4 d3g3 g2f1 e5b2 c1c2 b2c3 c4b5 a6b5 b6c5 g3h3 h4f5 h7g6 c5d6 c6b4 c2g2 g6f6 d6f4 b4d3
bestmove a8d8 ponder d1d8
quit

[jp]

Lc0 v21.2 JH T8-swa-610000(the TCEC sufi15 winner net)

Lc0 search of the original position, this time with network 42850 and more than 30,000,000 nodes:

Code: Select all

position fen r4rk1/pp6/1bn1R2p/6pb/5p2/1NP2N1P/PP3PPB/3R2K1 b - - 0 7
go infinite

So its eval is almost the same (-0.93) and PV looks the same as the first (i.e. second-last) line of the T8-swa-610000 output for about 18 plies and the second (i.e. last) line for about 12 plies.

But does the position fen in the code box mean you just started from that position? Aren't you supposed to feed in a few prior moves for the history panes?

[zullil]

Aren't you supposed to feed in a few prior moves for history panes filling?

Am I? Hope the Leela-police don't come and confiscate my GPU.

I'll search again, this time after having Lc0 step its way through the entire PGN.

So, when I search with an alpha-beta engine, should I also start prior to the posted position, to allow the engine to start filling its hash table?

[jp]

Aren't you supposed to feed in a few prior moves for history panes filling?

Am I? Hope the Leela-police don't come and confiscate my GPU.

I'll search again, this time after having Lc0 step its way through the entire PGN.

I don't know. And I don't know if the others started from the start of the pgn or not. As I wrote, your output agrees closely with George's with a different net ID. Maybe crem can tell us exactly what the supposed problem is.

A Leela blog post last year (but not written by crem) said

Note that the analysis ... for all positions by Leela should be done by not providing the simple FEN or EPD and let it analyze, but by providing a history of moves that ended in that position. ... But practically ... you do not need to provide Leela with a full history from starting position to the desired one, but only 2-3 moves. By making her analyze from a FEN, you are IN A HUGE DANGER of obtaining an analysis that is completely meaningless.

(But e.g. you'd think there must be some way for it to analyse the initial position of chess without a problem.)