LCZero: Progress and Scaling. Relation to CCRL Elo

[Albert Silver]

Lol, here is a test for you. Play against your strongest LC0 10s per move or even more if you wish. You with naked king and give LC0 bishop and knight. You can try couple of times. And please tell us the result.
Than take any below 2500 Elo engine without TBs and repeat. And tell us the result.

No need. It used to have trouble until it learned how to win them. All this has been tracked in detail:

https://docs.google.com/spreadsheets/d/ ... edit#gid=0

do you know who can do it in least amount of moves stockfish or lc0?

No, but it provides comparisons with optimal TB play, which is better IMO.

[glennsamuel32]

I noticed yesterday that surprisingly Leela "solves" a good position.

White to play 1.Nd5!! and win. 1.Re4 just draws....
[d]8/2p2p1p/3k1p2/pn6/PNp4p/r3Rp1P/2P2P2/BK6 w - - 0 1

It finds Nd5!! A superb move! In a superb position(mate in 178).
But don't get too excited, as of course she finds it for wrong reasons and doesn't see it is winning. But she finds it and for someone that tries to solve a position the fact that Nd5 has so many visits and it's the preferable move of Leela, should ring a bell when analyzing a position that this move has its merits.
To be fair to other engines, Leela underestimates Re4 and thinks black is better, which is not the case as white puts much pressure to black with Re4, but can't win.



Good old [d] doesn't wok as it seems.

Could you post the winning PV ?
Thanks...

[yanquis1972]

been watching a 1'1 match between leela & SF8 on one core, & it's transparently clear that, while tactics are an obvious issue, leela's woeful misevaluation of endgames is bleeding a lot of wins (& draws). i don't really understand how, after 15 million games, it could be so bad, but she has repeatedly dissolved into drawn endgames she thinks are +2 or +3. KRB vs KR is the last example off the top of my ahead, & i think the game eval was something like +2.50.

is there any concrete evidence this is improving?

Well, for this to be a provable issue (compared to other top engines) you would need to show that other engines did better in the exact same time. Share a position where she made a mistake, entering the wrong endgame, and that SF or Komodo find the solution in that same time control.

You are aware that KRB vs KR is quite winnable, and depends very much on the defender having the right position and not making any mistakes even then? There are excellent practical chances to win it.

ha -- actually, i had no idea. i assumed (for computers, even without TBs) it was an easy draw in unexceptional cases. that definitely makes the current eval there understandable. when i have time i'll see if i can find 2-3 examples of other instances & put them to your test.

[George Tsavdaris]

I noticed yesterday that surprisingly Leela "solves" a good position.

White to play 1.Nd5!! and win. 1.Re4 just draws....
[d]8/2p2p1p/3k1p2/pn6/PNp4p/r3Rp1P/2P2P2/BK6 w - - 0 1

It finds Nd5!! A superb move! In a superb position(mate in 178).
But don't get too excited, as of course she finds it for wrong reasons and doesn't see it is winning. But she finds it and for someone that tries to solve a position the fact that Nd5 has so many visits and it's the preferable move of Leela, should ring a bell when analyzing a position that this move has its merits.
To be fair to other engines, Leela underestimates Re4 and thinks black is better, which is not the case as white puts much pressure to black with Re4, but can't win.



Good old [d] doesn't wok as it seems.

Could you post the winning PV ?
Thanks...

[Event "??"]
[Date "??.??.??"]
[Round "?"]
[White "??"]
[Black "??"]
[Result "1-0"]
[TimeControl "-"]
[ECO "?"]
[Opening "?"]
[FEN "8/2p2p1p/3k1p2/pn6/PNp4p/r3Rp1P/2P2P2/BK6 w - - 0 1"]
[SetUp "1"]

1. Nd5 Kxd5 2. Rxa3 Nxa3+ 3. Kb2 c3+ 4. Kxa3 Kc4 5. Ka2 Kb4 6. Kb1 Kc4 7. Kc1 Kb4 8. Kd1 Kc4 9. Ke1 Kb4 10. Kf1 Kc4 11. Kg1 Kb4 12. Kh1 Kc4 13. Kh2 Kb4 14. Kg1 Kc4 15. Kf1 Kb4 16. Ke1 Kc4 17. Kd1 Kb4 18. Kc1 Kc4 19. Kb1 Kb4 20. Ka2 Kc4 21. Ka3 h6 22. Ka2 Kb4 23. Kb1 Kc4 24. Kc1 Kb4 25. Kd1 Kc4 26. Ke1 Kb4 27. Kf1 Kc4 28. Kg1 Kb4 29. Kh2 Kc4 30. Kh1 Kb4 31. Kg1 Kc4 32. Kf1 Kb4 33. Ke1 Kc4 34. Kd1 Kb4 35. Kc1 Kc4 36. Kb1 Kb4 37. Ka2 Kc4 38. Ka3 h5 39. Ka2 Kb4 40. Kb1 Kc4 41. Kc1 Kb4 42. Kd1 Kc4 43. Ke1 Kb4 44. Kf1 Kc4 45. Kg1 Kb4 46. Kh2 Kc4 47. Kh1 Kb4 48. Kg1 Kc4 49. Kf1 Kb4 50. Ke1 Kc4 51. Kd1 Kb4 52. Kc1 Kc4 53. Kb1 Kb4 54. Ka2 Kc4 55. Ka3 f5 56. Ka2 Kb4 57. Kb1 Kc4 58. Kc1 Kb4 59. Kd1 Kc4 60. Ke1 Kb4 61. Kf1 Kc4 62. Kg1 Kb4 63. Kh1 Kc4 64. Kh2 Kb4 65. Kg1 Kc4 66. Kf1 Kb4 67. Ke1 Kc4 68. Kd1 Kb4 69. Kc1 Kc4 70. Kb1 Kb4 71. Ka2 Kc4 72. Ka3 f4 73. Ka2 Kb4 74. Kb1 Kc4 75. Kc1 Kb4 76. Kd1 Kc4 77. Ke1 Kb4 78. Kf1 Kc4 79. Kg1 Kb4 80. Kh2 Kc4 81. Kh1 Kb4 82. Kg1 Kc4 83. Kf1 Kb4 84. Ke1 Kc4 85. Kd1 Kb4 86. Kc1 Kc4 87. Kb1 Kb4 88. Ka2 Kc4 89. Ka3 f6 90. Ka2 Kb4 91. Kb1 Kc4 92. Kc1 Kb4 93. Kd1 Kc4 94. Ke1 Kb4 95. Kf1 Kc4 96. Kg1 Kb4 97. Kh2 Kc4 98. Kh1 Kb4 99. Kg1 Kc4 100. Kf1 Kb4 101. Ke1 Kc4 102. Kd1 Kb4 103. Kc1 Kc4 104. Kb1 Kb4 105. Ka2 Kc4 106. Ka3 f5 107. Ka2 Kb4 108. Kb1 Kc4 109. Kc1 Kb4 110. Kd1 Kc4 111. Ke1 Kb4 112. Kf1 Kc4 113. Kg1 Kb4 114. Kh2 Kc4 115. Kh1 Kb4 116. Kg1 Kc4 117. Kf1 Kb4 118. Ke1 Kc4 119. Kd1 Kb4 120. Kc1 Kc4 121. Kb1 Kb4 122. Ka2 Kc4 123. Ka3 c6 124. Ka2 Kb4 125. Kb1 Kc4 126. Kc1 Kb4 127. Kd1 Kc4 128. Ke1 Kb4 129. Kf1 Kc4 130. Kg1 Kb4 131. Kh2 Kc4 132. Kh1 Kb4 133. Kg1 Kc4 134. Kf1 Kb4 135. Ke1 Kc4 136. Kd1 Kb4 137. Kc1 Kc4 138. Kb1 Kb4 139. Ka2 Kc4 140. Ka3 c5 141. Ka2 Kb4 142. Kb1 Kc4 143. Kc1 Kb4 144. Kd1 Kc4 145. Ke1 Kb4 146. Kf1 Kc4 147. Kg1 Kb4 148. Kh2 Kc4 149. Kh1 Kb4 150. Kg1 Kc4 151. Kf1 Kb4 152. Ke1 Kc4 153. Kd1 Kb4 154. Kc1 Kc4 155. Kb1 Kb4 156. Ka2 Kc4 157. Ka3 Kd4 158. Kb3 c4+ 159. Ka3 Kc5 160. Bxc3 { Black resigns. } 1-0

[glennsamuel32]

White to play 1.Nd5!! and win. 1.Re4 just draws....
[d]8/2p2p1p/3k1p2/pn6/PNp4p/r3Rp1P/2P2P2/BK6 w - - 0 1

It finds Nd5!! A superb move! In a superb position(mate in 178).
But don't get too excited, as of course she finds it for wrong reasons and doesn't see it is winning. But she finds it and for someone that tries to solve a position the fact that Nd5 has so many visits and it's the preferable move of Leela, should ring a bell when analyzing a position that this move has its merits.
To be fair to other engines, Leela underestimates Re4 and thinks black is better, which is not the case as white puts much pressure to black with Re4, but can't win.



Good old [d] doesn't wok as it seems.
[/quote]

Thanks for posting the game.
This is far from any forced mate.
I ran Asmfish with matefinder on it.
Even at 4 PV's, black doesn't play ..5. Kb4
The names of the engines playing weren't available and so I'm sure the engine playing black wasn't any of the stronger ones.

[glennsamuel32]

This is Houdini tactical...

[George Tsavdaris]

White to play 1.Nd5!! and win. 1.Re4 just draws....
[d]8/2p2p1p/3k1p2/pn6/PNp4p/r3Rp1P/2P2P2/BK6 w - - 0 1

It finds Nd5!! A superb move! In a superb position(mate in 178).
But don't get too excited, as of course she finds it for wrong reasons and doesn't see it is winning. But she finds it and for someone that tries to solve a position the fact that Nd5 has so many visits and it's the preferable move of Leela, should ring a bell when analyzing a position that this move has its merits.
To be fair to other engines, Leela underestimates Re4 and thinks black is better, which is not the case as white puts much pressure to black with Re4, but can't win.



Good old [d] doesn't wok as it seems.

Thanks for posting the game.
This is far from any forced mate.

This is a forced mate! A checkmate in 178. A forced one.

I ran Asmfish with matefinder on it.
Even at 4 PV's, black doesn't play ..5. Kb4

Asmfish is helpless to understand this position.
And you also don't seem to understand it.
Black tries NOT to play any Pawn moves. Because if he does, then eventually after he has no other Pawn moves, white King will go to b3 and win.
So black has to play 4...Kb4. Because if he doesn't white gets closer to mating him. If black plays a Pawn move then white goes to a3 and then black has to play a Pawn move again!
So black has to play Kb4.
But after ...Kb4 then white does the journey with the triangulation g1->h1->h2->g1 and then after he reaches a3 he forces black to move his Pawns as Kb4 is not possible.

[glennsamuel32]

Asmfish is helpless to understand this position.
And you also don't seem to understand it.
Black tries NOT to play any Pawn moves. Because if he does, then eventually after he has no other Pawn moves, white King will go to b3 and win.
So black has to play 4...Kb4. Because if he doesn't white gets closer to mating him. If black plays a Pawn move then white goes to a3 and then black has to play a Pawn move again!
So black has to play Kb4.
But after ...Kb4 then white does the journey with the triangulation g1->h1->h2->g1 and then after he reaches a3 he forces black to move his Pawns as Kb4 is not possible.

Doesn't make any sense...
You say black tries not to play any pawn moves and plays Kb4.
I have visually showed you asmfish (which is stockfish) and houdini, prefering 4 alternate pawn moves at high depths.
So your analysis does not hold water.
Maybe you could post the game with the depth and evals included as well as the engine names so I can try to replicate it...
Also, what was the game tc ?

[yanquis1972]

you're working the wrong way. do a backwards analysis & if the engine thinks it sees a draw, try to prove it wrong. you're expecting x engine to see a (proposed) mate in 174 in a structure like that; it's ridiculous.

[glennsamuel32]

What's the point in working backwards from move 175 when move 5 is dubious ???
Also, I'm showing that move 5 could not have happened with stronger engines...
Btw, what's the TC ?