LCZero update

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

jhellis3
Posts: 546
Joined: Sat Aug 17, 2013 12:36 am

Re: LCZero update

Post by jhellis3 »

LCZero went 23 - 75 - 2 against Stockfish (L0), which would give it a conservative Elo rating around 900 by now.
It is also improving at a rate of ~100 "real world" Elo per day (conservatively) with quite limited resources.

Of course that won't hold for too long before saturation and overall strength requiring more playouts/depth becomes an issue. Still, I wonder how many people believe a human based eval engine will be the strongest available by the end of the year.
Kiudee
Posts: 29
Joined: Tue Feb 02, 2010 10:12 pm
Location: Germany
Full name: Karlson Pfannschmidt

Re: LCZero update

Post by Kiudee »

Here is a game between Leela Chess (6d2eaec0) at 50k playouts against Lichess Stockfish Level 2:

[pgn]
[Date "2018.03.14"]
[Round "-"]
[White "Leela Chess 6d2eaec0"]
[Black "lichess AI level 2"]
[Result "1-0"]
[ECO "D02"]
[Opening "Queen's Pawn Game: Symmetrical Variation"]

1. d4 Nf6 2. Nf3 d5 { D02 Queen's Pawn Game: Symmetrical Variation } 3. Nc3 Nc6 4. a4 e6 5. Rb1 Ne4 6. Bd2 Bd7 7. Bf4 Bb4 8. Bd2 O-O 9. Nxe4 dxe4 10. c3 exf3 11. exf3 Be7 12. b3 Bd6 13. d5 exd5 14. f4 Bf5 15. Rc1 Ba3 16. Be2 Qe8 17. Ra1 Bb2 18. Rc1 Bxc1 19. Qxc1 Bd3 20. Be3 Bf5 21. Bd2 Bg4 22. f3 Bf5 23. Kd1 a6 24. c4 Nd4 25. Be3 Nc2 26. Bc5 c6 27. cxd5 Qd8 28. d6 Re8 29. g4 Bg6 30. f5 Rxe2 31. Kxe2 Qe8+ 32. Kf2 f6 33. Qxc2 Bf7 34. Qe4 Bxb3 35. Rb1 Qxe4 36. fxe4 Bxa4 37. Rxb7 Bb5 38. d7 Rd8 39. Bb6 Ra8 40. Ra7 Rb8 41. d8=Q+ Rxd8 42. Bxd8 Bd3 43. Ke3 Bb5 44. Bb6 Bc4 45. Kf3 Bd3 46. h4 g6 47. Bd8 c5 48. Bxf6 gxf5 49. exf5 h6 50. Ke3 Bf1 51. Bg7 Bh3 52. Bxh6 Kh8 53. f6 Bxg4 54. f7 Be6 55. f8=Q+ Bg8 56. Qg7# { White wins by checkmate. } 1-0
[/pgn]
User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: LCZero update

Post by Ozymandias »

Uri Blass wrote:From CCRL:

LaMoSca 0.10-Brutus RND 10.5-9.5(only one win for LaMoSca)
Ram 2.0-Brutus RND 20-0

Ram 2.0-LaMoSca 0.10 11.5-9.5(only 2 wins for Ram)


Without seeing the games I suspect that LaMoSca is one of the program that should be removed from CCRL because it artificially increase the rating of the random player.

Probably LaMoSca is relatively strong in the opening but later allow stalemates or repetitions and without seeing the games I suspect that in most games it get a winning advantage against Ram 2.0 but translate it to a draw.
Absolutely right, look at this game, for example; after move 33, black is left without any more choices other than moving the king around, and white just refuses to checkmate:

[pgn][Event "CCRL 40/4"]
[Site "CCRL"]
[Date "2012.08.10"]
[Round "173.4.75"]
[White "LaMoSca 0.10"]
[Black "Brutus RND"]
[Result "1/2-1/2"]
[WhiteElo "477"]
[BlackElo "202"]
[ECO "D10"]
[Opening "QGD Slav defence"]
[PlyCount "158"]

1. d4 d5 2. c4 c6 3. Qc2 Nf6 4. Nf3 g6 5. e3 Bf5 6. Bd3 Bxd3 7. Qxd3 Bg7 8. Qd1 Qd7 9. cxd5 Qxd5 10. h4 b5 11. Nc3 Qh5 12. Ne2 Bh6 13. b4 Bg7 14. Bb2 e6 15. Nf4 h6 16. Nxh5 e5 17. Nxf6+ Bxf6 18. Bc3 e4 19. Nd2 Na6 20. Qc1 Ke7 21. Nb3 Nc5 22. Nxc5 Bxd4 23. Bxd4 Rhc8 24. Bf6+ Ke8 25. f4 a6 26. Qa3 g5 27. fxg5 Rd8 28. Bxd8 h5 29. Bf6 Rc8 30. Qxa6 Rb8 31. Qxc6+ Kf8 32. Nd7+ Ke8 33. Nxb8+ Kf8 34. Qxe4 Kg8 35. Nc6 Kf8 36. Ne5 Kg8 37. Qd5 Kh7 38. Qxb5 Kg8 39. Qe2 Kf8 40. Qxh5 Ke8 41. Nxf7 Kd7 42. Bd4 Ke7 43. Ne5 Kd8 44. e4 Ke7 45. g6 Kd8 46. g7 Kc7 47. b5 Kd6 48. b6 Ke7 49. Qg6 Kd8 50. a4 Kc8 51. g4 Kb7 52. a5 Kc8 53. a6 Kd8 54. b7 Ke7 55. a7 Kd8 56. Qc6 Ke7 57. g5 Kd8 58. g6 Ke7 59. h5 Kd8 60. h6 Ke7 61. Qd7+ Kf6 62. Qc8 Kg5 63. Qc6 Kf4 64. Qc2 Kg3 65. h7 Kf4 66. Nd7 Kg3 67. Nb6 Kg4 68. Nd5 Kg3 69. e5 Kg4 70. e6 Kg3 71. e7 Kf3 72. Nc7 Kg4 73. Nd5 Kf3 74. Nc7 Kg4 75. Nd5 Kg3 76. Nc7 Kf4 77. Nd5+ Kg5 78. Nc7 Kf4 79. Nd5+ Kf3 1/2-1/2
[/pgn]
CMCanavessi wrote:I have ran LaMoSca in my tournaments A LOT and it has an issue, it can't win (or maybe it was programed that way deliberately). It will not promote pawns, they will all remain in the 7th rank forever (except very rare cases where they can take a piece in the 8th rank). Its "real" strenght is around 800-1000 elo, but after capturing every piece of the oponent, it will 3-fold. That's why it will have a lot of draws, and no wins.
That's a perfect review of the above game.
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: LCZero update

Post by CMCanavessi »

jhellis3 wrote:
LCZero went 23 - 75 - 2 against Stockfish (L0), which would give it a conservative Elo rating around 900 by now.
It is also improving at a rate of ~100 "real world" Elo per day (conservatively) with quite limited resources.

Of course that won't hold for too long before saturation and overall strength requiring more playouts/depth becomes an issue. Still, I wonder how many people believe a human based eval engine will be the strongest available by the end of the year.
I guess that as LCZ strenght goes up, people will start paying more attention to it, meaning more people will help the project by training it. IMHO, it will have to get to ~2700 elo to catch some attraction. After that, it may become the new stockfish, or maybe stockfish will become the new LCZ :D
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: LCZero update

Post by CMCanavessi »

Ozymandias wrote:
Uri Blass wrote:From CCRL:

LaMoSca 0.10-Brutus RND 10.5-9.5(only one win for LaMoSca)
Ram 2.0-Brutus RND 20-0

Ram 2.0-LaMoSca 0.10 11.5-9.5(only 2 wins for Ram)


Without seeing the games I suspect that LaMoSca is one of the program that should be removed from CCRL because it artificially increase the rating of the random player.

Probably LaMoSca is relatively strong in the opening but later allow stalemates or repetitions and without seeing the games I suspect that in most games it get a winning advantage against Ram 2.0 but translate it to a draw.
Absolutely right, look at this game, for example; after move 33, black is left without any more choices other than moving the king around, and white just refuses to checkmate:

[pgn][Event "CCRL 40/4"]
[Site "CCRL"]
[Date "2012.08.10"]
[Round "173.4.75"]
[White "LaMoSca 0.10"]
[Black "Brutus RND"]
[Result "1/2-1/2"]
[WhiteElo "477"]
[BlackElo "202"]
[ECO "D10"]
[Opening "QGD Slav defence"]
[PlyCount "158"]

1. d4 d5 2. c4 c6 3. Qc2 Nf6 4. Nf3 g6 5. e3 Bf5 6. Bd3 Bxd3 7. Qxd3 Bg7 8. Qd1 Qd7 9. cxd5 Qxd5 10. h4 b5 11. Nc3 Qh5 12. Ne2 Bh6 13. b4 Bg7 14. Bb2 e6 15. Nf4 h6 16. Nxh5 e5 17. Nxf6+ Bxf6 18. Bc3 e4 19. Nd2 Na6 20. Qc1 Ke7 21. Nb3 Nc5 22. Nxc5 Bxd4 23. Bxd4 Rhc8 24. Bf6+ Ke8 25. f4 a6 26. Qa3 g5 27. fxg5 Rd8 28. Bxd8 h5 29. Bf6 Rc8 30. Qxa6 Rb8 31. Qxc6+ Kf8 32. Nd7+ Ke8 33. Nxb8+ Kf8 34. Qxe4 Kg8 35. Nc6 Kf8 36. Ne5 Kg8 37. Qd5 Kh7 38. Qxb5 Kg8 39. Qe2 Kf8 40. Qxh5 Ke8 41. Nxf7 Kd7 42. Bd4 Ke7 43. Ne5 Kd8 44. e4 Ke7 45. g6 Kd8 46. g7 Kc7 47. b5 Kd6 48. b6 Ke7 49. Qg6 Kd8 50. a4 Kc8 51. g4 Kb7 52. a5 Kc8 53. a6 Kd8 54. b7 Ke7 55. a7 Kd8 56. Qc6 Ke7 57. g5 Kd8 58. g6 Ke7 59. h5 Kd8 60. h6 Ke7 61. Qd7+ Kf6 62. Qc8 Kg5 63. Qc6 Kf4 64. Qc2 Kg3 65. h7 Kf4 66. Nd7 Kg3 67. Nb6 Kg4 68. Nd5 Kg3 69. e5 Kg4 70. e6 Kg3 71. e7 Kf3 72. Nc7 Kg4 73. Nd5 Kf3 74. Nc7 Kg4 75. Nd5 Kg3 76. Nc7 Kf4 77. Nd5+ Kg5 78. Nc7 Kf4 79. Nd5+ Kf3 1/2-1/2
[/pgn]
CMCanavessi wrote:I have ran LaMoSca in my tournaments A LOT and it has an issue, it can't win (or maybe it was programed that way deliberately). It will not promote pawns, they will all remain in the 7th rank forever (except very rare cases where they can take a piece in the 8th rank). Its "real" strenght is around 800-1000 elo, but after capturing every piece of the oponent, it will 3-fold. That's why it will have a lot of draws, and no wins.
That's a perfect review of the above game.
I have only seen it win twice, both due to capturing moves. It looks to be the only way it can promote a pawn or checkmate (maybe a bug in the code that deliberately prevents it making winning moves, just speculating).

Here's one game:

[pgn][Event "009 - Qualy League"]
[EventDate "?"]
[Site "RYZEN"]
[Date "2018.03.11"]
[Round "7"]
[White "LaMoSca v0.10 x32"]
[Black "CPP1 0.1038 x32"]
[Result "1-0"]
[TimeControl "60+1"]
[Opening "English"]
[ECO "A15"]
[Time "20:23:17"]
[Variation "Anglo-Indian, 2.Nf3"]
[Termination "normal"]
[PlyCount "53"]
[WhiteType "program"]
[BlackType "program"]

1. Nf3 Nf6 2. c4 Nc6 3. g3 e5 4. Bg2 d6 5. Nc3 Be7 6. d4 exd4 7. Nxd4 Ne5 8. f4 Ned7
9. Kd2 b6 10. Nc6 a6 11. Nxd8 Bxd8 12. Bxa8 c5 13. e4 g6 14. Qe1 Rg8 15. Kd3 Rf8 16. e5 Ng4
17. exd6+ Nge5+ 18. fxe5 h5 19. Kc2 a5 20. a4 Ba6 21. e6 Nb8 22. e7 Rg8 23. Bd5 f5 24. Bxg8 Kd7
25. Bg5 Bc8 26. exd8=Q+ Kc6 27. Bd5+ 1-0
[/pgn]


And the other one:

[pgn][Event "011 - Qualy League"]
[EventDate "?"]
[Site "RYZEN"]
[Date "2018.03.14"]
[Round "3"]
[White "Teki Random Mover x64"]
[Black "LaMoSca v0.10 x32"]
[Result "0-1"]
[TimeControl "60+1"]
[Opening "Nimzowitsch Defence"]
[ECO "B00"]
[Time "07:51:15"]
[Variation "2.Nc3 e6"]
[Termination "normal"]
[PlyCount "66"]
[WhiteType "program"]
[BlackType "program"]

1. e4 e6 2. Nc3 Nc6 3. g3 Bb4 4. Bg2 Nge7 5. Nge2 O-O 6. O-O d6 7. d3 f5 8. a3 Ba5
9. Nb1 Kf7 10. c3 fxe4 11. b3 exd3 12. h4 dxe2 13. Be3 exd1=Q 14. Bd4 e5 15. Kh2 Nf5 16. Bxe5 g5
17. Bh3 Nxe5 18. g4 Bxc3 19. Ra2 Nxg4+ 20. Kg1 Be5 21. Ra1 Bxa1 22. Re1 Nxh4 23. Kh1 Nxf2+ 24. Kh2 Qf6
25. Bf1 Qf4+ 26. Kg1 Qfg4+ 27. Bg2 Nd3 28. a4 Bd4+ 29. Kf1 Kg8+ 30. Bf3 Nxf3 31. Nc3 Bxc3 32. b4 Bxe1
33. a5 Bxb4+ 0-1

[/pgn]
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
Leo
Posts: 1080
Joined: Fri Sep 16, 2016 6:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: LCZero update

Post by Leo »

Not a very good start in my opinion.
Advanced Micro Devices fan.
Alexander Schmidt
Posts: 1203
Joined: Thu May 10, 2007 2:49 pm

Re: LCZero update

Post by Alexander Schmidt »

Jhoravi wrote:Hi. I explored some games on the given link http://162.217.248.187/user/GaryS But the blunderfeasted games are nowhere near 2000 elo IMO. Am I missing something?
The ELO value has nothing to do with the usual ELO values.

Leela Zero Go is now near 10.000 ELO in its selfplay list, catching up the strongest Go bots which are in real somewhere between 3000 and 3500 ELO.

http://zero.sjeng.org/
jhellis3
Posts: 546
Joined: Sat Aug 17, 2013 12:36 am

Re: LCZero update

Post by jhellis3 »

Not a very good start in my opinion.
I'll bite. Why?
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: LCZero update

Post by CMCanavessi »

Leo wrote:Not a very good start in my opinion.
Lol
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
flok

Re: LCZero update

Post by flok »

Got it to run. Which username/password should I use instead?