LCZero update

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Locked
gladius
Posts: 536
Joined: Tue Dec 12, 2006 9:10 am

LCZero update

Post by gladius » Wed Mar 14, 2018 4:30 am

We have kicked off the reinforcement learning phase, where the network starts from random, and learns from self-play games.

The server is up at http://162.217.248.187/, with over 280,000 games played so far. Huge thanks to all the people donating their machine time! We are up almost 2,000 elo from random play. The fun part is these games are totally unlike a traditional weak alpha-beta searcher. See some examples games from a user page, like this: http://162.217.248.187/user/GaryS (not actually me :).

People have been running it in tournaments against things like SF level 0 version to try and gauge the strength. It's just getting strong enough to actually start to measure. Here is an example tournament from zz4032:

Code: Select all

   # PLAYER                          :  RATING  ERROR  POINTS   GAMES  WINS(%)  WON  DRAWN  LOST  DRAWS(%)
   1 Stockfish8_level20_1min/game    :    3355   ----   210.5     400     52.6   60    301    39      75.2
   8 Stockfish8_level3               :    1381     97   402.0     800     50.2  396     12   392       1.5
   9 Stockfish8_level0               :     996    104   807.5    1200     67.3  805      5   390       0.4
  10 lczero_7f8f_playouts3300        :     532    115    26.5     400      6.6   26      1   373       0.2
  11 lczero_b91f_playouts3300        :     263    146     6.0     400      1.5    6      0   394       0.0

Jhoravi
Posts: 243
Joined: Wed May 08, 2013 4:49 am

Re: LCZero update

Post by Jhoravi » Wed Mar 14, 2018 6:35 am

Hi. I explored some games on the given link http://162.217.248.187/user/GaryS But the blunderfeasted games are nowhere near 2000 elo IMO. Am I missing something?

User avatar
Ozymandias
Posts: 962
Joined: Sun Oct 25, 2009 12:30 am

Re: LCZero update

Post by Ozymandias » Wed Mar 14, 2018 6:38 am

In that tournament, I see lczero_7f8f_playouts3300 with a performance of 532, and you say its Elo is approximately 2,000 points ahead of random play. That either means getting some negative Elo values, or some seriously inflated assumptions (probably tracing back all the way to human chess).

User avatar
Ozymandias
Posts: 962
Joined: Sun Oct 25, 2009 12:30 am

Re: LCZero update

Post by Ozymandias » Wed Mar 14, 2018 6:41 am

Jhoravi wrote:Hi. I explored some games on the given link http://162.217.248.187/user/GaryS But the blunderfeasted games are nowhere near 2000 elo IMO. Am I missing something?
He's not saying that they're at a 2,000 Elo level, they're 2,000 Elo points ahead of random play. Now the question would be, what's the Elo for a random player?

Milos
Posts: 3193
Joined: Wed Nov 25, 2009 12:47 am

Re: LCZero update

Post by Milos » Wed Mar 14, 2018 10:48 am

Ozymandias wrote:
Jhoravi wrote:Hi. I explored some games on the given link http://162.217.248.187/user/GaryS But the blunderfeasted games are nowhere near 2000 elo IMO. Am I missing something?
He's not saying that they're at a 2,000 Elo level, they're 2,000 Elo points ahead of random play. Now the question would be, what's the Elo for a random player?
Since random play is certainly not 5000 Elo weaker than SF, the only logical assumption is that initial LCZero was much weaker than random play.
Authors should have maybe thought of using it for suicide chess ;).

Uri Blass
Posts: 8221
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: LCZero update

Post by Uri Blass » Wed Mar 14, 2018 10:49 am

Ozymandias wrote:
Jhoravi wrote:Hi. I explored some games on the given link http://162.217.248.187/user/GaryS But the blunderfeasted games are nowhere near 2000 elo IMO. Am I missing something?
He's not saying that they're at a 2,000 Elo level, they're 2,000 Elo points ahead of random play. Now the question would be, what's the Elo for a random player?
202 elo based on CCRL 40/4
http://www.computerchess.org.uk/ccrl/40 ... t_all.html

Uri Blass
Posts: 8221
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: LCZero update

Post by Uri Blass » Wed Mar 14, 2018 10:59 am

From CCRL:

LaMoSca 0.10-Brutus RND 10.5-9.5(only one win for LaMoSca)
Ram 2.0-Brutus RND 20-0

Ram 2.0-LaMoSca 0.10 11.5-9.5(only 2 wins for Ram)


Without seeing the games I suspect that LaMoSca is one of the program that should be removed from CCRL because it artificially increase the rating of the random player.

Probably LaMoSca is relatively strong in the opening but later allow stalemates or repetitions and without seeing the games I suspect that in most games it get a winning advantage against Ram 2.0 but translate it to a draw.

CheckersGuy
Posts: 273
Joined: Wed Aug 24, 2016 7:49 pm

Re: LCZero update

Post by CheckersGuy » Wed Mar 14, 2018 11:14 am

Milos wrote:
Ozymandias wrote:
Jhoravi wrote:Hi. I explored some games on the given link http://162.217.248.187/user/GaryS But the blunderfeasted games are nowhere near 2000 elo IMO. Am I missing something?
He's not saying that they're at a 2,000 Elo level, they're 2,000 Elo points ahead of random play. Now the question would be, what's the Elo for a random player?
Since random play is certainly not 5000 Elo weaker than SF, the only logical assumption is that initial LCZero was much weaker than random play.
Authors should have maybe thought of using it for suicide chess ;).
Impossible for lcZero to be weaker than random play

koedem
Posts: 44
Joined: Fri Mar 18, 2016 9:45 pm

Re: LCZero update

Post by koedem » Wed Mar 14, 2018 11:31 am

Why wouldn't random play be 5000 Elo weaker than SF? If we assume random play at -1500 Elo and SF at 3500 Elo (both seem reasonable) we get to a difference of 5000. Seems logical to me.

User avatar
CMCanavessi
Posts: 735
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: LCZero update

Post by CMCanavessi » Wed Mar 14, 2018 11:48 am

Uri Blass wrote:From CCRL:

LaMoSca 0.10-Brutus RND 10.5-9.5(only one win for LaMoSca)
Ram 2.0-Brutus RND 20-0

Ram 2.0-LaMoSca 0.10 11.5-9.5(only 2 wins for Ram)


Without seeing the games I suspect that LaMoSca is one of the program that should be removed from CCRL because it artificially increase the rating of the random player.

Probably LaMoSca is relatively strong in the opening but later allow stalemates or repetitions and without seeing the games I suspect that in most games it get a winning advantage against Ram 2.0 but translate it to a draw.
I have ran LaMoSca in my tournaments A LOT and it has an issue, it can't win (or maybe it was programed that way deliberately). It will not promote pawns, they will all remain in the 7th rank forever (except very rare cases where they can take a piece in the 8th rank). Its "real" strenght is around 800-1000 elo, but after capturing every piece of the oponent, it will 3-fold. That's why it will have a lot of draws, and no wins.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls

Locked