LCZero: Progress and Scaling. Relation to CCRL Elo

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
nabildanial
Posts: 104
Joined: Thu Jun 05, 2014 3:29 am
Location: Malaysia

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by nabildanial » Thu Apr 26, 2018 9:23 pm

JJJ wrote:I m not sure the FIDE estimate is accurate either, since no human could defeat recent version of LCzero, so probably none can defeat the ID 190 version.
The explanation on FIDE estimate in the Google Sheet makes perfect sense to me.

"The FIDE estimate indicates the strength of Leela playing at 1'+1'' time control, against a human that's playing at Classical time control. All estimates should be further adjusted based on the Elo scaling on the Scaling tab. For example, ID 125v7 (~2441 FIDE) would scale ~300 Elo with Rapid TC on this machine. Meaning at Rapid TC, it would play like a human player with a Classical Rating of ~2741 FIDE playing at Classical TC"

duncan
Posts: 9984
Joined: Mon Jul 07, 2008 8:50 pm

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by duncan » Thu Apr 26, 2018 10:57 pm

so elo 2552 on april 14th

now april 26 it is elo 2803. so a 251 elo gain in 12 days and 21 points a day gain. (including regression). To catch stockfish at 3563, another 760 points needed which should take about 5 weeks unless it slows down. so beginning of June. just maybe.

JJJ
Posts: 1285
Joined: Sat Apr 19, 2014 11:47 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by JJJ » Thu Apr 26, 2018 11:43 pm

nabildanial wrote:
JJJ wrote:I m not sure the FIDE estimate is accurate either, since no human could defeat recent version of LCzero, so probably none can defeat the ID 190 version.
The explanation on FIDE estimate in the Google Sheet makes perfect sense to me.

"The FIDE estimate indicates the strength of Leela playing at 1'+1'' time control, against a human that's playing at Classical time control. All estimates should be further adjusted based on the Elo scaling on the Scaling tab. For example, ID 125v7 (~2441 FIDE) would scale ~300 Elo with Rapid TC on this machine. Meaning at Rapid TC, it would play like a human player with a Classical Rating of ~2741 FIDE playing at Classical TC"
So that's mean with just more time control than bullet, Leela is now stronger than Carslen playing with 90 min.

User avatar
Laskos
Posts: 9313
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Laskos » Fri Apr 27, 2018 7:56 pm

One interesting aspect about the openings. I took Komodo 11.3.1 on 4 threads, which seems to perform the best in opening test suites, and maybe has the best opening phase of all standard engines, and pitted it against LC0 CPU ID201 on 4 threads (on a strong GPU it would be some 5 times faster) in three LTC games at 3600''+ 36''. In all three games the same happened: LC0 (as White) held its own until some tactics started to be involved, and not only that, it slowly gained some slight advantage in this initial quiet phase.

An example of an interrupted game, after LC0 started to lose clearly (all games were lost, obviously), Sicilian Steinitz variation:

[pgn][Event "My Tournament"]
[Site "?"]
[Date "2018.04.27"]
[Round "1"]
[White "LC0_CPU"]
[Black "Komodo 11.3.1"]
[Result "*"]
[ECO "B20"]
[Opening "Sicilian"]
[PlyCount "56"]
[Termination "unterminated"]
[TimeControl "3600+36"]
[Variation "Steinitz Variation"]

1. e4 {+0.13/23 63s} c5 {-0.15/29 184s} 2. g3 {+0.14/24 103s} Nc6 {0.00/29 106s}
3. Nf3 {+0.13/24 77s} Nf6 {-0.04/30 107s} 4. d3 {+0.05/24 91s}
d6 {-0.02/32 171s} 5. Bg2 {+0.14/24 103s} g6 {0.00/31 90s}
6. O-O {+0.11/25 107s} Bg7 {0.00/31 104s} 7. Re1 {+0.11/25 96s}
O-O {-0.04/31 133s} 8. c3 {+0.11/25 59s} Rb8 {0.00/29 175s} 9. a4 {+0.15/24 83s}
Be6 {0.00/30 105s} 10. d4 {+0.38/24 92s} cxd4 {-0.05/32 200s}
11. Nxd4 {+0.36/24 54s} Bd7 {0.00/28 100s} 12. Qe2 {+0.39/24 100s}
Qc8 {-0.02/30 217s} 13. Nxc6 {+0.52/24 74s} bxc6 {+0.12/30 109s}
14. a5 {+0.46/25 109s} Bg4 {+0.18/31 183s} 15. f3 {+0.52/23 67s}
Be6 {+0.11/31 118s} 16. a6 {+0.51/24 104s} Qc7 {+0.14/30 245s}
17. h3 {+0.68/24 101s} Nd7 {+0.32/30 208s} 18. Kh2 {+0.55/25 112s}
c5 {+0.42/31 199s} 19. f4 {+0.43/25 121s} c4 {+0.35/31 84s}
20. Be3 {+0.31/25 81s} Nc5 {+0.62/30 84s} 21. Na3 {+0.33/25 51s}
Nd3 {+1.32/30 104s} 22. Reb1 {+0.21/24 55s} Rb3 {+1.31/30 82s}
23. Nc2 {-0.46/25 123s} Rfb8 {+1.63/32 118s} 24. Nd4 {-0.20/24 62s}
Rxb2 {+1.61/30 78s} 25. Nxe6 {-0.19/24 54s} fxe6 {+1.54/31 105s}
26. Rxb2 {-0.14/24 1.0s} Rxb2 {+2.01/31 106s} 27. Qg4 {-0.14/24 1.0s}
Kf7 {+2.39/28 90s} 28. f5 {-0.52/24 72s} exf5 {+3.59/29 63s, No result} *[/pgn]

The thing is:
LC0 move 9.a2-a4

[D]1rbq1rk1/pp2ppbp/2np1np1/2p5/P3P3/2PP1NP1/1P3PBP/RNBQR1K1 b - - 0 9

Looking in databases of human games:

Chessbase Mega Database 2017:
White performance: 58.1% in 43 games

Chess Tempo Database:
White performance: 59.6% in 26 games

At this point is was Komodo exiting human theory with 9...Bc8-e6, not found in my large databases. Even SF9 sees some advantage for White (LC0) here, although standard engine analysis might be pretty worthless in this case.

In all three games, it was Komodo exiting the theory with some disadvantage, then, soon after, tactical complications completely demolished LC0.

Albert Silver
Posts: 2829
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Albert Silver » Fri Apr 27, 2018 8:51 pm

It must really depend on the hardware. On my laptop, which according to the benchmark reaches 2031 NPS, it is about 2700+ CCRL. I just finished a 100-game match at g/10 against Nebula 2.0 (2726 CCRL) and Leela NN189 scored 50.5-49.5.

The laptop's hardware is i7-6700HQ (4 x 2.6GHz with HT) and GTX 980M (8GB version).
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

User avatar
George Tsavdaris
Posts: 1559
Joined: Thu Mar 09, 2006 11:35 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by George Tsavdaris » Fri Apr 27, 2018 9:46 pm

Laskos wrote:One interesting aspect about the openings. I took Komodo 11.3.1 on 4 threads, which seems to perform the best in opening test suites, and maybe has the best opening phase of all standard engines, and pitted it against LC0 CPU ID201 on 4 threads (on a strong GPU it would be some 5 times faster) in three LTC games at 3600''+ 36''.
Better not to run LC0 with increment time controls as i think it doesn't support them. I think.
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....

User avatar
Laskos
Posts: 9313
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Laskos » Fri Apr 27, 2018 10:15 pm

George Tsavdaris wrote:
Laskos wrote:One interesting aspect about the openings. I took Komodo 11.3.1 on 4 threads, which seems to perform the best in opening test suites, and maybe has the best opening phase of all standard engines, and pitted it against LC0 CPU ID201 on 4 threads (on a strong GPU it would be some 5 times faster) in three LTC games at 3600''+ 36''.
Better not to run LC0 with increment time controls as i think it doesn't support them. I think.
It does support them.

User avatar
Laskos
Posts: 9313
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Laskos » Sat Apr 28, 2018 8:18 am

The progress up to ID205:

Image

Werewolf
Posts: 1187
Joined: Thu Sep 18, 2008 8:24 pm

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Werewolf » Sat Apr 28, 2018 9:35 am

Very nice.

JJJ
Posts: 1285
Joined: Sat Apr 19, 2014 11:47 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by JJJ » Sat Apr 28, 2018 4:05 pm

It is normal LCzero outplay engines at the opening sometimes, and lack of tactics then. In the long run it will overcome the lack of tactics as well, it just needs many many many more games to try some sacrifice or some tactics with his actual level of play.

Post Reply