Lc0 question

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
mclane
Posts: 16267
Joined: Thu Mar 09, 2006 5:40 pm
Location: US of Europe, germany
Full name: Thorsten Czub
Contact:

Re: Lc0 question

Post by mclane » Sun Jul 07, 2019 6:56 am

Stockfish plays machine chess, although developed by humans.
LC0 plays human chess, although developed by a machine.


The philosophical impact of this cannot be underestimated.
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....

chrisw
Posts: 2190
Joined: Tue Apr 03, 2012 2:28 pm

Re: Lc0 question

Post by chrisw » Sun Jul 07, 2019 8:25 am

mclane wrote:
Sun Jul 07, 2019 6:56 am
Stockfish plays machine chess, although developed by humans.
LC0 plays human chess, although developed by a machine.


The philosophical impact of this cannot be underestimated.
Dispute that LCZero plays human chess. It plays chess in a way that humans would like to imagine they play chess.

User avatar
mclane
Posts: 16267
Joined: Thu Mar 09, 2006 5:40 pm
Location: US of Europe, germany
Full name: Thorsten Czub
Contact:

Re: Lc0 question

Post by mclane » Sun Jul 07, 2019 8:51 am

maybe.

when i put lc0 on my slowest tablet to make 1-15 nps,
and let it run against my modern dedicated chess computers (300 mhz arm cpu with Johan de Konings THE KING engine), it plays like a human would do it. nice games. sometimes saccing a piece for a plan. or like we dream we would play.

50 years of computerchess development in a trash bin because we suddenly find out that the development in computerchess was running in the wrong direction.

isnt that a little frustrating to see computerchess development in this stage ?
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....

dkappe
Posts: 273
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lc0 question

Post by dkappe » Sun Jul 07, 2019 10:44 am

mclane wrote:
Sun Jul 07, 2019 8:51 am
maybe.

when i put lc0 on my slowest tablet to make 1-15 nps,
and let it run against my modern dedicated chess computers (300 mhz arm cpu with Johan de Konings THE KING engine), it plays like a human would do it. nice games. sometimes saccing a piece for a plan. or like we dream we would play.

50 years of computerchess development in a trash bin because we suddenly find out that the development in computerchess was running in the wrong direction.

isnt that a little frustrating to see computerchess development in this stage ?
First i’d suggest using some of the smaller nets on your tablet.

Second, the first chapter on hybrid engine has barely started.

Last, I’m able to train nets using ab engines that are stronger tactically than those trained via self-play. A few chapters there as well.

USGroup1
Posts: 33
Joined: Sun Oct 14, 2018 5:01 pm
Full name: Sina Vaziri

Re: Lc0 question

Post by USGroup1 » Sun Jul 07, 2019 11:23 am

lkaufman wrote:
Sun Jul 07, 2019 3:17 am
... For analysis, this makes Lc0 much more useful, as you never know when to stop analyzing with SF, and no two people will reach the same conclusions.
On the other hand Lc0 is close to useless for analysis because it doesn't give 0.00 evaluation on drawn positions, not even tb draw positions.

User avatar
kranium
Posts: 1824
Joined: Thu May 29, 2008 8:43 am

Re: Lc0 question

Post by kranium » Sun Jul 07, 2019 12:32 pm

mclane wrote:
Sun Jul 07, 2019 6:56 am
Stockfish plays machine chess, although developed by humans.
LC0 plays human chess, although developed by a machine.

The philosophical impact of this cannot be underestimated.
mclane wrote:
Sun Jul 07, 2019 8:51 am
50 years of computerchess development in a trash bin because we suddenly find out that the development in computerchess was running in the wrong direction.

isnt that a little frustrating to see computerchess development in this stage ?
I'm really surprised to hear you say that...
because IMO everything that has happened in computer chess the last 50 years has led to this point.

This reminds me of the topics proclaiming 'end an era' (for AB engines)...
which seems premature considering Stockfish is still stronger than LC0.

Here's the result with a Leela ratio of close to 1:1 after many games
(http://ccrl.chessdom.com/ccrl/404/)

Code: Select all

1 Stockfish 10 64-bit 4CPU	3546
2 Houdini 6 64-bit 4CPU		3519
3 Komodo 11.2 64-bit 4CPU	3503
4 Lc0 0.21.1 JH.T6.532 GPU	3487
Here's the current score of a chess.com tourney with LC0 running on a powerful server with 4 state-of-the-art GPUs (4x RTX 2080ti 44 GB GPU memory)
costing $1200.00 each
(ccc-9-the-gauntlet-semifinals)

Code: Select all

1 Stockfish (3679) 	109.0/192
2 Leelenstein (3646) 	108.0/192
3 Lc0 (3656) 		107.5/19
Don't forget: strength has been THE critical consideration when evaluating engines for a very long time now
Perceptions concerning whether or not the engine plays 'human-like' or not are just plain silly. These engines are playing at a level none of us can understand, and human-like quality of play can't be measured or quantified...except via Elo, which would mean any engine stronger than Carlsen is not playing human-like...including LC0.
Last edited by kranium on Sun Jul 07, 2019 12:51 pm, edited 3 times in total.

User avatar
kranium
Posts: 1824
Joined: Thu May 29, 2008 8:43 am

Re: Lc0 question

Post by kranium » Sun Jul 07, 2019 12:46 pm

chrisw wrote:
Sun Jul 07, 2019 8:25 am
mclane wrote:
Sun Jul 07, 2019 6:56 am
Stockfish plays machine chess, although developed by humans.
LC0 plays human chess, although developed by a machine.


The philosophical impact of this cannot be underestimated.
Dispute that LCZero plays human chess. It plays chess in a way that humans would like to imagine they play chess.
+1

Modern Times
Posts: 2420
Joined: Thu Jun 07, 2012 9:02 pm

Re: Lc0 question

Post by Modern Times » Sun Jul 07, 2019 1:28 pm

kranium wrote:
Sun Jul 07, 2019 12:32 pm
This reminds me of the topics proclaiming 'end an era' (for AB engines)...
which seems premature considering Stockfish is still stronger than LC0.

Here's the result with a Leela ratio of close to 1:1 after many games
(http://ccrl.chessdom.com/ccrl/404/)

Code: Select all

1 Stockfish 10 64-bit 4CPU	3546
2 Houdini 6 64-bit 4CPU		3519
3 Komodo 11.2 64-bit 4CPU	3503
4 Lc0 0.21.1 JH.T6.532 GPU	3487
Not quite - the Leela ratio is probably around 0.4 when playing a 4CPU opponent. The LCo results on the GTX1050 can only be compared to 1CPU engines:

Code: Select all

CCRL 40/4 Rating List - Single-CPU engines, best versions only
2062148 games played by 2380 programs, run by 21 testers
Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB
Time control: Equivalent to 40 moves in 4 minutes on Athlon 64 X2 4600+ (2.4 GHz), about 1.5 minutes on a modern Intel CPU.
Computed on July 6, 2019 with Bayeselo based on 2'062'148 games
Tested by CCRL team, 2005-2019, http://computerchess.org.uk/ccrl/404/

   Rank               Engine                Elo   +    -   Score  AvOp  Games
      1 Stockfish 10 64-bit                3495  +13  -13  75.1% -177.7  2127
      2 Lc0 0.21.1 JH.T6.532 GPU           3487  +17  -17  59.2%  -58.6  1100
      3 Houdini 6 64-bit                   3447   +8   -8  69.6% -144.6  7417
      4 Allie 0.5 nn42482 GPU              3442  +21  -20  53.5%  -22.5   708
      5 Komodo 11.2 64-bit                 3422   +9   -9  68.2% -136.8  4979
      6 Komodo 13.01 MCTS 64-bit           3359  +17  -17  65.9% -110.3  1201
      
So roughly equal given error margins.

When one of our testers comes along with a stronger GPU, then there will be something to compare to 4CPU.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

User avatar
Laskos
Posts: 9509
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Lc0 question

Post by Laskos » Sun Jul 07, 2019 1:36 pm

kranium wrote:
Sun Jul 07, 2019 12:32 pm
mclane wrote:
Sun Jul 07, 2019 6:56 am
Stockfish plays machine chess, although developed by humans.
LC0 plays human chess, although developed by a machine.

The philosophical impact of this cannot be underestimated.
mclane wrote:
Sun Jul 07, 2019 8:51 am
50 years of computerchess development in a trash bin because we suddenly find out that the development in computerchess was running in the wrong direction.

isnt that a little frustrating to see computerchess development in this stage ?
I'm really surprised to hear you say that...
because IMO everything that has happened in computer chess the last 50 years has led to this point.

This reminds me of the topics proclaiming 'end an era' (for AB engines)...
which seems premature considering Stockfish is still stronger than LC0.

Here's the result with a Leela ratio of close to 1:1 after many games
(http://ccrl.chessdom.com/ccrl/404/)

Code: Select all

1 Stockfish 10 64-bit 4CPU	3546
2 Houdini 6 64-bit 4CPU		3519
3 Komodo 11.2 64-bit 4CPU	3503
4 Lc0 0.21.1 JH.T6.532 GPU	3487
Here's the current score of a chess.com tourney with LC0 running on a powerful server with 4 state-of-the-art GPUs (4x RTX 2080ti 44 GB GPU memory)
costing $1200.00 each
(ccc-9-the-gauntlet-semifinals)

Code: Select all

1 Stockfish (3679) 	109.0/192
2 Leelenstein (3646) 	108.0/192
3 Lc0 (3656) 		107.5/19
Don't forget: strength has been THE critical consideration when evaluating engines for a very long time now
Perceptions concerning whether or not the engine plays 'human-like' or not are just plain silly. These engines are playing at a level none of us can understand, and human-like quality of play can't be measured or quantified...except via Elo, which would mean any engine stronger than Carlsen is not playing human-like...including LC0.
An inaccuracy about CCRL rating: Leela ratio is a about 1 against 1-core CPU engines, not 4-core.

Second issue: Leela will underperform in rating lists with many weaker opponents compared to a head to head match against the top engine, SF in both lists.

Nay Lin Tun
Posts: 530
Joined: Mon Jan 16, 2012 5:34 am

Re: Lc0 question

Post by Nay Lin Tun » Sun Jul 07, 2019 1:50 pm

kranium wrote:
Sun Jul 07, 2019 12:32 pm
mclane wrote:
Sun Jul 07, 2019 6:56 am
Stockfish plays machine chess, although developed by humans.
LC0 plays human chess, although developed by a machine.

The philosophical impact of this cannot be underestimated.
mclane wrote:
Sun Jul 07, 2019 8:51 am
50 years of computerchess development in a trash bin because we suddenly find out that the development in computerchess was running in the wrong direction.

isnt that a little frustrating to see computerchess development in this stage ?
I'm really surprised to hear you say that...
because IMO everything that has happened in computer chess the last 50 years has led to this point.

This reminds me of the topics proclaiming 'end an era' (for AB engines)...
which seems premature considering Stockfish is still stronger than LC0.

Here's the result with a Leela ratio of close to 1:1 after many games
(http://ccrl.chessdom.com/ccrl/404/)

Code: Select all

1 Stockfish 10 64-bit 4CPU	3546
2 Houdini 6 64-bit 4CPU		3519
3 Komodo 11.2 64-bit 4CPU	3503
4 Lc0 0.21.1 JH.T6.532 GPU	3487
Here's the current score of a chess.com tourney with LC0 running on a powerful server with 4 state-of-the-art GPUs (4x RTX 2080ti 44 GB GPU memory)
costing $1200.00 each
(ccc-9-the-gauntlet-semifinals)

Code: Select all

1 Stockfish (3679) 	109.0/192
2 Leelenstein (3646) 	108.0/192
3 Lc0 (3656) 		107.5/19
Don't forget: strength has been THE critical consideration when evaluating engines for a very long time now
Perceptions concerning whether or not the engine plays 'human-like' or not are just plain silly. These engines are playing at a level none of us can understand, and human-like quality of play can't be measured or quantified...except via Elo, which would mean any engine stronger than Carlsen is not playing human-like...including LC0.
Your two examples have some questionable technical problems.
1. CCRL net was using 20x256 which is suitable for stronger cards only. ( I saw in discord chat that they sent 10x128 net for CCRL hardware. Once they test that net , the result may change)
2. In your cccc, although you use 4x GPU, Leela cant use those 4x gpu and you get nothing or 5% speed boost at most from 2x to 4x GPU. (It is like I upgrade 16GB memory to 32 GB in my system but the system cannot use it , and it is nothing)
If you test half of your cores 45 cores for AB engines and 2x GPU, then Lc0 will perform much better.

Post Reply