Why does LC0 perfoms so poor here in CCC ?

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Chessqueen
Posts: 439
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Why does LC0 perfoms so poor here in CCC ?

Post by Chessqueen » Mon May 20, 2019 3:27 am

Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/

User avatar
Graham Banks
Posts: 32489
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: Why does LC0 perfoms so poor here in CCC ?

Post by Graham Banks » Mon May 20, 2019 3:37 am

Chessqueen wrote:
Mon May 20, 2019 3:27 am
Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/
Hardware is only benchmarked to the old Athlon.
Hardware we used ranges from the Q6600 and a dual Xeon octal to an AMD octal and modern generation i7s.

Ray is doing the GPU testing, but not with the latest expensive cards.
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

Modern Times
Posts: 2276
Joined: Thu Jun 07, 2012 9:02 pm

Re: Why does LC0 perfoms so poor here in CCC ?

Post by Modern Times » Mon May 20, 2019 3:42 am

http://ccrl.chessdom.com/ccrl/404/cgi/c ... librate=no

It is 7 Elo behind Stockfish on 1 CPU, you need to look at it in that context.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Chessqueen
Posts: 439
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Why does LC0 perfoms so poor here in CCC ?

Post by Chessqueen » Mon May 20, 2019 3:12 pm

Modern Times wrote:
Mon May 20, 2019 3:42 am
http://ccrl.chessdom.com/ccrl/404/cgi/c ... librate=no

It is 7 Elo behind Stockfish on 1 CPU, you need to look at it in that context.
No wonder why your GPU is a mere very low one compared to what TCEC is using, take a look and compare ==>
GPU (2): 1 x 2080 ti + 1 x 2080, CPU: Quad Core i5 3570k, RAM: 32GiB DDR3-2400, SSD: Samsung 840 Pro 256GB, Battery backup:
https://tcec.chessdom.com/

supersharp77
Posts: 626
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Re: Why does LC0 perfoms so poor in CCC? SHAM?

Post by supersharp77 » Mon May 20, 2019 5:02 pm

Chessqueen wrote:
Mon May 20, 2019 3:27 am
Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/
Excellent question..Thats the question MOST people in the Chess engine community are AFRAID to ask..In my opinion this whole Alpha Zero-Leela Zero-LC0 thing is a SCAM-SHAM-FRAUD-A Show for money and increased attention..TCEC said that "Viewership and Ratings are Way Up" (With this LCO phenomenom) Money talks..My contention which still stands is that Alpha Zero-LCO-Allie-Etc Are Not really Chess engines at all...Been testing LC0 for months at home and the LC0 results are 100% not repeatable...This engine is having trouble beating Comet, Bikjump, and others..Lost to Green Light Chess, Atlas Too Thinker 5.2 Destroyed by Toga II, Damirs positional Player, Stockfish, Thinksfish, Demolished by Houdini 3 & Komodo ...Basic Search without net help is primitive at best..Missed basic tactics..no to poor endgame play..etc..etc... Here are the tricks to why it functions well at TCEC...Huge Neural net -20 million plus...Superfast Processor Speed (to access the Neural Net games Database) by the time it gets out of 'BOOK' its probably at the drawn endgame or late middlegame stages...Is this really Chess? I think not...It's a SCANDAL..

TCEC Lc0 Configuration
Quote
Post by mwyoung » Fri May 17, 2019 6:52 pm

TCEC GPU Server
GPUs: 1 x 2080 ti + 1 x 2080
CPU: Quad Core i5 2600k
RAM: 16GB DDR3-2133
SSD:Samsung 840 Pro 256gb

I am watching the TCEC games with my Lc0. With the TCEC Lc0 running with an extra RTX 2080 over my setup. It is strange that my system is searching more plies always, and in some cases even more NPS.

I am running default setting except for using the fp-16 backend, and using NN = 9900000

Does anyone know what settings TCEC is using.

Either NN has a huge impact on the search depth, or using a extra GPU is not worth much, or TCEC is running a poor configuration for Lc0?
Professing themselves to be wise, they became fools,
Top
arunsoorya1309
Posts: 195
Joined: Tue Nov 03, 2015 3:55 pm
Re: TCEC Lc0 Configuration
Quote
Post by arunsoorya1309 » Fri May 17, 2019 7:13 pm

{
"name": "MoveOverheadMs",
"value": "2000"
},
{
"name": "WeightsFile",
"value": "xxxx/T40.T8.610.pb.gz"
},
{
"name": "Threads",
"value": "3"
},
{
"name": "NNCacheSize",
"value": "20000000" (20 Million games Neural Net) :D :wink:
},
{
"name": "MinibatchSize",
"value": "256"
},
{
"name": "MaxCollisionEvents",
"value": "32"
},
{
"name": "CPuct",
"value": "3.4"
},
{
"name": "CPuctBase",
"value": "10000"
},
{
"name": "MaxPrefetch",
"value": "32"
},
{
"name": "RamLimitMb",
"value": "22528"
},
{
"name": "LogFile",
"value": "Sufi.txt"
},
{
"name": "SyzygyPath",
"value": "xxx"
},
{
"name": "Ponder",
"value": "false"
},
{
"name": "CommandLineOptions",
"value": "--backend=roundrobin --backend-opts=\"(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)\""
}

Chessqueen
Posts: 439
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Why does LC0 perfoms so poor in CCC? SHAM?

Post by Chessqueen » Tue May 21, 2019 12:30 am

supersharp77 wrote:
Mon May 20, 2019 5:02 pm
Chessqueen wrote:
Mon May 20, 2019 3:27 am
Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/
Excellent question..Thats the question MOST people in the Chess engine community are AFRAID to ask..In my opinion this whole Alpha Zero-Leela Zero-LC0 thing is a SCAM-SHAM-FRAUD-A Show for money and increased attention..TCEC said that "Viewership and Ratings are Way Up" (With this LCO phenomenom) Money talks..My contention which still stands is that Alpha Zero-LCO-Allie-Etc Are Not really Chess engines at all...Been testing LC0 for months at home and the LC0 results are 100% not repeatable...This engine is having trouble beating Comet, Bikjump, and others..Lost to Green Light Chess, Atlas Too Thinker 5.2 Destroyed by Toga II, Damirs positional Player, Stockfish, Thinksfish, Demolished by Houdini 3 & Komodo ...Basic Search without net help is primitive at best..Missed basic tactics..no to poor endgame play..etc..etc... Here are the tricks to why it functions well at TCEC...Huge Neural net -20 million plus...Superfast Processor Speed (to access the Neural Net games Database) by the time it gets out of 'BOOK' its probably at the drawn endgame or late middlegame stages...Is this really Chess? I think not...It's a SCANDAL..

TCEC Lc0 Configuration
Quote
Post by mwyoung » Fri May 17, 2019 6:52 pm

TCEC GPU Server
GPUs: 1 x 2080 ti + 1 x 2080
CPU: Quad Core i5 2600k
RAM: 16GB DDR3-2133
SSD:Samsung 840 Pro 256gb

I am watching the TCEC games with my Lc0. With the TCEC Lc0 running with an extra RTX 2080 over my setup. It is strange that my system is searching more plies always, and in some cases even more NPS.

I am running default setting except for using the fp-16 backend, and using NN = 9900000

Does anyone know what settings TCEC is using.

Either NN has a huge impact on the search depth, or using a extra GPU is not worth much, or TCEC is running a poor configuration for Lc0?
Professing themselves to be wise, they became fools,
Top
arunsoorya1309
Posts: 195
Joined: Tue Nov 03, 2015 3:55 pm
Re: TCEC Lc0 Configuration
Quote
Post by arunsoorya1309 » Fri May 17, 2019 7:13 pm

{
"name": "MoveOverheadMs",
"value": "2000"
},
{
"name": "WeightsFile",
"value": "xxxx/T40.T8.610.pb.gz"
},
{
"name": "Threads",
"value": "3"
},
{
"name": "NNCacheSize",
"value": "20000000" (20 Million games Neural Net) :D :wink:
},
{
"name": "MinibatchSize",
"value": "256"
},
{
"name": "MaxCollisionEvents",
"value": "32"
},
{
"name": "CPuct",
"value": "3.4"
},
{
"name": "CPuctBase",
"value": "10000"
},
{
"name": "MaxPrefetch",
"value": "32"
},
{
"name": "RamLimitMb",
"value": "22528"
},
{
"name": "LogFile",
"value": "Sufi.txt"
},
{
"name": "SyzygyPath",
"value": "xxx"
},
{
"name": "Ponder",
"value": "false"
},
{
"name": "CommandLineOptions",
"value": "--backend=roundrobin --backend-opts=\"(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)\""
}


Your main poor performance is due to using an extra GPU keep it the same as TCEC and by you using an inferior NET ==> I am running default setting except for using the fp-16 backend, and using NN = 9900000 you should use the same network as TCEC

Chessqueen
Posts: 439
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Why does LC0 perfoms so poor here in CCC ?

Post by Chessqueen » Wed May 22, 2019 6:23 pm

Modern Times wrote:
Mon May 20, 2019 3:42 am
http://ccrl.chessdom.com/ccrl/404/cgi/c ... librate=no

It is 7 Elo behind Stockfish on 1 CPU, you need to look at it in that context.
So Stockfish lost the match against LCo 48.5 to 51.5 or I am reading this wrong ==> http://ccrl.chessdom.com/ccrl/404/cgi/e ... _10_64-bit

https://tcec.chessdom.com/live.html

Modern Times
Posts: 2276
Joined: Thu Jun 07, 2012 9:02 pm

Re: Why does LC0 perfoms so poor here in CCC ?

Post by Modern Times » Wed May 22, 2019 8:44 pm

That is correct, it lost 48.5 to 51.5 to Lc0 0.21.1 JH.T6.532 GPU. Of course the statistical margin of error with a single 100 game match is high. Run the matches again, and you may get a slightly different result.

On this ratings list, Lc0 seems to under-perform against weaker opponents. So although it did beat Stockfish 10 head-to-head, it has a slightly lower rating than Stockfish.

It won't be long before Lc0 overtakes Stockfish 10 1CPU on this list. I'm waiting until I'm pretty sure they have +20 Elo gains over JH.T6.532 before testing it again.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Chessqueen
Posts: 439
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Why does LC0 perfoms so poor here in CCC ?

Post by Chessqueen » Thu May 23, 2019 4:13 pm

Modern Times wrote:
Wed May 22, 2019 8:44 pm
That is correct, it lost 48.5 to 51.5 to Lc0 0.21.1 JH.T6.532 GPU. Of course the statistical margin of error with a single 100 game match is high. Run the matches again, and you may get a slightly different result.

On this ratings list, Lc0 seems to under-perform against weaker opponents. So although it did beat Stockfish 10 head-to-head, it has a slightly lower rating than Stockfish.

It won't be long before Lc0 overtakes Stockfish 10 1CPU on this list. I'm waiting until I'm pretty sure they have +20 Elo gains over JH.T6.532 before testing it again.
If I had ten fights against anybody and got my ass whip, do you think that I would like to fight a few more times, Simply ask Stockfish if he wants Leela to whip him a few more times. I guess Stockfish will tell you Hell NO, and now that Leela is getting hotter and hotter I am afraid that with all that Heat coming out of her GPU, she is preparing to have fish and I do NOT want to be her lunch. :mrgreen:

Chessqueen
Posts: 439
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Why does LC0 perfoms so poor here in CCC ?

Post by Chessqueen » Thu May 23, 2019 7:48 pm

Chessqueen wrote:
Thu May 23, 2019 4:13 pm
Modern Times wrote:
Wed May 22, 2019 8:44 pm
That is correct, it lost 48.5 to 51.5 to Lc0 0.21.1 JH.T6.532 GPU. Of course the statistical margin of error with a single 100 game match is high. Run the matches again, and you may get a slightly different result.

On this ratings list, Lc0 seems to under-perform against weaker opponents. So although it did beat Stockfish 10 head-to-head, it has a slightly lower rating than Stockfish.

It won't be long before Lc0 overtakes Stockfish 10 1CPU on this list. I'm waiting until I'm pretty sure they have +20 Elo gains over JH.T6.532 before testing it again.
If I had ten fights against anybody and got my ass whip, do you think that I would like to fight a few more times, Simply ask Stockfish if he wants Leela to whip him a few more times. I guess Stockfish will tell you Hell NO, and now that Leela is getting hotter and hotter I am afraid that with all that Heat coming out of her GPU, she is preparing to have fish and I do NOT want to be her lunch. :mrgreen:

And that is with a very low end GPU graphic card, imagine what LCO could have done with this one ==> https://www.newegg.com/msi-geforce-rtx- ... &gclsrc=ds

And in 3 months LC0 will be using a TPU ==> https://www.nextplatform.com/2017/04/05 ... hitecture/
https://tcec.chessdom.com/live.html

Post Reply