Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/
Why does LC0 perfoms so poor here in CCC ?
Moderators: hgm, Rebel, chrisw
-
- Posts: 5582
- Joined: Wed Sep 05, 2018 2:16 am
- Location: Moving
- Full name: Jorge Picado
Why does LC0 perfoms so poor here in CCC ?
Do NOT worry and be happy, we all live a short life
-
- Posts: 41423
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
Re: Why does LC0 perfoms so poor here in CCC ?
Hardware is only benchmarked to the old Athlon.Chessqueen wrote: ↑Mon May 20, 2019 5:27 am Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/
Hardware we used ranges from the Q6600 and a dual Xeon octal to an AMD octal and modern generation i7s.
Ray is doing the GPU testing, but not with the latest expensive cards.
gbanksnz at gmail.com
-
- Posts: 3546
- Joined: Thu Jun 07, 2012 11:02 pm
Re: Why does LC0 perfoms so poor here in CCC ?
http://ccrl.chessdom.com/ccrl/404/cgi/c ... librate=no
It is 7 Elo behind Stockfish on 1 CPU, you need to look at it in that context.
It is 7 Elo behind Stockfish on 1 CPU, you need to look at it in that context.
-
- Posts: 5582
- Joined: Wed Sep 05, 2018 2:16 am
- Location: Moving
- Full name: Jorge Picado
Re: Why does LC0 perfoms so poor here in CCC ?
No wonder why your GPU is a mere very low one compared to what TCEC is using, take a look and compare ==>Modern Times wrote: ↑Mon May 20, 2019 5:42 am http://ccrl.chessdom.com/ccrl/404/cgi/c ... librate=no
It is 7 Elo behind Stockfish on 1 CPU, you need to look at it in that context.
GPU (2): 1 x 2080 ti + 1 x 2080, CPU: Quad Core i5 3570k, RAM: 32GiB DDR3-2400, SSD: Samsung 840 Pro 256GB, Battery backup:
https://tcec.chessdom.com/
Do NOT worry and be happy, we all live a short life
-
- Posts: 1242
- Joined: Sat Jul 05, 2014 7:54 am
- Location: Southwest USA
Re: Why does LC0 perfoms so poor in CCC? SHAM?
Excellent question..Thats the question MOST people in the Chess engine community are AFRAID to ask..In my opinion this whole Alpha Zero-Leela Zero-LC0 thing is a SCAM-SHAM-FRAUD-A Show for money and increased attention..TCEC said that "Viewership and Ratings are Way Up" (With this LCO phenomenom) Money talks..My contention which still stands is that Alpha Zero-LCO-Allie-Etc Are Not really Chess engines at all...Been testing LC0 for months at home and the LC0 results are 100% not repeatable...This engine is having trouble beating Comet, Bikjump, and others..Lost to Green Light Chess, Atlas Too Thinker 5.2 Destroyed by Toga II, Damirs positional Player, Stockfish, Thinksfish, Demolished by Houdini 3 & Komodo ...Basic Search without net help is primitive at best..Missed basic tactics..no to poor endgame play..etc..etc... Here are the tricks to why it functions well at TCEC...Huge Neural net -20 million plus...Superfast Processor Speed (to access the Neural Net games Database) by the time it gets out of 'BOOK' its probably at the drawn endgame or late middlegame stages...Is this really Chess? I think not...It's a SCANDAL..Chessqueen wrote: ↑Mon May 20, 2019 5:27 am Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/
TCEC Lc0 Configuration
Quote
Post by mwyoung » Fri May 17, 2019 6:52 pm
TCEC GPU Server
GPUs: 1 x 2080 ti + 1 x 2080
CPU: Quad Core i5 2600k
RAM: 16GB DDR3-2133
SSD:Samsung 840 Pro 256gb
I am watching the TCEC games with my Lc0. With the TCEC Lc0 running with an extra RTX 2080 over my setup. It is strange that my system is searching more plies always, and in some cases even more NPS.
I am running default setting except for using the fp-16 backend, and using NN = 9900000
Does anyone know what settings TCEC is using.
Either NN has a huge impact on the search depth, or using a extra GPU is not worth much, or TCEC is running a poor configuration for Lc0?
Professing themselves to be wise, they became fools,
Top
arunsoorya1309
Posts: 195
Joined: Tue Nov 03, 2015 3:55 pm
Re: TCEC Lc0 Configuration
Quote
Post by arunsoorya1309 » Fri May 17, 2019 7:13 pm
{
"name": "MoveOverheadMs",
"value": "2000"
},
{
"name": "WeightsFile",
"value": "xxxx/T40.T8.610.pb.gz"
},
{
"name": "Threads",
"value": "3"
},
{
"name": "NNCacheSize",
"value": "20000000" (20 Million games Neural Net)
},
{
"name": "MinibatchSize",
"value": "256"
},
{
"name": "MaxCollisionEvents",
"value": "32"
},
{
"name": "CPuct",
"value": "3.4"
},
{
"name": "CPuctBase",
"value": "10000"
},
{
"name": "MaxPrefetch",
"value": "32"
},
{
"name": "RamLimitMb",
"value": "22528"
},
{
"name": "LogFile",
"value": "Sufi.txt"
},
{
"name": "SyzygyPath",
"value": "xxx"
},
{
"name": "Ponder",
"value": "false"
},
{
"name": "CommandLineOptions",
"value": "--backend=roundrobin --backend-opts=\"(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)\""
}
-
- Posts: 5582
- Joined: Wed Sep 05, 2018 2:16 am
- Location: Moving
- Full name: Jorge Picado
Re: Why does LC0 perfoms so poor in CCC? SHAM?
supersharp77 wrote: ↑Mon May 20, 2019 7:02 pmExcellent question..Thats the question MOST people in the Chess engine community are AFRAID to ask..In my opinion this whole Alpha Zero-Leela Zero-LC0 thing is a SCAM-SHAM-FRAUD-A Show for money and increased attention..TCEC said that "Viewership and Ratings are Way Up" (With this LCO phenomenom) Money talks..My contention which still stands is that Alpha Zero-LCO-Allie-Etc Are Not really Chess engines at all...Been testing LC0 for months at home and the LC0 results are 100% not repeatable...This engine is having trouble beating Comet, Bikjump, and others..Lost to Green Light Chess, Atlas Too Thinker 5.2 Destroyed by Toga II, Damirs positional Player, Stockfish, Thinksfish, Demolished by Houdini 3 & Komodo ...Basic Search without net help is primitive at best..Missed basic tactics..no to poor endgame play..etc..etc... Here are the tricks to why it functions well at TCEC...Huge Neural net -20 million plus...Superfast Processor Speed (to access the Neural Net games Database) by the time it gets out of 'BOOK' its probably at the drawn endgame or late middlegame stages...Is this really Chess? I think not...It's a SCANDAL..Chessqueen wrote: ↑Mon May 20, 2019 5:27 am Why does LC0 perfomed so bad here in CCCR rating and NOT in TCEC, is it because of the old hardware that CCC uses here compared to the newer and more powerful GPUs in TCEC ?
https://tcec.chessdom.com/
TCEC Lc0 Configuration
Quote
Post by mwyoung » Fri May 17, 2019 6:52 pm
TCEC GPU Server
GPUs: 1 x 2080 ti + 1 x 2080
CPU: Quad Core i5 2600k
RAM: 16GB DDR3-2133
SSD:Samsung 840 Pro 256gb
I am watching the TCEC games with my Lc0. With the TCEC Lc0 running with an extra RTX 2080 over my setup. It is strange that my system is searching more plies always, and in some cases even more NPS.
I am running default setting except for using the fp-16 backend, and using NN = 9900000
Does anyone know what settings TCEC is using.
Either NN has a huge impact on the search depth, or using a extra GPU is not worth much, or TCEC is running a poor configuration for Lc0?
Professing themselves to be wise, they became fools,
Top
arunsoorya1309
Posts: 195
Joined: Tue Nov 03, 2015 3:55 pm
Re: TCEC Lc0 Configuration
Quote
Post by arunsoorya1309 » Fri May 17, 2019 7:13 pm
{
"name": "MoveOverheadMs",
"value": "2000"
},
{
"name": "WeightsFile",
"value": "xxxx/T40.T8.610.pb.gz"
},
{
"name": "Threads",
"value": "3"
},
{
"name": "NNCacheSize",
"value": "20000000" (20 Million games Neural Net)
},
{
"name": "MinibatchSize",
"value": "256"
},
{
"name": "MaxCollisionEvents",
"value": "32"
},
{
"name": "CPuct",
"value": "3.4"
},
{
"name": "CPuctBase",
"value": "10000"
},
{
"name": "MaxPrefetch",
"value": "32"
},
{
"name": "RamLimitMb",
"value": "22528"
},
{
"name": "LogFile",
"value": "Sufi.txt"
},
{
"name": "SyzygyPath",
"value": "xxx"
},
{
"name": "Ponder",
"value": "false"
},
{
"name": "CommandLineOptions",
"value": "--backend=roundrobin --backend-opts=\"(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)\""
}
Your main poor performance is due to using an extra GPU keep it the same as TCEC and by you using an inferior NET ==> I am running default setting except for using the fp-16 backend, and using NN = 9900000 you should use the same network as TCEC
Do NOT worry and be happy, we all live a short life
-
- Posts: 5582
- Joined: Wed Sep 05, 2018 2:16 am
- Location: Moving
- Full name: Jorge Picado
Re: Why does LC0 perfoms so poor here in CCC ?
So Stockfish lost the match against LCo 48.5 to 51.5 or I am reading this wrong ==> http://ccrl.chessdom.com/ccrl/404/cgi/e ... _10_64-bitModern Times wrote: ↑Mon May 20, 2019 5:42 am http://ccrl.chessdom.com/ccrl/404/cgi/c ... librate=no
It is 7 Elo behind Stockfish on 1 CPU, you need to look at it in that context.
https://tcec.chessdom.com/live.html
Do NOT worry and be happy, we all live a short life
-
- Posts: 3546
- Joined: Thu Jun 07, 2012 11:02 pm
Re: Why does LC0 perfoms so poor here in CCC ?
That is correct, it lost 48.5 to 51.5 to Lc0 0.21.1 JH.T6.532 GPU. Of course the statistical margin of error with a single 100 game match is high. Run the matches again, and you may get a slightly different result.
On this ratings list, Lc0 seems to under-perform against weaker opponents. So although it did beat Stockfish 10 head-to-head, it has a slightly lower rating than Stockfish.
It won't be long before Lc0 overtakes Stockfish 10 1CPU on this list. I'm waiting until I'm pretty sure they have +20 Elo gains over JH.T6.532 before testing it again.
On this ratings list, Lc0 seems to under-perform against weaker opponents. So although it did beat Stockfish 10 head-to-head, it has a slightly lower rating than Stockfish.
It won't be long before Lc0 overtakes Stockfish 10 1CPU on this list. I'm waiting until I'm pretty sure they have +20 Elo gains over JH.T6.532 before testing it again.
-
- Posts: 5582
- Joined: Wed Sep 05, 2018 2:16 am
- Location: Moving
- Full name: Jorge Picado
Re: Why does LC0 perfoms so poor here in CCC ?
If I had ten fights against anybody and got my ass whip, do you think that I would like to fight a few more times, Simply ask Stockfish if he wants Leela to whip him a few more times. I guess Stockfish will tell you Hell NO, and now that Leela is getting hotter and hotter I am afraid that with all that Heat coming out of her GPU, she is preparing to have fish and I do NOT want to be her lunch.Modern Times wrote: ↑Wed May 22, 2019 10:44 pm That is correct, it lost 48.5 to 51.5 to Lc0 0.21.1 JH.T6.532 GPU. Of course the statistical margin of error with a single 100 game match is high. Run the matches again, and you may get a slightly different result.
On this ratings list, Lc0 seems to under-perform against weaker opponents. So although it did beat Stockfish 10 head-to-head, it has a slightly lower rating than Stockfish.
It won't be long before Lc0 overtakes Stockfish 10 1CPU on this list. I'm waiting until I'm pretty sure they have +20 Elo gains over JH.T6.532 before testing it again.
Do NOT worry and be happy, we all live a short life
-
- Posts: 5582
- Joined: Wed Sep 05, 2018 2:16 am
- Location: Moving
- Full name: Jorge Picado
Re: Why does LC0 perfoms so poor here in CCC ?
Chessqueen wrote: ↑Thu May 23, 2019 6:13 pmIf I had ten fights against anybody and got my ass whip, do you think that I would like to fight a few more times, Simply ask Stockfish if he wants Leela to whip him a few more times. I guess Stockfish will tell you Hell NO, and now that Leela is getting hotter and hotter I am afraid that with all that Heat coming out of her GPU, she is preparing to have fish and I do NOT want to be her lunch.Modern Times wrote: ↑Wed May 22, 2019 10:44 pm That is correct, it lost 48.5 to 51.5 to Lc0 0.21.1 JH.T6.532 GPU. Of course the statistical margin of error with a single 100 game match is high. Run the matches again, and you may get a slightly different result.
On this ratings list, Lc0 seems to under-perform against weaker opponents. So although it did beat Stockfish 10 head-to-head, it has a slightly lower rating than Stockfish.
It won't be long before Lc0 overtakes Stockfish 10 1CPU on this list. I'm waiting until I'm pretty sure they have +20 Elo gains over JH.T6.532 before testing it again.
And that is with a very low end GPU graphic card, imagine what LCO could have done with this one ==> https://www.newegg.com/msi-geforce-rtx- ... &gclsrc=ds
And in 3 months LC0 will be using a TPU ==> https://www.nextplatform.com/2017/04/05 ... hitecture/
https://tcec.chessdom.com/live.html
Do NOT worry and be happy, we all live a short life