Leela will use Net 10480 in TCEC Div3

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Gabor Szots
Posts: 303
Joined: Sat Jul 21, 2018 5:43 am
Location: Szentendre, Hungary
Full name: Gabor Szots

Re: Leela will use Net 10480 in TCEC Div3

Post by Gabor Szots » Wed Aug 08, 2018 11:15 am

whereagles wrote:
Wed Aug 08, 2018 9:51 am
Gabor, it is very confusing yes. I believe they are using a 15x192 network in the main server (ID in the 500s) and a test network of 20x256 in the test server (ID in the 10000s), but I'm not totally sure.

The 20x256 network is the same size as AlphaZero so, in theory, with proper training time it will be as strong as A0, provided it will subsequently run on hardware comparable to google's 4 TPU.

The 15x192 network runs quicker and is stronger when using "lesser" hardware, e.g. nvidia geforce below 1060 or thereabouts.
Thank you for th reply.
Gabor Szots
CCRL testing group

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Leela will use Net 10480 in TCEC Div3

Post by Milos » Wed Aug 08, 2018 11:44 am

whereagles wrote:
Wed Aug 08, 2018 9:51 am
Gabor, it is very confusing yes. I believe they are using a 15x192 network in the main server (ID in the 500s) and a test network of 20x256 in the test server (ID in the 10000s), but I'm not totally sure.

The 20x256 network is the same size as AlphaZero so, in theory, with proper training time it will be as strong as A0, provided it will subsequently run on hardware comparable to google's 4 TPU.

The 15x192 network runs quicker and is stronger when using "lesser" hardware, e.g. nvidia geforce below 1060 or thereabouts.
Yes main server has so far the strongest 15x192 net and test server contains the strongest 20x256 net.
Even on 1060, the strongest 20x256 net is at least 70Elo stronger than the strongest 15x192 net. That is because 15x192 definitively plateaued and there is not much chance it would improve noticeably in the future. 20x256 net still has some 100-150 Elo headroom.

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Leela will use Net 10480 in TCEC Div3

Post by Milos » Wed Aug 08, 2018 11:51 am

whereagles wrote:
Wed Aug 08, 2018 9:57 am
pohl4711 wrote:
Wed Aug 08, 2018 6:03 am
I doubt, that DeusX will be allowed to play in Div 3.
https://blog.lczero.org/2018/08/07/mistake/
Ok, I think that settles it. It is a bit too radical, but understandable. (I would allow DX to finish this season, but I'm not the admin lol)

Still, comparing learning approaches does have value per se. Would be nice to see how far one gets using supervised learning based on top human and engine game (of the non-selfplay variety, of course).
Well if Albert actually knew anything about programming he could have easily added back TB support to Lc0 (from LcZero) since it suites supervised approach and in the process increases drastically the strength of his net and avoided this thing all together...

JJJ
Posts: 1287
Joined: Sat Apr 19, 2014 11:47 am

Re: Leela will use Net 10480 in TCEC Div3

Post by JJJ » Wed Aug 08, 2018 4:43 pm

Milos wrote:
Wed Aug 08, 2018 11:44 am
whereagles wrote:
Wed Aug 08, 2018 9:51 am
Gabor, it is very confusing yes. I believe they are using a 15x192 network in the main server (ID in the 500s) and a test network of 20x256 in the test server (ID in the 10000s), but I'm not totally sure.

The 20x256 network is the same size as AlphaZero so, in theory, with proper training time it will be as strong as A0, provided it will subsequently run on hardware comparable to google's 4 TPU.

The 15x192 network runs quicker and is stronger when using "lesser" hardware, e.g. nvidia geforce below 1060 or thereabouts.
Yes main server has so far the strongest 15x192 net and test server contains the strongest 20x256 net.
Even on 1060, the strongest 20x256 net is at least 70Elo stronger than the strongest 15x192 net. That is because 15x192 definitively plateaued and there is not much chance it would improve noticeably in the future. 20x256 net still has some 100-150 Elo headroom.
I have GTX 1060 6GB , so which net should I download to have the best Leela ? And what is his best config ? I d like to test the best Leela against others engine. Thanks you.

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Leela will use Net 10480 in TCEC Div3

Post by Milos » Wed Aug 08, 2018 5:10 pm

JJJ wrote:
Wed Aug 08, 2018 4:43 pm
Milos wrote:
Wed Aug 08, 2018 11:44 am
whereagles wrote:
Wed Aug 08, 2018 9:51 am
Gabor, it is very confusing yes. I believe they are using a 15x192 network in the main server (ID in the 500s) and a test network of 20x256 in the test server (ID in the 10000s), but I'm not totally sure.

The 20x256 network is the same size as AlphaZero so, in theory, with proper training time it will be as strong as A0, provided it will subsequently run on hardware comparable to google's 4 TPU.

The 15x192 network runs quicker and is stronger when using "lesser" hardware, e.g. nvidia geforce below 1060 or thereabouts.
Yes main server has so far the strongest 15x192 net and test server contains the strongest 20x256 net.
Even on 1060, the strongest 20x256 net is at least 70Elo stronger than the strongest 15x192 net. That is because 15x192 definitively plateaued and there is not much chance it would improve noticeably in the future. 20x256 net still has some 100-150 Elo headroom.
I have GTX 1060 6GB , so which net should I download to have the best Leela ? And what is his best config ? I d like to test the best Leela against others engine. Thanks you.
I believe it is still 10445. I use the default settings. Probably something could be gained with CLOP-ing them, but since training is also done with them IMHO not much.

whereagles
Posts: 561
Joined: Thu Nov 13, 2014 11:03 am

Re: Leela will use Net 10480 in TCEC Div3

Post by whereagles » Wed Aug 08, 2018 7:03 pm

meanwhile the "lowly" 15x192 main server seems to be rising to unseen heights.. not bad for a would-be plateauing NN.

Well.. for self-play that is. Maybe it's not so hot in general play.

User avatar
Laskos
Posts: 9513
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Leela will use Net 10480 in TCEC Div3

Post by Laskos » Wed Aug 08, 2018 7:36 pm

pohl4711 wrote:
Wed Aug 08, 2018 4:50 am
Just read on discord (dev-log channel), that Leela will play with Net 10480 in TCEC in Divison 3. If anybody want to do some testing with TCEC-Leela...

Download the Net 10480 here: http://testserver.lczero.org/networks

Because of that, I restarted my long thinking-time testrun (https://www.sp-cc.de/long-thinkingtime.htm) with that Net - lets see, how Leela will score with 10480 (300 games with 12'+5''). Result not before next Wednesday. My first testrun with a 20x256 net - Leela-ratio with that bigger net in my testrun is 0.67 (which should be a little bit better for Leela than in TCEC)
Stay tuned!
I have very small samples at longer time control, but it might be that the newest nets are even stronger than ID10480 at LTC. In fast games they are similar, but at 600'' + 10'' time control I used, the small samples are:

Lc0 on GTX 1060 6GB
Deep Shredder 13 on 4 i7 cores 3.8 GHz

Lc0 ID10474 vs Deep Shredder 13 4 CPU
+8 -2 =10

A performance of 3430 CCRL 40/4' Elo points

With one of the newest nets:

Lc0 ID10526 vs Deep Shredder 13 4 CPU
+8 -0 =12

A performance of 3480 CCRL 40/4' Elo points

ID10526 literally destroys Shredder 13 on 4 cores at this time control, even the drawn games were usually in Lc0 advantage, but not sufficient to win.

Werewolf
Posts: 1198
Joined: Thu Sep 18, 2008 8:24 pm

Re: Leela will use Net 10480 in TCEC Div3

Post by Werewolf » Wed Aug 08, 2018 8:17 pm

Laskos wrote:
Wed Aug 08, 2018 7:36 pm


Lc0 ID10526 vs Deep Shredder 13 4 CPU
+8 -0 =12

A performance of 3480 CCRL 40/4' Elo points

ID10526 literally destroys Shredder 13 on 4 cores at this time control
:shock:

Nay Lin Tun
Posts: 530
Joined: Mon Jan 16, 2012 5:34 am

Re: Leela will use Net 10480 in TCEC Div3

Post by Nay Lin Tun » Thu Aug 09, 2018 5:54 am

This result shows + 147 elo than shredder, resulting same elo of Fire.
However latest SF is still +200 elo than Fire.
This is why I posted in another thread that Leela could be 4th best chess engine now.

User avatar
George Tsavdaris
Posts: 1620
Joined: Thu Mar 09, 2006 11:35 am

Re: Leela will use Net 10480 in TCEC Div3

Post by George Tsavdaris » Thu Aug 09, 2018 11:33 am

Laskos wrote:
Wed Aug 08, 2018 7:36 pm
I have very small samples at longer time control, but it might be that the newest nets are even stronger than ID10480 at LTC. In fast games they are similar, but at 600'' + 10'' time control I used, the small samples are:

Lc0 on GTX 1060 6GB
Deep Shredder 13 on 4 i7 cores 3.8 GHz

Lc0 ID10474 vs Deep Shredder 13 4 CPU
+8 -2 =10

A performance of 3430 CCRL 40/4' Elo points

With one of the newest nets:

Lc0 ID10526 vs Deep Shredder 13 4 CPU
+8 -0 =12

A performance of 3480 CCRL 40/4' Elo points

ID10526 literally destroys Shredder 13 on 4 cores at this time control, even the drawn games were usually in Lc0 advantage, but not sufficient to win.
Main contributor of testserver right now, Sergio Vieri, run a test a little before in his stream(he stop it and restarted it since he realized he did not save the PGNs) and the result of latest testnet versus SF9 was unbelievable. I saw just 1 game and Leela completely outplayed SF 9!
Time control is 40/1. Result was:
Lc0 10547 on 2 Titan V - Stockfish 9 on 31 cores(2xE5-2620): +6 -1 =4 with +170 Elo performance for Leela versus Stockfish 9!

His stream is the following and now he uses i think 40/2 TC and first game was draw:
https://www.twitch.tv/sergiovieri
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....

Post Reply