3497 elo Leela, possibly 4th best chess engine now!!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Nay Lin Tun
Posts: 514
Joined: Mon Jan 16, 2012 5:34 am

3497 elo Leela, possibly 4th best chess engine now!!

Post by Nay Lin Tun » Tue Aug 07, 2018 12:27 pm

Dear Computer Chess Commmunity,

Leela fans are happy to share recent amazing progress of Leela. In the last week, Leela gained about +100 elo securing the top 10 position in decent GPU hardwares (like GTX 1060) and possibly 4th best engine in GPU favoured hardwares(like GTX 1080Ti ). With TCEC hardware conditions, Leela should be between 4th and 6th position though.

Regards,
Nay

Credit: Thanks Csculie for testing. Hardware 4 cores i7 7700k and GTX 1080 Ti, time control 1+1

https://docs.google.com/spreadsheets/d/ ... 1316991135
Attachments
Capture.PNG
Capture.PNG (70.64 KiB) Viewed 2030 times

Henk
Posts: 5799
Joined: Mon May 27, 2013 8:31 am

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by Henk » Tue Aug 07, 2018 12:30 pm

But its performance on WCCC was a huge disappointment and that was only a few weeks ago.

Nay Lin Tun
Posts: 514
Joined: Mon Jan 16, 2012 5:34 am

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by Nay Lin Tun » Tue Aug 07, 2018 12:40 pm

Henk wrote:
Tue Aug 07, 2018 12:30 pm
But its performance on WCCC was a huge disappointment and that was only a few weeks ago.
Yeah, last week's improvement was amazing. In certain tests, it is up to +350 elo than the current Leela playing in TCEC. +100 elo is extremely skeptic view. :D

User avatar
Guenther
Posts: 3013
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by Guenther » Tue Aug 07, 2018 1:05 pm

Nay Lin Tun wrote:
Tue Aug 07, 2018 12:40 pm
Henk wrote:
Tue Aug 07, 2018 12:30 pm
But its performance on WCCC was a huge disappointment and that was only a few weeks ago.
Yeah, last week's improvement was amazing. In certain tests, it is up to +350 elo than the current Leela playing in TCEC. +100 elo is extremely skeptic view. :D
Well, nothing in this 'sheet' has any verified data...
Current foe list count : [92]
http://rwbc-chess.de/chronology.htm

Werewolf
Posts: 1190
Joined: Thu Sep 18, 2008 8:24 pm

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by Werewolf » Tue Aug 07, 2018 1:14 pm

Nay Lin Tun wrote:
Tue Aug 07, 2018 12:40 pm
Henk wrote:
Tue Aug 07, 2018 12:30 pm
But its performance on WCCC was a huge disappointment and that was only a few weeks ago.
Yeah, last week's improvement was amazing. In certain tests, it is up to +350 elo than the current Leela playing in TCEC. +100 elo is extremely skeptic view. :D
Nice work. Well done Leela team (assuming this is true) :D

flither
Posts: 16
Joined: Thu Aug 02, 2018 9:16 pm
Full name: Raf Levsky

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by flither » Tue Aug 07, 2018 2:04 pm

My first post here so don't take it seriously :)
You shouldn't say Leela is number 4 as it runs on different hardware, you should compare it to other NN engines running on the same GPU's.
Humans ratings are ok, humans can't extend their brains (except cheating), but comparing machines running on different hardwares is pointless and creating mess.
Just my 2 cents

User avatar
Laskos
Posts: 9410
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by Laskos » Tue Aug 07, 2018 2:21 pm

Guenther wrote:
Tue Aug 07, 2018 1:05 pm
Nay Lin Tun wrote:
Tue Aug 07, 2018 12:40 pm
Henk wrote:
Tue Aug 07, 2018 12:30 pm
But its performance on WCCC was a huge disappointment and that was only a few weeks ago.
Yeah, last week's improvement was amazing. In certain tests, it is up to +350 elo than the current Leela playing in TCEC. +100 elo is extremely skeptic view. :D
Well, nothing in this 'sheet' has any verified data...
I also have +100 Elo points above 10160-10180 nets for later nets (10161 used in TCEC), but the last several dozen nets are weaker. For ID10474 on GTX 1060 at longer 600'' + 10'' TC, I have a result of +8 -2 =10 against Deep Shredder 13 on 4 cores, which is performance of 3430 CCRL 40/4' Elo points (error bars are large). Generally, a performance of 3400 Elo points on GTX 1060 in CCRL 40/4' conditions with more games (smaller error margins), and probably higher rating at longer TC. From these, one would also infer that it should come in TCEC conditions on places 4-6, that is, lower Premiere Division or high First Division. But is seem to not perform at this level looking at current TCEC results. It is roughly the level of properly functioning IvanHoe. Add 100 Elo points to ID10161 net, and this would barely qualify it to the First Division, nowhere near Premiere Division. I am not sure why theoretical prediction doesn't seem to match TCEC games, either 2x1080Ti don't scale very well or 43 cores scale very well with current SMP used by engines.
Last edited by Laskos on Tue Aug 07, 2018 2:27 pm, edited 2 times in total.

RubiChess
Posts: 107
Joined: Fri Mar 30, 2018 5:20 am

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by RubiChess » Tue Aug 07, 2018 2:23 pm

flither wrote:
Tue Aug 07, 2018 2:04 pm
My first post here so don't take it seriously :)
... comparing machines running on different hardwares is pointless and creating mess.
Just my 2 cents
IMO one of the best first posts ever in this forum.

Jouni
Posts: 1979
Joined: Wed Mar 08, 2006 7:15 pm

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by Jouni » Tue Aug 07, 2018 2:24 pm

Even if Leela were at Stockfish dev level I am not interested yet! Because 1. You need additional hardware - no thanks and 2. it can't solve any difficult test position.
Jouni

Robert Pope
Posts: 510
Joined: Sat Mar 25, 2006 7:27 pm

Re: 3497 elo Leela, possibly 4th best chess engine now!!

Post by Robert Pope » Tue Aug 07, 2018 2:42 pm

flither wrote:
Tue Aug 07, 2018 2:04 pm
My first post here so don't take it seriously :)
You shouldn't say Leela is number 4 as it runs on different hardware, you should compare it to other NN engines running on the same GPU's.
Humans ratings are ok, humans can't extend their brains (except cheating), but comparing machines running on different hardwares is pointless and creating mess.
Just my 2 cents
While true, NN engines are only going to proliferate and be a significant portion of the top engines out there for playing chess. It just isn't practical to plug our ears and say "la la la we can't compare engines on different hardware". At some point, we have to be able to compare them and have some measure of hardware X for NN is roughly equivalent to hardware Y for traditional engines.

We already do that to some extent, by glossing over the fact that some CPUs are better for one engine than for others.

Post Reply