CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Eduard
Posts: 224
Joined: Fri Oct 26, 2018 10:58 pm
Location: Germany
Full name: Eduard Nemeth
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Eduard » Tue Feb 19, 2019 7:58 am

Question: On which CPU is LC Zero running? Only 1 core? NO distilled network and has more than 3000 Elo?!?
Last edited by Eduard on Tue Feb 19, 2019 8:06 am, edited 3 times in total.

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Rebel » Tue Feb 19, 2019 8:01 am

Graham Banks wrote:
Tue Feb 19, 2019 6:46 am
Rebel wrote:
Tue Feb 19, 2019 5:47 am

Code: Select all

Rank  Name                     Rating 
 45   Lc0 0.20.1 w36089 64-bit  3022
Perhaps it's an idea to include the average NPS in the name for LZ as a sort of indication of the strength of the GPU used.
That's on 1CPU only. No GPU.
Okay.

Now that we are talking, are other programs now also allowed to use datafiles that add strength?
90% of coding is debugging, the other 10% is writing bugs.

User avatar
Graham Banks
Posts: 33239
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Graham Banks » Tue Feb 19, 2019 9:28 am

Rebel wrote:
Tue Feb 19, 2019 8:01 am
Graham Banks wrote:
Tue Feb 19, 2019 6:46 am
Rebel wrote:
Tue Feb 19, 2019 5:47 am

Code: Select all

Rank  Name                     Rating 
 45   Lc0 0.20.1 w36089 64-bit  3022
Perhaps it's an idea to include the average NPS in the name for LZ as a sort of indication of the strength of the GPU used.
That's on 1CPU only. No GPU.
Okay.

Now that we are talking, are other programs now also allowed to use datafiles that add strength?
Not that I'm aware of, aside from a generic opening book and tablebases.
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Rebel » Tue Feb 19, 2019 3:18 pm

Graham Banks wrote:
Tue Feb 19, 2019 9:28 am
Rebel wrote:
Tue Feb 19, 2019 8:01 am
Graham Banks wrote:
Tue Feb 19, 2019 6:46 am
Rebel wrote:
Tue Feb 19, 2019 5:47 am

Code: Select all

Rank  Name                     Rating 
 45   Lc0 0.20.1 w36089 64-bit  3022
Perhaps it's an idea to include the average NPS in the name for LZ as a sort of indication of the strength of the GPU used.
That's on 1CPU only. No GPU.
Okay.

Now that we are talking, are other programs now also allowed to use datafiles that add strength?
Not that I'm aware of, aside from a generic opening book and tablebases.
So learning files are allowed?
90% of coding is debugging, the other 10% is writing bugs.

tpoppins
Posts: 919
Joined: Tue Nov 24, 2015 8:11 pm
Location: upstate

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by tpoppins » Tue Feb 19, 2019 4:28 pm

Rebel wrote:
Tue Feb 19, 2019 3:18 pm
So learning files are allowed?
No.
Tirsa Poppins
CCRL

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Rebel » Tue Feb 19, 2019 6:59 pm

tpoppins wrote:
Tue Feb 19, 2019 4:28 pm
Rebel wrote:
Tue Feb 19, 2019 3:18 pm
So learning files are allowed?
No.
Read it, thanks.

And exactly what are the network files that come with Lc0 ?

Not Learned?
90% of coding is debugging, the other 10% is writing bugs.

Modern Times
Posts: 2420
Joined: Thu Jun 07, 2012 9:02 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Modern Times » Tue Feb 19, 2019 8:05 pm

No learning is allowed during the course of running an engine. Prior fixed learning is allowed. I can't think of any examples at the moment. Any testing an engine author does prior to release is in effect learning.

I guess the exception is if the learning file is essentially an opening book, then that is a problem but things get a bit murky as well. In essence the engine is supposed to do the work and not use pre-prepared moves, except for tablebases as Graham says,
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

User avatar
Rebel
Posts: 4788
Joined: Thu Aug 18, 2011 10:04 am

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by Rebel » Wed Feb 20, 2019 9:47 am

Modern Times wrote:
Tue Feb 19, 2019 8:05 pm
No learning is allowed during the course of running an engine. Prior fixed learning is allowed. I can't think of any examples at the moment. Any testing an engine author does prior to release is in effect learning.

I guess the exception is if the learning file is essentially an opening book, then that is a problem but things get a bit murky as well. In essence the engine is supposed to do the work and not use pre-prepared moves, except for tablebases as Graham says,
Makes sense.

AZ and LZ are the biggest breakthrough in the history of computer chess, never seen an elo progression of 3400 in one year, most programmers would settle for 50. I am glad to see LZ rapidly climbing in your rating list.

Nevertheless I think LZ is a game changer to your rules. LZ learns by self-play and now other engines should have the same right.

I agree with you datafiles shouldn't be opening books in disguise and the learning part (if present) should be turned off during your testing. If you plan to play 3 openings against X,Y and Z the engine is not allowed to learn from the game against X.
90% of coding is debugging, the other 10% is writing bugs.

chrisw
Posts: 2188
Joined: Tue Apr 03, 2012 2:28 pm

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by chrisw » Wed Feb 20, 2019 10:12 am

Rebel wrote:
Wed Feb 20, 2019 9:47 am
Modern Times wrote:
Tue Feb 19, 2019 8:05 pm
No learning is allowed during the course of running an engine. Prior fixed learning is allowed. I can't think of any examples at the moment. Any testing an engine author does prior to release is in effect learning.

I guess the exception is if the learning file is essentially an opening book, then that is a problem but things get a bit murky as well. In essence the engine is supposed to do the work and not use pre-prepared moves, except for tablebases as Graham says,
Makes sense.

AZ and LZ are the biggest breakthrough in the history of computer chess, never seen an elo progression of 3400 in one year, most programmers would settle for 50. I am glad to see LZ rapidly climbing in your rating list.

Nevertheless I think LZ is a game changer to your rules. LZ learns by self-play and now other engines should have the same right.

I agree with you datafiles shouldn't be opening books in disguise and the learning part (if present) should be turned off during your testing. If you plan to play 3 openings against X,Y and Z the engine is not allowed to learn from the game against X.
There’s a difference between a traditional opening book and a LC0 network, or even LC0 sub network trained only on opening positions for say twenty moves.
The traditional program absolutely knows if it is in book, and can play a book move. LC0 doesn’t know, it can’t tell from its NN if the evaluation it gets back is “opening” or not. The former plays moves with absolute reliability but no intelligence at all, its just a look up. The LC0 “lookup” is as reliable or unreliable as with any other move, opening or not. And, in practice, LC0 isn’t looking up it's NN at root and finding a book move, it’s doing a search.

*if* traditional engines were allowed to use an opening book root-NN (as opposed to a lookup book), I think that would be ok, and parallel with what LC0 does. If they were clever enoughto also give the NN a categorical output “this is or isn’t a known book move” then, why not? It wouldn’t always be correct anyway.

User avatar
hgm
Posts: 23775
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

Post by hgm » Wed Feb 20, 2019 10:13 am

Modern Times wrote:
Tue Feb 19, 2019 8:05 pm
No learning is allowed during the course of running an engine. Prior fixed learning is allowed. I can't think of any examples at the moment.
All .exe files are examples, aren't they? They contain the evaluation parameters that were learned/tuned by playing millions of games. LC0 learning is just tuning of its NN parameters; there is no difference between this kind of learning and tuning of any eval, (other than that the LC0 eval has more parameters). I don't know if there are any Chess engines that store their eval parameters in a separate parameter file (e.g. to facilitate tuning without having to recompile), but I know for sure that most Shogi engines do this. (E.g. for Bonanza you would have to download a >200MB file with eval parameters, in addition to the executable.)

Whether engine authors choose to put their program (code + data) in a single file or distribute them over several files is pretty immaterial, right? It would be easy enough to compile the network file into the LC0 executable.

Post Reply