My failed attempt to change TCEC NN clone rules

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
crem
Posts: 123
Joined: Wed May 23, 2018 7:29 pm

My failed attempt to change TCEC NN clone rules

Post by crem » Sat Sep 14, 2019 7:32 am

I wanted to bring up this topic several times already, but my drafts were too long to post.
Now, as Allie has good chances to go into finals, I think it’s time to bring this topic back.

Also I get frequent questions why don’t I bring this topic up if I don’t agree.

So, I tried to bring this topic up with TCEC administration, with no success.

Timeline:

March 10th (TCEC 15 just started, and Allie suddenly appeared there)

I contacted Anton Mihailov, TD director, telling that:

1. Current TCEC clone rules are not applied correctly:
Allie+Stein is a clone according to TCEC rules (both Allie is not unique relative to Lc0, and Stein is not unique comparing to Leela’s weights, see also viewtopic.php?p=792755).
2. Rules themselves are poor and have to be changed.
3. I proposed various changes and ideas to the rules (with most of them Allie+Stein would be able to participate, and in some also “DeusX” could).
4. TCEC15 is already messed up, but let’s do that right for TCEC16, we have plenty of time (3 months until the new season start).

Anton responded:
1. This is very important, please keep writing.
2. Please keep it TOP SECRET! Noone should know!
(I tried to convince them that such discussions should be public, only got irrelevant answers, that I don’t know how to manage large communities)

I wrote lots of material, in different forms: one-line summary, one paragraph summary, diagrams, very detailed description etc.

In the end it was clear that noone from TCEC read even one-line summary.

March 29th
(not very relevant but for completeness)
We created a discord server, and Anton invited an undisclosed guest expert to the discussion, with whom we had an interesting, but short and not very relevant to TCEC rules discussion (because TCEC rules are up to TCEC team, really).

March 31th
I received the last message from Anton stating how important is this, and that I should continue writing my proposals.

April
I was pinging them periodically, with no reaction.

May 1st
I deleted the conversations (it was Google Doc and Discord server). Noone seemed to notice. Noone contacted me after that.

In the end I wasted ~15 hours of my time drawing all the diagrams and explanations of different detailization, all to save TCEC admins time.
From chats with them it was clear though that they didn’t even spend 5 minutes on that, they didn’t even read the 1-line summary.

Then TCEC-16 started with no changes at all.

At this point I decided that trying to convey any message to TCEC doesn't worth the effort.



Some screenshots from the document:
Image
Image
Image
Image

Modern Times
Posts: 2410
Joined: Thu Jun 07, 2012 9:02 pm

Re: My failed attempt to change TCEC NN clone rules

Post by Modern Times » Sat Sep 14, 2019 7:59 am

The whole clone and derivatives issue is a minefield. So many different opinions and viewpoints and "definitions". I would advise Anton to replace any clone "rules" with a simple statement "engines are entered at our sole discretion" then there are no arguments about rules being followed or not. Job done.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

crem
Posts: 123
Joined: Wed May 23, 2018 7:29 pm

Re: My failed attempt to change TCEC NN clone rules

Post by crem » Sat Sep 14, 2019 8:05 am

Modern Times wrote:
Sat Sep 14, 2019 7:59 am
The whole clone and derivatives issue is a minefield. So many different opinions and viewpoints and "definitions". I would advise Anton to replace any clone "rules" with a simple statement "engines are entered at our sole discretion" then there are no arguments about rules being followed or not. Job done.
I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.

User avatar
xr_a_y
Posts: 762
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: My failed attempt to change TCEC NN clone rules

Post by xr_a_y » Sat Sep 14, 2019 8:08 am

crem wrote:
Sat Sep 14, 2019 8:05 am
Modern Times wrote:
Sat Sep 14, 2019 7:59 am
The whole clone and derivatives issue is a minefield. So many different opinions and viewpoints and "definitions". I would advise Anton to replace any clone "rules" with a simple statement "engines are entered at our sole discretion" then there are no arguments about rules being followed or not. Job done.
I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.
Looks like an impossible task to me, not only for NN engines.

And a lot thanks for your diagram, that are very clear.

Modern Times
Posts: 2410
Joined: Thu Jun 07, 2012 9:02 pm

Re: My failed attempt to change TCEC NN clone rules

Post by Modern Times » Sat Sep 14, 2019 8:16 am

crem wrote:
Sat Sep 14, 2019 8:05 am

I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.
Ridiculous response, that could never work
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

crem
Posts: 123
Joined: Wed May 23, 2018 7:29 pm

Re: My failed attempt to change TCEC NN clone rules

Post by crem » Sat Sep 14, 2019 8:29 am

Modern Times wrote:
Sat Sep 14, 2019 8:16 am
crem wrote:
Sat Sep 14, 2019 8:05 am

I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.
Ridiculous response, that could never work
I cannot find exact response in logs, so it's possible that I exaggerated the response.
By my first proposal was to "define only vague rules, and they let admins apply common sense", and they didn't like it.

noobpwnftw
Posts: 360
Joined: Sun Nov 08, 2015 10:10 pm

Re: My failed attempt to change TCEC NN clone rules

Post by noobpwnftw » Sat Sep 14, 2019 8:37 am

So to summarize, it's a story of how two dictators pretending to be open to opinions while one suggests the other shouldn't and the other suggesting to keep it up.

P.S. I was also told that I get to decide many things but in fact my opinion doesn't matter, either. :D

User avatar
Ozymandias
Posts: 1126
Joined: Sun Oct 25, 2009 12:30 am

Re: My failed attempt to change TCEC NN clone rules

Post by Ozymandias » Sat Sep 14, 2019 8:56 am

noobpwnftw wrote:
Sat Sep 14, 2019 8:37 am
I was also told that I get to decide many things but in fact my opinion doesn't matter
Sounds like you talked to some politician.

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: My failed attempt to change TCEC NN clone rules

Post by Ferdy » Sat Sep 14, 2019 10:35 am

crem wrote:
Sat Sep 14, 2019 7:32 am
I wanted to bring up this topic several times already, but my drafts were too long to post.
Now, as Allie has good chances to go into finals, I think it’s time to bring this topic back.

Also I get frequent questions why don’t I bring this topic up if I don’t agree.

So, I tried to bring this topic up with TCEC administration, with no success.
Crosschecking your Alliestein image and TCEC rules I get this.

Code: Select all

Enter engine name? Alliestein

Rule 1: The code for training the NN, unique? (y/n) n

Rule 2: The NN weights file, unique? (y/n) y

Rule 3: The engine that executes the NN, unique? (y/n) n

TCEC Season 16, NN uniqueness test summary:
name: Alliestein
Pass Rule 1: n
Pass Rule 2: y
Pass Rule 3: n
Eligibility for participation: Fail
A minimum of 2 rules should pass!

Ref:
Rules/sections 16.b, 16.c and 16.d
https://tcec-chess.com/
Lets hear from TCEC TD.

dkappe
Posts: 260
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: My failed attempt to change TCEC NN clone rules

Post by dkappe » Sat Sep 14, 2019 1:12 pm

At some point a NN engine will beat lc0 in TCEC, and like the swallows returning to Capistrano, you will pen another missive about how they need to be banned. I look forward to your ongoing contortions.

Post Reply