Leela doing well in chess 960 in TCEC!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Nay Lin Tun
Posts: 694
Joined: Mon Jan 16, 2012 5:34 am

Leela doing well in chess 960 in TCEC!

Post by Nay Lin Tun » Sun Nov 22, 2020 11:38 pm

There were some myths (or data) that Leela will not be good at chess 960.
Now she is the only engine in TCEC chess 960 (FRC) without a loss in the whole tourney so far!
(Afaik, there were a few FRC positions in the book that was briefly used in T60 but training with a book was stopped because there was no improvement for Leela using a book in training)

This is current standing of 4 engines RR and there will be two engines in 50 games final!

https://imgur.com/JesXld9

Alayan
Posts: 516
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Leela doing well in chess 960 in TCEC!

Post by Alayan » Sun Nov 22, 2020 11:49 pm

Stockfish's NN has not been trained at all on Chess960 and so is quite bad at it. NNUE gained a lot of strength in classical but almost none in FRC.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Leela doing well in chess 960 in TCEC!

Post by Laskos » Mon Nov 23, 2020 12:35 am

Nay Lin Tun wrote:
Sun Nov 22, 2020 11:38 pm
There were some myths (or data) that Leela will not be good at chess 960.
Now she is the only engine in TCEC chess 960 (FRC) without a loss in the whole tourney so far!
(Afaik, there were a few FRC positions in the book that was briefly used in T60 but training with a book was stopped because there was no improvement for Leela using a book in training)

This is current standing of 4 engines RR and there will be two engines in 50 games final!

https://imgur.com/JesXld9
It will perform well on things it will be trained for. With older nets, it under-performed by more than 50 Elo points in FRC compared to standard chess.

MMarco
Posts: 170
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: Leela doing well in chess 960 in TCEC!

Post by MMarco » Mon Nov 23, 2020 8:11 am

Laskos wrote:
Mon Nov 23, 2020 12:35 am
Nay Lin Tun wrote:
Sun Nov 22, 2020 11:38 pm
There were some myths (or data) that Leela will not be good at chess 960.
Now she is the only engine in TCEC chess 960 (FRC) without a loss in the whole tourney so far!
(Afaik, there were a few FRC positions in the book that was briefly used in T60 but training with a book was stopped because there was no improvement for Leela using a book in training)

This is current standing of 4 engines RR and there will be two engines in 50 games final!

https://imgur.com/JesXld9
It will perform well on things it will be trained for. With older nets, it under-performed by more than 50 Elo points in FRC compared to standard chess.
According to one of Leela dev on discord:
Tilps wrote: It has seen some FRC positions - specifically from tcec FRC1 IIRC
but its network is not FRC adapted
so its still clueless about the specifics of FRC castling
(like which of opponents rooks can still be castled with)
...
the real question is whether any of the training window [with FRC games] was used to train J92 nets
even if T60 did learn to change its weights abit - and then subsequently didn't unlearn it
that doesn't mean much for J92
Apparently, J92-330 didn't received much training at FRC, if any.

So much for the theory "Leela is an opening book"? 🙂

Nay Lin Tun
Posts: 694
Joined: Mon Jan 16, 2012 5:34 am

Re: Leela doing well in chess 960 in TCEC!

Post by Nay Lin Tun » Mon Nov 23, 2020 11:32 am

Leela is a book for FRC as well !!! :P :mrgreen: :twisted:

Alayan
Posts: 516
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Leela doing well in chess 960 in TCEC!

Post by Alayan » Mon Nov 23, 2020 1:01 pm

Comparing Leela against two others NN that are not trained on FRC either, and Komodo that's not tuned on FRC and is much weaker in standard chess, doesn't show that it doesn't lose elo in FRC.

In FRC semi-league 2, Leela drew all 6 games against Ethereal, and without putting it in a difficult position out of the opening in any game. This is an indication of underperformance, considering that Leela beats Eth by 200 elo at FastGM 16 cores in standard chess.

Nay Lin Tun
Posts: 694
Joined: Mon Jan 16, 2012 5:34 am

Re: Leela doing well in chess 960 in TCEC!

Post by Nay Lin Tun » Mon Nov 23, 2020 1:23 pm

Alayan wrote:
Mon Nov 23, 2020 1:01 pm
Comparing Leela against two others NN that are not trained on FRC either, and Komodo that's not tuned on FRC and is much weaker in standard chess, doesn't show that it doesn't lose elo in FRC.

In FRC semi-league 2, Leela drew all 6 games against Ethereal, and without putting it in a difficult position out of the opening in any game. This is an indication of underperformance, considering that Leela beats Eth by 200 elo at FastGM 16 cores in standard chess.
What is your vote for FRC final ?

Now, majority of tcec mobs are voting for lco!! :mrgreen: :twisted: :wink:

Alayan
Posts: 516
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Leela doing well in chess 960 in TCEC!

Post by Alayan » Mon Nov 23, 2020 3:57 pm

Leela might well beat Stockfish. NNUE without training on FRC positions seem to suffer a lot there.

Nay Lin Tun
Posts: 694
Joined: Mon Jan 16, 2012 5:34 am

Re: Leela doing well in chess 960 in TCEC!

Post by Nay Lin Tun » Mon Nov 23, 2020 4:21 pm

Alayan wrote:
Mon Nov 23, 2020 3:57 pm
Leela might well beat Stockfish. NNUE without training on FRC positions seem to suffer a lot there.
TCEC S19 Sufi showed that " Leela did not lose Stockfish in well balanced opening positions in the first half, but she lose Stockfish in TCEC custom opening positions, one sided almost losing positions in the second half ".

To solve TCEC specific Sufi problem, Leela team had trained T60 with all those TCEC sufi opening books from the last 4 seasons plus FRC positions :P :) :shock:

Modern Times
Posts: 2702
Joined: Thu Jun 07, 2012 9:02 pm

Re: Leela doing well in chess 960 in TCEC!

Post by Modern Times » Mon Nov 23, 2020 8:08 pm

I'm not following this, but if Dragon by Komodo is in the FRC final then that would be my pick for the winner.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Post Reply