TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Laskos
Posts: 9509
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by Laskos » Tue Jul 31, 2018 10:05 pm

jkiliani wrote:
Tue Jul 31, 2018 8:59 pm
CMCanavessi wrote:
Tue Jul 31, 2018 8:35 pm
I think this move may backfire and will hurt TCEC more than it will do good. People are not stupid and want diversity, not 32 clones of SF. Deus X will use lc0 with a custom net. Shouldn't be allowed imho.
I agree, it seems rather difficult to me to justify allowing Deus X while not letting developers of other strong engines enter clones with changed parameters under a new name. Let's see how TCEC spins this once the complaints to this move start coming in...

For Season 13, it seems the decision is made, but for the season after this they'll have to come up with a new set of rules and follow it consistently.
I don't agree. One uses supervised learning, another reinforcement learning. They will play completely different chess. Also, as the 2 groups (main and test) seem to stall at significantly below A0 level (even if put on the same hardware), supervised learning is worth give a try. I hope this Deus X wins the TCEC, and all these discussions will be put to rest.

AndrewGrant
Posts: 494
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by AndrewGrant » Tue Jul 31, 2018 10:12 pm

Laskos wrote:
Tue Jul 31, 2018 10:05 pm
jkiliani wrote:
Tue Jul 31, 2018 8:59 pm
CMCanavessi wrote:
Tue Jul 31, 2018 8:35 pm
I think this move may backfire and will hurt TCEC more than it will do good. People are not stupid and want diversity, not 32 clones of SF. Deus X will use lc0 with a custom net. Shouldn't be allowed imho.
I agree, it seems rather difficult to me to justify allowing Deus X while not letting developers of other strong engines enter clones with changed parameters under a new name. Let's see how TCEC spins this once the complaints to this move start coming in...

For Season 13, it seems the decision is made, but for the season after this they'll have to come up with a new set of rules and follow it consistently.
I don't agree. One uses supervised learning, another reinforcement learning. They will play completely different chess. Also, as the 2 groups (main and test) seem to stall at significantly below A0 level (even if put on the same hardware), supervised learning is worth give a try. I hope this Deus X wins the TCEC, and all these discussions will be put to rest.
I don't think anyone here is questioning the possible promise of this supervised learning.

They are questioning why TCEC is allowing the same engine to play twice. There is more to it than just the weights.

Daniel Shawul
Posts: 3762
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by Daniel Shawul » Tue Jul 31, 2018 10:28 pm

Graham Banks wrote:
Tue Jul 31, 2018 1:33 am
Nay Lin Tun wrote:
Sun Jul 29, 2018 3:58 pm
Graham Banks wrote:
Sun Jul 29, 2018 6:30 am
Nay Lin Tun wrote:
Sun Jul 29, 2018 6:24 am
Well, there is extreme high possibility that Deus X authour is Deep Junior Authour, Shay Bushinsky.
https://www.aaai.org/ojs/index.php/aima ... /view/2255

So you will be seeing updated version(work of art) of Junior!!
:D :)
No - you're wrong, but my lips are sealed. :wink:
What about Tencent company?
https://www.chess.com/forum/view/genera ... -season-13
http://www.chessdom.com/deus-x-the-nn-c ... rt-silver/
Meanwhile, Scorpio with neural networks is ignored ...

It uses supervised learning (just like the deuce) with different NN architecture, search, backend etc...

Hurts to be ignored after investing so much time in it ... is it because I am black :) :) (obviously I am joking here)

User avatar
CMCanavessi
Posts: 844
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by CMCanavessi » Tue Jul 31, 2018 10:36 pm

Daniel Shawul wrote:
Tue Jul 31, 2018 10:28 pm
Graham Banks wrote:
Tue Jul 31, 2018 1:33 am
Nay Lin Tun wrote:
Sun Jul 29, 2018 3:58 pm
Graham Banks wrote:
Sun Jul 29, 2018 6:30 am
Nay Lin Tun wrote:
Sun Jul 29, 2018 6:24 am
Well, there is extreme high possibility that Deus X authour is Deep Junior Authour, Shay Bushinsky.
https://www.aaai.org/ojs/index.php/aima ... /view/2255

So you will be seeing updated version(work of art) of Junior!!
:D :)
No - you're wrong, but my lips are sealed. :wink:
What about Tencent company?
https://www.chess.com/forum/view/genera ... -season-13
http://www.chessdom.com/deus-x-the-nn-c ... rt-silver/
Meanwhile, Scorpio with neural networks is ignored ...

It uses supervised learning (just like the deuce) with different NN architecture, search, backend etc...

Hurts to be ignored after investing so much time in it ... is it because I am black :) :) (obviously I am joking here)
True, I would have preferred Scorpio much more than Deus X. At least it's a completely different an original approach _and_ engine.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls

AndrewGrant
Posts: 494
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by AndrewGrant » Tue Jul 31, 2018 10:41 pm

CMCanavessi wrote:
Tue Jul 31, 2018 10:36 pm
Daniel Shawul wrote:
Tue Jul 31, 2018 10:28 pm
Graham Banks wrote:
Tue Jul 31, 2018 1:33 am
Nay Lin Tun wrote:
Sun Jul 29, 2018 3:58 pm
Graham Banks wrote:
Sun Jul 29, 2018 6:30 am
Nay Lin Tun wrote:
Sun Jul 29, 2018 6:24 am
Well, there is extreme high possibility that Deus X authour is Deep Junior Authour, Shay Bushinsky.
https://www.aaai.org/ojs/index.php/aima ... /view/2255

So you will be seeing updated version(work of art) of Junior!!
:D :)
No - you're wrong, but my lips are sealed. :wink:
What about Tencent company?
https://www.chess.com/forum/view/genera ... -season-13
http://www.chessdom.com/deus-x-the-nn-c ... rt-silver/
Meanwhile, Scorpio with neural networks is ignored ...

It uses supervised learning (just like the deuce) with different NN architecture, search, backend etc...

Hurts to be ignored after investing so much time in it ... is it because I am black :) :) (obviously I am joking here)
True, I would have preferred Scorpio much more than Deus X. At least it's a completely different an original approach _and_ engine.
I too would prefer Scorpio. I have been ragging on it because of the crashes, but putting that aside -- If TCEC is actually interested in showcasing a diverse suite of emerging Neural Networks, Leela + Scorpio is what you choose. We have Leela doing an OpenSource version of AlphaZero, and we have Scorpio trying to merge the idea into the existing chess engine literature. Two applications of NN, two VERY different approaches, two different results, two reasons to watch TCEC.

jkiliani
Posts: 143
Joined: Wed Jan 17, 2018 12:26 pm

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by jkiliani » Tue Jul 31, 2018 11:06 pm

Daniel Shawul wrote:
Tue Jul 31, 2018 10:28 pm
Meanwhile, Scorpio with neural networks is ignored ...

It uses supervised learning (just like the deuce) with different NN architecture, search, backend etc...

Hurts to be ignored after investing so much time in it ... is it because I am black :) :) (obviously I am joking here)
I still haven't seen any information who the last three nominations for Div 4 are going to be... having Scorpio-NN included would have been nice, but I presume you haven't been approached?

AndrewGrant
Posts: 494
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by AndrewGrant » Tue Jul 31, 2018 11:11 pm

jkiliani wrote:
Tue Jul 31, 2018 11:06 pm
Daniel Shawul wrote:
Tue Jul 31, 2018 10:28 pm
Meanwhile, Scorpio with neural networks is ignored ...

It uses supervised learning (just like the deuce) with different NN architecture, search, backend etc...

Hurts to be ignored after investing so much time in it ... is it because I am black :) :) (obviously I am joking here)
I still haven't seen any information who the last three nominations for Div 4 are going to be... having Scorpio-NN included would have been nice, but I presume you haven't been approached?
Cato hinted at chess22k, Rodent III, and IvanHoe .... the last of which raises even more questions.
Last edited by AndrewGrant on Tue Jul 31, 2018 11:13 pm, edited 1 time in total.

User avatar
Graham Banks
Posts: 33244
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by Graham Banks » Tue Jul 31, 2018 11:13 pm

AndrewGrant wrote:
Tue Jul 31, 2018 11:11 pm
jkiliani wrote:
Tue Jul 31, 2018 11:06 pm
Daniel Shawul wrote:
Tue Jul 31, 2018 10:28 pm
Meanwhile, Scorpio with neural networks is ignored ...

It uses supervised learning (just like the deuce) with different NN architecture, search, backend etc...

Hurts to be ignored after investing so much time in it ... is it because I am black :) :) (obviously I am joking here)
I still haven't seen any information who the last three nominations for Div 4 are going to be... having Scorpio-NN included would have been nice, but I presume you haven't been approached?
Cato hinted at chess22k, Rodent III, and IvanHoe .... the last of which raises even more questions.
Marvin has also been suggested.
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

Error323
Posts: 10
Joined: Sun Jun 17, 2018 4:35 pm

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by Error323 » Wed Aug 01, 2018 12:12 am

Laskos wrote:
Tue Jul 31, 2018 10:05 pm
jkiliani wrote:
Tue Jul 31, 2018 8:59 pm
CMCanavessi wrote:
Tue Jul 31, 2018 8:35 pm
I think this move may backfire and will hurt TCEC more than it will do good. People are not stupid and want diversity, not 32 clones of SF. Deus X will use lc0 with a custom net. Shouldn't be allowed imho.
I agree, it seems rather difficult to me to justify allowing Deus X while not letting developers of other strong engines enter clones with changed parameters under a new name. Let's see how TCEC spins this once the complaints to this move start coming in...

For Season 13, it seems the decision is made, but for the season after this they'll have to come up with a new set of rules and follow it consistently.
I don't agree. One uses supervised learning, another reinforcement learning. They will play completely different chess. Also, as the 2 groups (main and test) seem to stall at significantly below A0 level (even if put on the same hardware), supervised learning is worth give a try. I hope this Deus X wins the TCEC, and all these discussions will be put to rest.
It might be worth a try for fun, we also did it in the initial phase of lc0 development, see https://github.com/glinscott/leela-ches ... -361063249. We didn't have the fast engine with cudnn then and limited ourselves to 64x6. It was done to make sure the actual network architecture was sane (not just weights) .

I think it's very unlikely that this approach will beat A0 ever. It's based on human games, with its human flaws and limitations. It'll obtain generalized knowledge across many humans which is of course better, but it will still not see beyond human capability (given just a nn forward inference, with MCTS combined it will somewhat).

A nn trained through Selfplay from scratch, however won't be bound by humans. And looking at deepmind results for Go, it far surpassed the supervised approach. We're (lc0) just struggling/learning to find the optimal parameters for training and avoiding overfitting, as it's much more challenging to get right (and actually write the entire engine etc). But do note that our 20b net is still in it's infancy, it'll become interesting as it drops learning-rate and starts fine-tuning its performance.

frankp
Posts: 216
Joined: Sun Mar 12, 2006 2:11 pm

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Post by frankp » Wed Aug 01, 2018 8:37 am

[/quote]

It might be worth a try for fun, we also did it in the initial phase of lc0 development, see https://github.com/glinscott/leela-ches ... -361063249. We didn't have the fast engine with cudnn then and limited ourselves to 64x6. It was done to make sure the actual network architecture was sane (not just weights) .

I think it's very unlikely that this approach will beat A0 ever. It's based on human games, with its human flaws and limitations. It'll obtain generalized knowledge across many humans which is of course better, but it will still not see beyond human capability (given just a nn forward inference, with MCTS combined it will somewhat).

A nn trained through Selfplay from scratch, however won't be bound by humans. And looking at deepmind results for Go, it far surpassed the supervised approach. We're (lc0) just struggling/learning to find the optimal parameters for training and avoiding overfitting, as it's much more challenging to get right (and actually write the entire engine etc). But do note that our 20b net is still in it's infancy, it'll become interesting as it drops learning-rate and starts fine-tuning its performance.
[/quote]

This is the key interest for me. A0 and Leela seem to play chess in a different and superior way to humans - which of course includes standard ABsearch engines that mimic/encode the human approach.

Constraining leela through blinkered 'human expert'' training seems fundamentally an incorrect approach, as seems to have been demonstrated with AlphaZero (Go) as you say.

This is not to say, of course, that a NN trained on human games will not be interesting.

Post Reply