Stockfish TCEC 14 Champion

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Leo
Posts: 785
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo

Re: Stockfish TCEC 14 Champion

Post by Leo » Mon Mar 04, 2019 5:58 pm

jp wrote:
Mon Mar 04, 2019 10:52 am
Gary Internet wrote:
Mon Mar 04, 2019 7:07 am
All they did was change the time control from 7200+15 to 720+3 and remove the opening book.
What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.
I totally agree.
BrainFish-2 190310 bmi2 Elo 3576.

Gary Internet
Posts: 37
Joined: Thu Jan 04, 2018 6:09 pm

Re: Stockfish TCEC 14 Champion

Post by Gary Internet » Tue Mar 05, 2019 6:09 am

Leo wrote:
Mon Mar 04, 2019 5:58 pm
jp wrote:
Mon Mar 04, 2019 10:52 am
Gary Internet wrote:
Mon Mar 04, 2019 7:07 am
All they did was change the time control from 7200+15 to 720+3 and remove the opening book.
What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.
I totally agree.
Problems for who? TCEC ran a 100 game match between the same versions of Lc0 and Stockfish that played in the superfinal. They used the same hardware but reduced the time control from 7200+15 to 720+3. Yes, every game started from the standard initial start position. From what I saw, and I watched a fair bit of the match, it ran very smoothly.

jp
Posts: 644
Joined: Mon Apr 23, 2018 5:54 am

Re: Stockfish TCEC 14 Champion

Post by jp » Tue Mar 05, 2019 7:10 am

Gary Internet wrote:
Tue Mar 05, 2019 6:09 am
Leo wrote:
Mon Mar 04, 2019 5:58 pm
jp wrote:
Mon Mar 04, 2019 10:52 am
What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.
I totally agree.
Problems for who?
Problems getting chess results that mean anything. It's totally flawed.

I didn't mean TCEC would e.g. have problems with computer crashes, etc. I'm sure we can all run meaningless things smoothly.

User avatar
Laskos
Posts: 9227
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish TCEC 14 Champion

Post by Laskos » Tue Mar 05, 2019 7:20 am

Gary Internet wrote:
Tue Mar 05, 2019 6:09 am
Leo wrote:
Mon Mar 04, 2019 5:58 pm
jp wrote:
Mon Mar 04, 2019 10:52 am
Gary Internet wrote:
Mon Mar 04, 2019 7:07 am
All they did was change the time control from 7200+15 to 720+3 and remove the opening book.
What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.
I totally agree.
Problems for who? TCEC ran a 100 game match between the same versions of Lc0 and Stockfish that played in the superfinal. They used the same hardware but reduced the time control from 7200+15 to 720+3. Yes, every game started from the standard initial start position. From what I saw, and I watched a fair bit of the match, it ran very smoothly.
It was discussed on topic of DeemMind paper here too. 100 games from the same position, moreover from the sole starting position Lc0 is trained with, is a very bad practice. The games have large coorrelation as outcomes go due to the sole starting chosen position, and favor Lc0. Even playing from 2moves_v1.pgn, on which SF is trained on, is not that bad, as while maybe favoring SF, they are at least diverse. I usually test on 3-4 mover pgn files of GM games, a thing favoring Lc0 a bit compared to 2moves_v1.pgn openings, but in my view are fair, nobody serious will play most of these 2-mover often silly positions of SF testing framework.

User avatar
M ANSARI
Posts: 3398
Joined: Thu Mar 16, 2006 6:10 pm

Re: Stockfish TCEC 14 Champion

Post by M ANSARI » Tue Mar 05, 2019 7:40 am

TCEC is just an entertainment venue for chess engine lovers. It probably will not give you the most accurate indication of the strongest chess engine as the opening positions are designed to be unbalanced to make as many decisive games as possible. One of the main reasoning behind opening book choices is to see what year the books caused the percentage of the most decisive results. I think for Lc0 it would be more advantageous if equal opening positions were chosen ... and thus you could say that by choosing certain opening positions, you could dictate who wins TCEC. Obviously that was not the case in TCEC 14 and I am sure no one had a clue which opening lines would favor which engine. I think SF deserved to win as if Lc0 had won with so many glaring weaknesses that even 2000 ELO engines don't have ... well that would have been scandalous. In the end the more mature engine won out (just barely) and what is clear is that once Lc0 fixes a few of those weaknesses, it will easily beat any engine out there in any opening position choice. Another fact that works in Lc0 favor is that the hardware for AI is just getting started ... I expect a dramatic improvement in AI hardware which obviously would tilt the balance even more so in Lc0 favor.

jp
Posts: 644
Joined: Mon Apr 23, 2018 5:54 am

Re: Stockfish TCEC 14 Champion

Post by jp » Tue Mar 05, 2019 8:18 am

M ANSARI wrote:
Tue Mar 05, 2019 7:40 am
I think SF deserved to win as if Lc0 had won with so many glaring weaknesses that even 2000 ELO engines don't have ... well that would have been scandalous. In the end the more mature engine won out (just barely) and what is clear is that once Lc0 fixes a few of those weaknesses, it will easily beat any engine out there in any opening position choice.
There are ideas how to fix Lc0's endgame weakness, but we don't know if they'll work and they are non-zero ideas. If they (TCEC, etc.) give Lc0 TBs, obviously there's no reason not to give other engines opening books.

As you say, it's about entertainment for CCC & TCEC, and entertainment means handicapping to make it all close.

Leo
Posts: 785
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo

Re: Stockfish TCEC 14 Champion

Post by Leo » Tue Mar 05, 2019 4:17 pm

I would like to see a match that allows both engines to use any book they want. Like a freestyle match. I think SF would do well if its given a powerful book.
BrainFish-2 190310 bmi2 Elo 3576.

User avatar
M ANSARI
Posts: 3398
Joined: Thu Mar 16, 2006 6:10 pm

Re: Stockfish TCEC 14 Champion

Post by M ANSARI » Tue Mar 05, 2019 6:45 pm

Theoretically, a powerful book will beat any other engine without a powerful book. I think a powerful book takes away from an engine playing chess as it depends completely on the book maker's ability. Better would be to have an opening and both sides have to play colors reversed.

lovetb
Posts: 16
Joined: Thu Dec 13, 2018 5:57 am
Full name: MR PRASHANTH KRISHNACHARI

Re: Stockfish TCEC 14 Champion

Post by lovetb » Thu Mar 07, 2019 2:12 pm

Guenther wrote:
Mon Mar 04, 2019 11:46 am

I have a script, which creates nice column style traditional pgn output from the TCEC format.
The file is attached. Note that I was too lazy to work around the unusual 'timeleft', thus only score/depth is included.
Can you please share the script ? It looks great. The pgn is so easily readable.

Post Reply