Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
supersharp77
Posts: 884
Joined: Sat Jul 05, 2014 5:54 am
Location: Southwest USA

Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by supersharp77 » Sat May 25, 2019 6:58 am

https://www.chess.com/news/view/compute ... ckfish-lc0

viewtopic.php?f=6&t=70816 :D :wink:

In a thrilling battle that came down to the final games, Stockfish won the Chess.com Computer Chess Championship this week, edging out the machine-learning engine Lc0.

Stockfish scored 82.5/150 to reclaim its championship title and top Lc0, which came in at 81 points. Lc0 had won the last Computer Chess Championship in a historic performance.

The neural-network engine Leelenstein was third with 76/150, and Houdini trailed far behind in fourth at 60.5 points in the CCC 8: Deep Dive finals.

Lc0 defeated Stockfish in their head-to-head match, four wins to three. Lc0 also won its head-to-head matches with the other two finalists. But Stockfish performed better in the four-engine round-robin to take back the title by beating up on the two weaker engines.

User avatar
Laskos
Posts: 10240
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by Laskos » Sat May 25, 2019 7:38 am

supersharp77 wrote:
Sat May 25, 2019 6:58 am
https://www.chess.com/news/view/compute ... ckfish-lc0

viewtopic.php?f=6&t=70816 :D :wink:

In a thrilling battle that came down to the final games, Stockfish won the Chess.com Computer Chess Championship this week, edging out the machine-learning engine Lc0.

Stockfish scored 82.5/150 to reclaim its championship title and top Lc0, which came in at 81 points. Lc0 had won the last Computer Chess Championship in a historic performance.

The neural-network engine Leelenstein was third with 76/150, and Houdini trailed far behind in fourth at 60.5 points in the CCC 8: Deep Dive finals.

Lc0 defeated Stockfish in their head-to-head match, four wins to three. Lc0 also won its head-to-head matches with the other two finalists. But Stockfish performed better in the four-engine round-robin to take back the title by beating up on the two weaker engines.
Head to head matters. Lc0 always underperforms against weaker engines. It will always show deflated rating in rating lists. Lc0 doesn't obey the Elo model in a pool of regular engines. In TCEC Premiere Division, Lc0 came second too, after SF, but in head-to-head Superfinal it is clobbering SF.

User avatar
Ozymandias
Posts: 1182
Joined: Sun Oct 25, 2009 12:30 am

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by Ozymandias » Sat May 25, 2019 8:00 am

Laskos wrote:
Sat May 25, 2019 7:38 am
Lc0 doesn't obey the Elo model in a pool of regular engines. In TCEC Premiere Division, Lc0 came second too, after SF, but in head-to-head Superfinal it is clobbering SF.
It could be that the SF team has worked intensively on contempt since before version 9, while Lc0 doesn't play that game, just chess.

User avatar
Laskos
Posts: 10240
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by Laskos » Sat May 25, 2019 8:15 am

Ozymandias wrote:
Sat May 25, 2019 8:00 am
Laskos wrote:
Sat May 25, 2019 7:38 am
Lc0 doesn't obey the Elo model in a pool of regular engines. In TCEC Premiere Division, Lc0 came second too, after SF, but in head-to-head Superfinal it is clobbering SF.
It could be that the SF team has worked intensively on contempt since before version 9, while Lc0 doesn't play that game, just chess.
Both SF contempt and the misses of Leela against weaker opponents. Leela can easily fail to convert a win against a much weaker engine.

jp
Posts: 1345
Joined: Mon Apr 23, 2018 5:54 am

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by jp » Sat May 25, 2019 8:16 am

Ozymandias wrote:
Sat May 25, 2019 8:00 am
It could be that the SF team has worked intensively on contempt since before version 9, while Lc0 doesn't play that game, just chess.
Is there really intensive work to be done there, though? You can set it more aggressively (no real work). You can make it adjust dynamically (some work). What else is there to do?
Laskos wrote:
Sat May 25, 2019 8:15 am
Both SF contempt and the misses of Leela against weaker opponents. Leela can easily fail to convert a win against a much weaker engine.
Yes.

User avatar
Ozymandias
Posts: 1182
Joined: Sun Oct 25, 2009 12:30 am

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by Ozymandias » Sat May 25, 2019 12:17 pm

jp wrote:
Sat May 25, 2019 8:16 am
Is there really intensive work to be done there, though?
Comparatively; in the last 18 months, I count 9 related patches. before that... you have to go all the way to the pre-Fishtest era.

User avatar
M ANSARI
Posts: 3435
Joined: Thu Mar 16, 2006 6:10 pm

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by M ANSARI » Sat May 25, 2019 3:35 pm

There is still a lot of easily accessible low lying fruit with regards to Lc0. Lot of areas that can be improved and it is just a matter of time before that happens. Time management is something that needs a serious look with regards to the version that is playing at TCEC and that probably cost Lc0 several unnecessary losses. And although it has improved a lot since last TCEC, there is still some very obvious tactical blindness that causes some blunders and some weak endgame play. I think once there is less ELO to be gained by simply choosing a newer network, these things will start to be worked on. At the moment there still seems a good ways to go with regards to networks. I also think that in TCEC the +10 rule should be changed as +10 for Lc0 does not mean the same as +10 for Komodo or SF or Houdini and on many occasions Lc0 will not find the win even if it is +10 ahead.

Modern Times
Posts: 2554
Joined: Thu Jun 07, 2012 9:02 pm

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by Modern Times » Sat May 25, 2019 4:24 pm

M ANSARI wrote:
Sat May 25, 2019 3:35 pm
I also think that in TCEC the +10 rule should be changed as +10 for Lc0 does not mean the same as +10 for Komodo or SF or Houdini and on many occasions Lc0 will not find the win even if it is +10 ahead.
Yes, I saw the danger with adjudication and Lc0, so all the matches I've played so far have been with adjudication turned off completely. Makes for long tortuous games sometimes that is for sure. I'll find a compromise perhaps.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

jp
Posts: 1345
Joined: Mon Apr 23, 2018 5:54 am

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by jp » Tue May 28, 2019 1:06 pm

Modern Times wrote:
Sat May 25, 2019 4:24 pm
M ANSARI wrote:
Sat May 25, 2019 3:35 pm
I also think that in TCEC the +10 rule should be changed as +10 for Lc0 does not mean the same as +10 for Komodo or SF or Houdini and on many occasions Lc0 will not find the win even if it is +10 ahead.
Yes, I saw the danger with adjudication and Lc0, so all the matches I've played so far have been with adjudication turned off completely. Makes for long tortuous games sometimes that is for sure. I'll find a compromise perhaps.
Yes, this is a big problem with TCEC. A compromise is to have draw adjudication rules but play out wins to checkmate or a TB win.

jorose
Posts: 289
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: Stockfish Beats LC0 to Win 2019 Chess.com Tourney

Post by jorose » Tue May 28, 2019 4:19 pm

jp wrote:
Tue May 28, 2019 1:06 pm
Modern Times wrote:
Sat May 25, 2019 4:24 pm
M ANSARI wrote:
Sat May 25, 2019 3:35 pm
I also think that in TCEC the +10 rule should be changed as +10 for Lc0 does not mean the same as +10 for Komodo or SF or Houdini and on many occasions Lc0 will not find the win even if it is +10 ahead.
Yes, I saw the danger with adjudication and Lc0, so all the matches I've played so far have been with adjudication turned off completely. Makes for long tortuous games sometimes that is for sure. I'll find a compromise perhaps.
Yes, this is a big problem with TCEC. A compromise is to have draw adjudication rules but play out wins to checkmate or a TB win.
I think this is a problem with Leela, not TCEC. The fact that her wpct -> cp formula results in such absurd numbers which don't fit the rather rough values that other engines output are the issue here, not the +10 for 10 rule. If you are -10 for 10 ply you should be hopelessly lost and if not then there is an issue (eg #fortress) with your eval.

The rule used to be -6 for a couple of moves iirc. That worked completely fine and nobody really complained with the very rare exception when the engines both clearly saw the win, but your average chess joe did not. I don't think I myself ever encountered a winning result under those rules that I had to really disagree with. Now can you imagine Leela under those rules?

Imo there is no justification whatsoever to not adjust the formula. You can complain about CP not meaning anything, you don't have to like CP, you don't have to use CP when you run the program at home, but it is a standard that people are used to and you would like to be on a similar scale to all the other engines that added this output.
-Jonathan

Post Reply