Komodo 13.01 MCTS - chess960 results and site updated

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Modern Times
Posts: 2366
Joined: Thu Jun 07, 2012 9:02 pm

Komodo 13.01 MCTS - chess960 results and site updated

Post by Modern Times » Fri May 10, 2019 11:59 am

http://ccrl.chessdom.com/ccrl/404FRC/index.html

No comparison to 12.3 because that was never run for this list. But it looks pretty much as expected.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

lkaufman
Posts: 3641
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by lkaufman » Fri May 10, 2019 4:48 pm

Modern Times wrote:
Fri May 10, 2019 11:59 am
http://ccrl.chessdom.com/ccrl/404FRC/index.html

No comparison to 12.3 because that was never run for this list. But it looks pretty much as expected.
Yes, a result above Fire 7.1 means that Komodo MCTS has succeeded in passing all cpu engines other than Stockfish, Houdini, and regular Komodo, which was the goal of this release.
Komodo rules!

lkaufman
Posts: 3641
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by lkaufman » Fri May 10, 2019 7:22 pm

Modern Times wrote:
Fri May 10, 2019 11:59 am
http://ccrl.chessdom.com/ccrl/404FRC/index.html

No comparison to 12.3 because that was never run for this list. But it looks pretty much as expected.
I notice that the top FRC ratings are more than 200 elo above the standard 40/4 ratings, whereas the engines at the bottom are much lower on FRC. Presumably this is due to a substantially lower draw rate in FRC; if that's the case, this is a strong "selling point" for FRC, which is important now that FIDE has recognized it and is holding a true FRC World Championship. Question: is there some anchor rating that is identical for both standard 40/4 and FRC 40/4?
Komodo rules!

Chessqueen
Posts: 564
Joined: Wed Sep 05, 2018 12:16 am
Full name: Nancy M Pichardo

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by Chessqueen » Fri May 10, 2019 7:55 pm

lkaufman wrote:
Fri May 10, 2019 7:22 pm
Modern Times wrote:
Fri May 10, 2019 11:59 am
http://ccrl.chessdom.com/ccrl/404FRC/index.html

No comparison to 12.3 because that was never run for this list. But it looks pretty much as expected.
I notice that the top FRC ratings are more than 200 elo above the standard 40/4 ratings, whereas the engines at the bottom are much lower on FRC. Presumably this is due to a substantially lower draw rate in FRC; if that's the case, this is a strong "selling point" for FRC, which is important now that FIDE has recognized it and is holding a true FRC World Championship. Question: is there some anchor rating that is identical for both standard 40/4 and FRC 40/4?
Approximately when will you have Komodo NN Engine ready so we can see it here beating Stockfish and Lc0 ? https://tcec.chessdom.com/

lkaufman
Posts: 3641
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by lkaufman » Fri May 10, 2019 9:13 pm

Chessqueen wrote:
Fri May 10, 2019 7:55 pm
lkaufman wrote:
Fri May 10, 2019 7:22 pm
Modern Times wrote:
Fri May 10, 2019 11:59 am
http://ccrl.chessdom.com/ccrl/404FRC/index.html

No comparison to 12.3 because that was never run for this list. But it looks pretty much as expected.
I notice that the top FRC ratings are more than 200 elo above the standard 40/4 ratings, whereas the engines at the bottom are much lower on FRC. Presumably this is due to a substantially lower draw rate in FRC; if that's the case, this is a strong "selling point" for FRC, which is important now that FIDE has recognized it and is holding a true FRC World Championship. Question: is there some anchor rating that is identical for both standard 40/4 and FRC 40/4?
Approximately when will you have Komodo NN Engine ready so we can see it here beating Stockfish and Lc0 ? https://tcec.chessdom.com/
Since other people at chess.com are working on Komodo NN, we (Mark and I) can't tell you when that will be. What I can say is that once we have a Komodo NN that plays high level chess, we will be able to improve it from then on.
Komodo rules!

Modern Times
Posts: 2366
Joined: Thu Jun 07, 2012 9:02 pm

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by Modern Times » Sat May 11, 2019 12:01 pm

lkaufman wrote:
Fri May 10, 2019 7:22 pm
I notice that the top FRC ratings are more than 200 elo above the standard 40/4 ratings, whereas the engines at the bottom are much lower on FRC. Presumably this is due to a substantially lower draw rate in FRC; if that's the case, this is a strong "selling point" for FRC, which is important now that FIDE has recognized it and is holding a true FRC World Championship. Question: is there some anchor rating that is identical for both standard 40/4 and FRC 40/4?
Over the years I've had 2 or 3 goes trying to make the scaling on the list sensible, but given up for the reasons you just describe. Exactly what the anchor point was originally I don't know. But there are engines that are similar rating on the standard 404 list and the 404FRC list, e.g.

Hiarcs 13.2 - FRC 2928, std chess 2932
Cheese 2.0 64-bit - FRC 2759, std chess 2765
Arminius 2018-12-23 64-bit - FRC 2786, std chess 2760
Bobcat 8.0 64-bit FRC 2918, std chess 2960


So roughly around the 2700 - 3000 Elo mark the lists are fairly close.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

JohnWoe
Posts: 74
Joined: Sat Mar 02, 2013 10:31 pm

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by JohnWoe » Sat May 11, 2019 12:36 pm

Thanks for testing Sapeli!

I have to take a look at those games at some point. Sapeli has problems with passed pawns ( I don't want to disable underpromotions etc... ). It's a bit too aggressive.

I don't think this new Sapeli 1.03 is much stronger. As I couldn't find any big elo improvements in my own testing. So I can't claim any ELO number. And 1.02 didn't contain any bad bugs. I needed to clean up its code(search + time manag).

lkaufman
Posts: 3641
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by lkaufman » Sat May 11, 2019 9:57 pm

Modern Times wrote:
Sat May 11, 2019 12:01 pm
lkaufman wrote:
Fri May 10, 2019 7:22 pm
I notice that the top FRC ratings are more than 200 elo above the standard 40/4 ratings, whereas the engines at the bottom are much lower on FRC. Presumably this is due to a substantially lower draw rate in FRC; if that's the case, this is a strong "selling point" for FRC, which is important now that FIDE has recognized it and is holding a true FRC World Championship. Question: is there some anchor rating that is identical for both standard 40/4 and FRC 40/4?
Over the years I've had 2 or 3 goes trying to make the scaling on the list sensible, but given up for the reasons you just describe. Exactly what the anchor point was originally I don't know. But there are engines that are similar rating on the standard 404 list and the 404FRC list, e.g.

Hiarcs 13.2 - FRC 2928, std chess 2932
Cheese 2.0 64-bit - FRC 2759, std chess 2765
Arminius 2018-12-23 64-bit - FRC 2786, std chess 2760
Bobcat 8.0 64-bit FRC 2918, std chess 2960


So roughly around the 2700 - 3000 Elo mark the lists are fairly close.
fr

So it seems that the FRC ratings are expanded from normal by a bit over a 4/3 ratio due to fewer draws. That's quite significant and a huge selling point for FRC. By the way, I noticed that there are 18 FRC positions (including the standard one) that start with kings and rooks on normal squares and hence can be played by any engine without any special rules, castling being normal. So in principle anyone can run FRC tournaments with any engines if matches are 36 games and only those positions are used. I guess instead of chess960 you would call it chess18. It has the appeal that it feels more like normal chess since castling is normal and the rooks start on the edges. But not enough positions to avoid memorizing theory on each one.
Komodo rules!

Modern Times
Posts: 2366
Joined: Thu Jun 07, 2012 9:02 pm

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by Modern Times » Sat May 11, 2019 10:11 pm

The ratings are expanded even further if you run Ordo against the database instead of bayeselo.

A pity more engines don't support chess960. I have heard about shuffle chess but haven't researched that yet. Stefan Pohl at one point produced a pgn of 50 opening positions with no castling that any engine can play. I have that, but have not tried it yet.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

lkaufman
Posts: 3641
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 13.01 MCTS - chess960 results and site updated

Post by lkaufman » Sun May 12, 2019 2:00 am

Modern Times wrote:
Sat May 11, 2019 10:11 pm
The ratings are expanded even further if you run Ordo against the database instead of bayeselo.

A pity more engines don't support chess960. I have heard about shuffle chess but haven't researched that yet. Stefan Pohl at one point produced a pgn of 50 opening positions with no castling that any engine can play. I have that, but have not tried it yet.
Shuffle chess is just FRC with no castling except where legal by normal rules and no requirement that king be between the rooks. So any engine can play that, there are no special rules. I don't understand your comment about the 50 positions. With no special FRC castling rules, most of the 960 (or 1440 if kings don't have to be between rooks) positions have no castling and can be played by any engine. But only 18 allow normal castling. How do you get 50 positions? Maybe ones with castling legal on one side only? That might provide a decent compromise between wanting to stick with normal rules while still allowing castling and yet having enough positions for variety and for long engine matches. It seems to me that shuffle chess with no castling is not a good game, because getting the kings to their optimum positions would require weird, slow maneuvers. Or maybe I should not say it's a bad game, just that it is too different from normal chess. To me the ideal is to find a variant that is most like normal chess but without the possibility of prepared computer opening lines and with less draws. FRC is the prime candidate, now that FIDE has recognized it.
Komodo rules!

Post Reply