Komodo 14 released.

[Nordlandia]

Couple of questions:

With armageddon enabled, Is it intentional that evaluation should not be going downhill if the outcome will be draw. For example if the game turns out be draw even with armageddon enabled, Komodo may or may not display losing evaluation.

Should i leave contempt at defaut against Houdini 6 or maybe increase it, considering the relative age of ~ 2.5 years since last release.

[lkaufman]

Couple of questions:

With armageddon enabled, Is it intentional that evaluation should not be going downhill if the outcome will be draw. For example if the game turns out be draw even with armageddon enabled, Komodo may or may not display losing evaluation.

Should i leave contempt at defaut against Houdini 6 or maybe increase it, considering the relative age of ~ 2.5 years since last release.

If Komodo sees a definite draw when it is set to Must Win, it should show a -20 pawn score. But a drawish endgame will still show a fairly normal score if the draw isn't clear. Default Contempt is optimized for slightly weaker opponent, so it is probably ideal for Houdini 6. Higher Contempt values will score better against engines rated a couple hundred elo below Komodo.

[Modern Times]

Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode.

Yes I have +48 Elo so far for MCTS, or +56 Elo if you use Ordo, on the FRC blitz list. Testing still ongoing.

[lkaufman]

Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode.

Yes I have +48 Elo so far for MCTS, or +56 Elo if you use Ordo, on the FRC blitz list. Testing still ongoing.

We use Ordo for all our testing and estimates, as it corresponds to the way ratings are done in human chess. So our estimated gains should be fairly accurate for CEGT and FastGM, but a bit higher than we would expect CCRL to show on average. Ironically, although BayesElo is not the way human ratings are calculated, when used on engine ratings it is actually a better estimator of human rating differences, because it compensates (partly) for the tendency of engine rating differences to expand. So the choice comes down to whether you want the rating system to correspond to human chess or the actual rating differences!

[Wolfgang]

Testrun for CEGT 5'+3" with Ponder is running, but rather disappointing so far. 400 games only, but -26 to 13.3 is not what I expected...
Let's wait and see

https://cegt.forumieren.com/t1297-testing-komodo-14-0

[Modern Times]

Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode.

Yes I have +48 Elo so far for MCTS, or +56 Elo if you use Ordo, on the FRC blitz list. Testing still ongoing.

We use Ordo for all our testing and estimates, as it corresponds to the way ratings are done in human chess. So our estimated gains should be fairly accurate for CEGT and FastGM, but a bit higher than we would expect CCRL to show on average. Ironically, although BayesElo is not the way human ratings are calculated, when used on engine ratings it is actually a better estimator of human rating differences, because it compensates (partly) for the tendency of engine rating differences to expand. So the choice comes down to whether you want the rating system to correspond to human chess or the actual rating differences!

It is at 1,500 games now on the website. It has lost 5 Elo since the last post, but it still meets your +50 Elo Ordo estimate.

[ChiefPushesWood]

Testrun for CEGT 5'+3" with Ponder is running, but rather disappointing so far. 400 games only, but -26 to 13.3 is not what I expected...
Let's wait and see

https://cegt.forumieren.com/t1297-testing-komodo-14-0

Well thats not good.

Chief

[Leto]

Testrun for CEGT 5'+3" with Ponder is running, but rather disappointing so far. 400 games only, but -26 to 13.3 is not what I expected...
Let's wait and see

https://cegt.forumieren.com/t1297-testing-komodo-14-0

The 3375 performance result against Stockfish 11 makes sense, in comparison Komodo 13.3 (3370) only managed a 3320 performance against Stockfish 11. The performance against the weaker opposition such as Boot 6.4 (rated almost 200 elo below K13.3), Ginkgo 2.18 (rated 200 elo below K13.3), and Slowchess Classic 2.1 (rated 226 elo below K13.3) can be due to contempt issues. I imagine once the matches against the top end of the competitors are completed the rating will rise to expected levels.

[lkaufman]

Testrun for CEGT 5'+3" with Ponder is running, but rather disappointing so far. 400 games only, but -26 to 13.3 is not what I expected...
Let's wait and see

https://cegt.forumieren.com/t1297-testing-komodo-14-0

The 3375 performance result against Stockfish 11 makes sense, in comparison Komodo 13.3 (3370) only managed a 3320 performance against Stockfish 11. The performance against the weaker opposition such as Boot 6.4 (rated almost 200 elo below K13.3), Ginkgo 2.18 (rated 200 elo below K13.3), and Slowchess Classic 2.1 (rated 226 elo below K13.3) can be due to contempt issues. I imagine once the matches against the top end of the competitors are completed the rating will rise to expected levels.

Yes, we reduced Contempt substantially because we are so often paired now with NN engines and Stockfish derivatives of top strength. It will hurt us against the opponents rated 200 below us. So the closer our average opponent is to our rating, the better K14 should do relative to earlier versions.

[Laskos]

We have released Komodo 14 at komodochess.com. It is a significant strength improvement over 13.3, the last release, about 20 elo in MCTS mode and 10 elo in standard mode. Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode. The improvements cover a wide range of issues, including extensions, null move, King Safety, time management, and many more. There is one new feature added since Komodo 13.3, "Armageddon" mode, which tells Komodo that White (or Black) must win, draws are scored as losses for that color. When draws are counted as losses for White, knowing this improves Komodo's performance as White by about 30 elo according to our tests. In Armageddon mode, the specified color (usually White) not only avoids clear draws, but adjusts Contempt by an amount we determined was most helpful for Armageddon play. Testing on one thread in Armageddon mode suggests that on a modern laptop, if White gets one minute total plus 1% increment, and Black gets half a minute plus 1% increment plus draw odds, then chances are fairly close to balanced. Other features added since Komodo 13 include more levels, Personalities, and Auto-Skill. Standard mode remains the strongest when Multi-PV is not in use, while MCTS mode is generally strongest when both are using Multi-PV.
As usual, Komodo 14 is free for subscribers and available for a 20% discount to those who bought Komodo 13 or any later version from us. Komodo 11 will become free, and Komodo 12 and 13 will go on sale.

I am getting a bit better result in ultra-fast games in Armageddon chess and scoring with that option, but significantly worse "regular" score:

2000 games each match self games of K14 1/White Must Win and 2/Regular Komodo

NBSC chess:

1/ +947 =627 -426 White Must Win
2/ +933 =940 -127 Off

It seems the little improvement in White win rate gives very many White losses from a borderline White Win / Draw position.