Yes, we reduced Contempt substantially because we are so often paired now with NN engines and Stockfish derivatives of top strength. It will hurt us against the opponents rated 200 below us. So the closer our average opponent is to our rating, the better K14 should do relative to earlier versions.Leto wrote: ↑Fri May 15, 2020 5:25 pmThe 3375 performance result against Stockfish 11 makes sense, in comparison Komodo 13.3 (3370) only managed a 3320 performance against Stockfish 11. The performance against the weaker opposition such as Boot 6.4 (rated almost 200 elo below K13.3), Ginkgo 2.18 (rated 200 elo below K13.3), and Slowchess Classic 2.1 (rated 226 elo below K13.3) can be due to contempt issues. I imagine once the matches against the top end of the competitors are completed the rating will rise to expected levels.Wolfgang wrote: ↑Fri May 15, 2020 10:41 am Testrun for CEGT 5'+3" with Ponder is running, but rather disappointing so far. 400 games only, but -26 to 13.3 is not what I expected...
Let's wait and see
https://cegt.forumieren.com/t1297-testing-komodo-14-0
Komodo 14 released.
Moderators: hgm, Rebel, chrisw
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Komodo 14 released.
Komodo rules!
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Komodo 14 released.
I am getting a bit better result in ultra-fast games in Armageddon chess and scoring with that option, but significantly worse "regular" score:lkaufman wrote: ↑Wed May 13, 2020 12:17 am We have released Komodo 14 at komodochess.com. It is a significant strength improvement over 13.3, the last release, about 20 elo in MCTS mode and 10 elo in standard mode. Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode. The improvements cover a wide range of issues, including extensions, null move, King Safety, time management, and many more. There is one new feature added since Komodo 13.3, "Armageddon" mode, which tells Komodo that White (or Black) must win, draws are scored as losses for that color. When draws are counted as losses for White, knowing this improves Komodo's performance as White by about 30 elo according to our tests. In Armageddon mode, the specified color (usually White) not only avoids clear draws, but adjusts Contempt by an amount we determined was most helpful for Armageddon play. Testing on one thread in Armageddon mode suggests that on a modern laptop, if White gets one minute total plus 1% increment, and Black gets half a minute plus 1% increment plus draw odds, then chances are fairly close to balanced. Other features added since Komodo 13 include more levels, Personalities, and Auto-Skill. Standard mode remains the strongest when Multi-PV is not in use, while MCTS mode is generally strongest when both are using Multi-PV.
As usual, Komodo 14 is free for subscribers and available for a 20% discount to those who bought Komodo 13 or any later version from us. Komodo 11 will become free, and Komodo 12 and 13 will go on sale.
2000 games each match self games of K14 1/White Must Win and 2/Regular Komodo
NBSC chess:
1/ +947 =627 -426 White Must Win
2/ +933 =940 -127 Off
It seems the little improvement in White win rate gives very many White losses from a borderline White Win / Draw position.
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Komodo 14 released.
Laskos wrote: ↑Fri May 15, 2020 6:27 pmI am getting a bit better result in ultra-fast games in Armageddon chess and scoring with that option, but significantly worse "regular" score:lkaufman wrote: ↑Wed May 13, 2020 12:17 am We have released Komodo 14 at komodochess.com. It is a significant strength improvement over 13.3, the last release, about 20 elo in MCTS mode and 10 elo in standard mode. Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode. The improvements cover a wide range of issues, including extensions, null move, King Safety, time management, and many more. There is one new feature added since Komodo 13.3, "Armageddon" mode, which tells Komodo that White (or Black) must win, draws are scored as losses for that color. When draws are counted as losses for White, knowing this improves Komodo's performance as White by about 30 elo according to our tests. In Armageddon mode, the specified color (usually White) not only avoids clear draws, but adjusts Contempt by an amount we determined was most helpful for Armageddon play. Testing on one thread in Armageddon mode suggests that on a modern laptop, if White gets one minute total plus 1% increment, and Black gets half a minute plus 1% increment plus draw odds, then chances are fairly close to balanced. Other features added since Komodo 13 include more levels, Personalities, and Auto-Skill. Standard mode remains the strongest when Multi-PV is not in use, while MCTS mode is generally strongest when both are using Multi-PV.
As usual, Komodo 14 is free for subscribers and available for a 20% discount to those who bought Komodo 13 or any later version from us. Komodo 11 will become free, and Komodo 12 and 13 will go on sale.
2000 games each match self games of K14 1/White Must Win and 2/Regular Komodo
NBSC chess:
1/ +947 =627 -426 White Must Win
2/ +933 =940 -127 Off
It seems the little improvement in White win rate gives very many White losses from a borderline White Win / Draw position.
[/quote
When you ran the White Must Win games, was that option set for both colors, or just for White? Probably it was for both colors. If so then White would be pushing to win at all costs, but Black would be countering this by pushing to draw, so it's not particularly clear which side should prevail with Armageddon scoring. My testing indicated that if "White Must Win" was set for White only, then in games where it played White it showed a significant (around 30 elo) gain with Armageddon scoring. But if Black also had this setting, the White gains vanished. Knowing the Armageddon situation helps both sides.
Komodo rules!
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Komodo 14 released.
Both colors knew. Armageddon White scoring still improved a bit, but at a huge cost as normal White scoring goes.lkaufman wrote: ↑Fri May 15, 2020 6:46 pmLaskos wrote: ↑Fri May 15, 2020 6:27 pmI am getting a bit better result in ultra-fast games in Armageddon chess and scoring with that option, but significantly worse "regular" score:lkaufman wrote: ↑Wed May 13, 2020 12:17 am We have released Komodo 14 at komodochess.com. It is a significant strength improvement over 13.3, the last release, about 20 elo in MCTS mode and 10 elo in standard mode. Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode. The improvements cover a wide range of issues, including extensions, null move, King Safety, time management, and many more. There is one new feature added since Komodo 13.3, "Armageddon" mode, which tells Komodo that White (or Black) must win, draws are scored as losses for that color. When draws are counted as losses for White, knowing this improves Komodo's performance as White by about 30 elo according to our tests. In Armageddon mode, the specified color (usually White) not only avoids clear draws, but adjusts Contempt by an amount we determined was most helpful for Armageddon play. Testing on one thread in Armageddon mode suggests that on a modern laptop, if White gets one minute total plus 1% increment, and Black gets half a minute plus 1% increment plus draw odds, then chances are fairly close to balanced. Other features added since Komodo 13 include more levels, Personalities, and Auto-Skill. Standard mode remains the strongest when Multi-PV is not in use, while MCTS mode is generally strongest when both are using Multi-PV.
As usual, Komodo 14 is free for subscribers and available for a 20% discount to those who bought Komodo 13 or any later version from us. Komodo 11 will become free, and Komodo 12 and 13 will go on sale.
2000 games each match self games of K14 1/White Must Win and 2/Regular Komodo
NBSC chess:
1/ +947 =627 -426 White Must Win
2/ +933 =940 -127 Off
It seems the little improvement in White win rate gives very many White losses from a borderline White Win / Draw position.
[/quote
When you ran the White Must Win games, was that option set for both colors, or just for White? Probably it was for both colors. If so then White would be pushing to win at all costs, but Black would be countering this by pushing to draw, so it's not particularly clear which side should prevail with Armageddon scoring. My testing indicated that if "White Must Win" was set for White only, then in games where it played White it showed a significant (around 30 elo) gain with Armageddon scoring. But if Black also had this setting, the White gains vanished. Knowing the Armageddon situation helps both sides.
-
- Posts: 511
- Joined: Sun Apr 26, 2020 11:40 pm
- Full name: Brian D. Smith
Re: Komodo 14 released.
This only concerns 'game play' right? Or is the default contempt for K 14 not optimum for deep analysis of given positions?lkaufman wrote: ↑Fri May 15, 2020 6:09 pmYes, we reduced Contempt substantially because we are so often paired now with NN engines and Stockfish derivatives of top strength. It will hurt us against the opponents rated 200 below us. So the closer our average opponent is to our rating, the better K14 should do relative to earlier versions.Leto wrote: ↑Fri May 15, 2020 5:25 pmThe 3375 performance result against Stockfish 11 makes sense, in comparison Komodo 13.3 (3370) only managed a 3320 performance against Stockfish 11. The performance against the weaker opposition such as Boot 6.4 (rated almost 200 elo below K13.3), Ginkgo 2.18 (rated 200 elo below K13.3), and Slowchess Classic 2.1 (rated 226 elo below K13.3) can be due to contempt issues. I imagine once the matches against the top end of the competitors are completed the rating will rise to expected levels.Wolfgang wrote: ↑Fri May 15, 2020 10:41 am Testrun for CEGT 5'+3" with Ponder is running, but rather disappointing so far. 400 games only, but -26 to 13.3 is not what I expected...
Let's wait and see
https://cegt.forumieren.com/t1297-testing-komodo-14-0
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Komodo 14 released.
Contempt is off by default in analysis mode. Just as in game play, if you think your opponent is weaker (or stronger) you can set appropriate Contempt using the White Contempt option (flipping the sign if you are playing Black). Something like 10% of the estimated Elo difference is a good guess for Contempt in either game play or analysis mode.Cornfed wrote: ↑Fri May 15, 2020 7:38 pmThis only concerns 'game play' right? Or is the default contempt for K 14 not optimum for deep analysis of given positions?lkaufman wrote: ↑Fri May 15, 2020 6:09 pmYes, we reduced Contempt substantially because we are so often paired now with NN engines and Stockfish derivatives of top strength. It will hurt us against the opponents rated 200 below us. So the closer our average opponent is to our rating, the better K14 should do relative to earlier versions.Leto wrote: ↑Fri May 15, 2020 5:25 pmThe 3375 performance result against Stockfish 11 makes sense, in comparison Komodo 13.3 (3370) only managed a 3320 performance against Stockfish 11. The performance against the weaker opposition such as Boot 6.4 (rated almost 200 elo below K13.3), Ginkgo 2.18 (rated 200 elo below K13.3), and Slowchess Classic 2.1 (rated 226 elo below K13.3) can be due to contempt issues. I imagine once the matches against the top end of the competitors are completed the rating will rise to expected levels.Wolfgang wrote: ↑Fri May 15, 2020 10:41 am Testrun for CEGT 5'+3" with Ponder is running, but rather disappointing so far. 400 games only, but -26 to 13.3 is not what I expected...
Let's wait and see
https://cegt.forumieren.com/t1297-testing-komodo-14-0
Komodo rules!
-
- Posts: 896
- Joined: Sat May 13, 2006 1:08 am
Re: Komodo 14 released.
In the meantime 200 additional games are played, 600 in total, still to few of course.lkaufman wrote: ↑Fri May 15, 2020 6:09 pm ...
Yes, we reduced Contempt substantially because we are so often paired now with NN engines and Stockfish derivatives of top strength. It will hurt us against the opponents rated 200 below us. So the closer our average opponent is to our rating, the better K14 should do relative to earlier versions.
Rating gone up 5 points to -21 compared to 13.3, NOT because of a good result vs. a top engine (Houdini, performance 3350 only) but a good result vs. a "weak" engine (Laser, performance 3367) .
In one of my currently running matches vs. Andscacs (even a bit "weaker" than Laser) the performance after 60 of 100 is around 3400...
A slight contradiction in my eyes, but to few games though
-
- Posts: 3550
- Joined: Thu Jun 07, 2012 11:02 pm
Re: Komodo 14 released.
In my FRC testing, Komodo 14 MCTS played 5 x weaker opponents and 5 x stronger opponents, all within +/- 180 Elo so it is a balanced test. I'm not always able to achieve that ! So I have got the claimed +50 Ordo Elo to 13.01
-
- Posts: 12541
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Komodo 14 released.
Turning off contempt for analysis is wonderful. Thanks for that.lkaufman wrote: ↑Fri May 15, 2020 7:44 pm Contempt is off by default in analysis mode. Just as in game play, if you think your opponent is weaker (or stronger) you can set appropriate Contempt using the White Contempt option (flipping the sign if you are playing Black). Something like 10% of the estimated Elo difference is a good guess for Contempt in either game play or analysis mode.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Komodo 14 released.
I played 2,000 ultrafast games K 14 Armageddon versus K 14 regular. White performances are following:lkaufman wrote: ↑Fri May 15, 2020 6:46 pmWhen you ran the White Must Win games, was that option set for both colors, or just for White? Probably it was for both colors. If so then White would be pushing to win at all costs, but Black would be countering this by pushing to draw, so it's not particularly clear which side should prevail with Armageddon scoring. My testing indicated that if "White Must Win" was set for White only, then in games where it played White it showed a significant (around 30 elo) gain with Armageddon scoring. But if Black also had this setting, the White gains vanished. Knowing the Armageddon situation helps both sides.Laskos wrote: ↑Fri May 15, 2020 6:27 pmI am getting a bit better result in ultra-fast games in Armageddon chess and scoring with that option, but significantly worse "regular" score:lkaufman wrote: ↑Wed May 13, 2020 12:17 am We have released Komodo 14 at komodochess.com. It is a significant strength improvement over 13.3, the last release, about 20 elo in MCTS mode and 10 elo in standard mode. Total gains from Komodo 13 (or 13.01) are about 50 elo in MCTS mode and 40 elo in standard mode. The improvements cover a wide range of issues, including extensions, null move, King Safety, time management, and many more. There is one new feature added since Komodo 13.3, "Armageddon" mode, which tells Komodo that White (or Black) must win, draws are scored as losses for that color. When draws are counted as losses for White, knowing this improves Komodo's performance as White by about 30 elo according to our tests. In Armageddon mode, the specified color (usually White) not only avoids clear draws, but adjusts Contempt by an amount we determined was most helpful for Armageddon play. Testing on one thread in Armageddon mode suggests that on a modern laptop, if White gets one minute total plus 1% increment, and Black gets half a minute plus 1% increment plus draw odds, then chances are fairly close to balanced. Other features added since Komodo 13 include more levels, Personalities, and Auto-Skill. Standard mode remains the strongest when Multi-PV is not in use, while MCTS mode is generally strongest when both are using Multi-PV.
As usual, Komodo 14 is free for subscribers and available for a 20% discount to those who bought Komodo 13 or any later version from us. Komodo 11 will become free, and Komodo 12 and 13 will go on sale.
2000 games each match self games of K14 1/White Must Win and 2/Regular Komodo
NBSC chess:
1/ +947 =627 -426 White Must Win
2/ +933 =940 -127 Off
It seems the little improvement in White win rate gives very many White losses from a borderline White Win / Draw position.
K14 Arm White: +490 =278 -232 ||| 62.9%
K14 White: +474 =525 -1 ||| 73.7%
K 14 Arm again comes just a bit better in Armageddon scoring, at a very high cost in normal scoring performance (many more losses). Regular K14 only lost 1 game as White to K 14 Armageddon.