Komodo 12.2.1 x64 @ CEGT 3'+1" ponder=on

[ThatsIt]

Hi !

just started.

http://cegt.forumieren.com/t1042-testin ... 12-2-1-x64

First interim results in about 6 hours.

Best wishes,
G.S.
(CEGT team)

[lkaufman]

Although we will be releasing a bugfix 12.2.2 shortly, this should only affect testing on more than one thread, so this test should be fine. I think the closest match for Komodo MCTS of the engines tested should be Ethereal 11.00 (rated 3176); I'll be happy with any rating above that. This would give Komodo MCTS a solid claim to being among the top five engines.

[lkaufman]

So far 3193 after 340 games, comfortably above Ethereal 11.00 in the battle for fifth place among all engine families (fourth if you remove normal Komodo).

[lkaufman]

Still 3193 after about 560 games. I ran about 3250 games at the same time control (but no ponder, which I consider a minor factor) against Ethereal 11, and got +12 elo for MCTS = 3188 on your scale, quite close agreement.

[ThatsIt]

Update, 1000 games are played.

http://cegt.forumieren.com/t1042-testin ... 12-2-1-x64

Best wishes,
G.S.
(CEGT team)

[lkaufman]

So 3188 compared to my result of 3188 vs. Ethereal, difference zero!

[ThatsIt]

Update, 2000 games are played.

http://cegt.forumieren.com/t1042-testin ... 12-2-1-x64

Best wishes,
G.S.
(CEGT team)

[lkaufman]

Update, 2000 games are played.

http://cegt.forumieren.com/t1042-testin ... 12-2-1-x64

Best wishes,
G.S.
(CEGT team)

We have confirmed that there is a bug in 12.2.2 MCTS in MP mode as reported by CCRL, and have a fix for it at least in the position shown. Fortunately this bug does not affect single-thread games such as yours here. We'll post when a fixed version is available.

[lkaufman]

Update, 2000 games are played.

http://cegt.forumieren.com/t1042-testin ... 12-2-1-x64

Best wishes,
G.S.
(CEGT team)

I see that the final result was precisely one elo point above the 3188 result I estimated from my own tests. Since I test almost exclusively with increments, it makes sense that my results correlate much better with your 3' + 1" list than with repeating time control lists. I think that the MCTS Komodo does better with increments than with 40/x levels, not sure yet why.

[ThatsIt]

Hi Larry,

I see that the final result was precisely one elo point above the 3188 result I estimated from my own tests. Since I test almost exclusively with increments, it makes sense that my results correlate much better with your 3' + 1" list than with repeating time control lists. I think that the MCTS Komodo does better with increments than with 40/x levels, not sure yet why.

Yes, it looks like. But keep in mind that we
have only 1500 games for the 40/4-list so far.

The CEGT 3'+1" ponder=on update is online now:

http://www.cegt.net/rating3plus1pbon.htm

Code: Select all

...
Komodo 12.1.1 x64    3351  +- 9
Fire 7.1 x64         3242  +- 8
Komodo 12.2.1 MCTS   3190  +- 8
Ethereal 11.00 x64   3178  +- 8
Deep Shredder 13 x64 3150  +- 8
...

Best wishes,
G.S.
(CEGT team)

Any idea as to the MP-fix for version 12.2.2 MCTS ?