Komodo 12.2 x64 (MCTS) @ CEGT 40/4 + ...

[ThatsIt]

Hi !

Just started.

http://cegt.forumieren.com/t1043-testin ... -mcts#2184

Best wishes,
G.S.
(CEGT team)

[lkaufman]

I better make my prediction before any results are posted. I think it will be a bit over 3200 on your scale, I'll say 3210. This would be more than a 250 elo gain from the last release if I am correct. We shall see.

[ThatsIt]

Hi Larry,

I better make my prediction before any results are posted. I think it will be a bit over 3200 on your scale, I'll say 3210. This would be more than a 250 elo gain from the last release if I am correct. We shall see.

Sure?
You wrote:
"The strength of the MCTS version is vastly improved, up around 200 elo points at blitz levels ..."

But, wait and see ...

Best wishes and many THX for the new version,
G.S.
(CEGT team)

[lkaufman]

Hi Larry,

I better make my prediction before any results are posted. I think it will be a bit over 3200 on your scale, I'll say 3210. This would be more than a 250 elo gain from the last release if I am correct. We shall see.

Sure?
You wrote:
"The strength of the MCTS version is vastly improved, up around 200 elo points at blitz levels ..."

But, wait and see ...

Best wishes and many THX for the new version,
G.S.
(CEGT team)

I was looking at CCRL ratings when I said 200, I think your ratings are further spread because CCRL uses bayeselo which contracts the rating scale somewhat. Komodo 12.2 MCTS on one thread has come out slightly ahead of Stockfish 6 and Komodo 9.1 in our tests, that's the basis of my forecast.

[lkaufman]

Hi Larry,

I better make my prediction before any results are posted. I think it will be a bit over 3200 on your scale, I'll say 3210. This would be more than a 250 elo gain from the last release if I am correct. We shall see.

Sure?
You wrote:
"The strength of the MCTS version is vastly improved, up around 200 elo points at blitz levels ..."

But, wait and see ...

Best wishes and many THX for the new version,
G.S.
(CEGT team)

I note that your current result shows a gain of EXACTLY 250 elo points at this writing! Overnight I ran the MCTS version against Komodo 9 under your test conditions (first time I ever ran the MCTS version at repeating tc) and got a win for MCTS by 130 to 126, which works out to 3203 on your 40/4 list, just four elo above your current result. Very nice agreement so far.

[lkaufman]

It looks like we will have a bugfix 12.21 in a day or two. As far as I know now, it will only affect testing on the Fritz gui, which is reporting Komodo mates as wins for the wrong side. So no need to stop testing unless you are using the Fritz gui (or any gui that shows Komodo losing when it gave checkmate). If you are using Fritz gui, you can either throw out the result or just check to see if there were mates credited to the wrong side, and fix the score if so.

[ThatsIt]

Update, 500 games are played.

http://cegt.forumieren.com/t1043-testin ... o-12-2-x64

+ 214 to version 12.1.1 so far !

Best wishes,
G.S.
(CEGT team)

[ThatsIt]

Hi Larry !

I always use the Shredder Classic GUI for testing.

Best wishes,
G.S.
(CEGT team)

[lkaufman]

Hi Larry !

I always use the Shredder Classic GUI for testing.

Best wishes,
G.S.
(CEGT team)

I'd like to confirm some details of testing and ask a couple questions, so that I can test the same way as much as possible or at least note differences. I believe that you (and the other testers) always test with hyperthreading off, and use a time control of forty moves in 3 minutes on your machines for the "40/4" list, and you set Hash to 256 Mb. Is this correct so far? Now for my questions: Do you set threads to equal cores, or do you set threads to equal cores minus one? How much memory is the minimum allowed or used for a test machine? This could potentially be an issue with the MCTS Komodo on a laptop for example. Finally, does anyone use the Fritz gui as far as you know?

Best regards,
Larry

[ThatsIt]

Hi Larry,

I believe that you (and the other testers) always test with hyperthreading off,
and use a time control of forty moves in 3 minutes on your machines for the
"40/4" list, and you set Hash to 256 Mb.

Thats correct.
I myself don't own a cpu with hyperthreading.

How much memory is the minimum allowed or used for a test machine?

All my machines have 4 or more GB RAM.

This could potentially be an issue with the MCTS Komodo on a laptop for example.

I would never use a notebook or laptop for testing.
Those devices have no sufficient cooling management.

Finally, does anyone use the Fritz gui as far as you know?

Not for our 40/4 + .

Best wishes,
G.S.
(CEGT team)