MCTS-Komodo 12.0 x64 @ CEGT 40/4

[ThatsIt]

Hi !

http://cegt.forumieren.com/t989-testing ... -mcts#2081

Best wishes,
G.S.
(CEGT team)

[lkaufman]

Is the test actually at 40/4, or is it at a faster level equivalent to 40/4 on old hardware? If it's actually 40/4 on an i7 or equivalent that should be okay, but if it is 40/4 on old hardware it's probably too fast for MCTS. We consider four seconds per move average on modern hardware to be the minimum for MCTS Komodo to be rated meaningfully.

[ThatsIt]

Is the test actually at 40/4, or is it at a faster level equivalent to 40/4 on old hardware? If it's actually 40/4 on an i7 or equivalent that should be okay, but if it is 40/4 on old hardware it's probably too fast for MCTS. We consider four seconds per move average on modern hardware to be the minimum for MCTS Komodo to be rated meaningfully.

Hi Larry,

40/3 + ... (our limit) @ fast Intel + AMD.
At the moment it looks like 2830-2840 CEGT-ELO.

Best wishes,
G.S.
(CEGT team)

[lkaufman]

Is the test actually at 40/4, or is it at a faster level equivalent to 40/4 on old hardware? If it's actually 40/4 on an i7 or equivalent that should be okay, but if it is 40/4 on old hardware it's probably too fast for MCTS. We consider four seconds per move average on modern hardware to be the minimum for MCTS Komodo to be rated meaningfully.

Hi Larry,

40/3 + ... (our limit) @ fast Intel + AMD.
At the moment it looks like 2830-2840 CEGT-ELO.

Best wishes,
G.S.
(CEGT team)

The average difference between your 40/4 list and the CCRL 40/40 list for single thread engines in the 2800 to 2900 range on your liset is 124 elo. So my 3000 CCRL 40/40 forecast would equate to 2876 on your 40/4 list. Since your test is almost at the minimum time we give for MCTS (4.5 seconds vs 4 seconds) I suppose a rating in the 2830 to 2840 range is not inconsistent with our estimate. Is there a substantial difference in the speed of the intel and AMD processors you use? If so I'll predict that the faster one (presumably Intel) will show better results for Komodo MCTS than the other one.

Best wishes,
Larry

[Daniel Shawul]

Is the test actually at 40/4, or is it at a faster level equivalent to 40/4 on old hardware? If it's actually 40/4 on an i7 or equivalent that should be okay, but if it is 40/4 on old hardware it's probably too fast for MCTS. We consider four seconds per move average on modern hardware to be the minimum for MCTS Komodo to be rated meaningfully.

Hi Larry,

40/3 + ... (our limit) @ fast Intel + AMD.
At the moment it looks like 2830-2840 CEGT-ELO.

Best wishes,
G.S.
(CEGT team)

The average difference between your 40/4 list and the CCRL 40/40 list for single thread engines in the 2800 to 2900 range on your liset is 124 elo. So my 3000 CCRL 40/40 forecast would equate to 2876 on your 40/4 list. Since your test is almost at the minimum time we give for MCTS (4.5 seconds vs 4 seconds) I suppose a rating in the 2830 to 2840 range is not inconsistent with our estimate. Is there a substantial difference in the speed of the intel and AMD processors you use? If so I'll predict that the faster one (presumably Intel) will show better results for Komodo MCTS than the other one.

Best wishes,
Larry

Komodo 11.3.1 is 3342 elo at CEGT 40/4, and if it indeed finishes up at 2830-40 elo, that is like 510 elos weeker ... hmmm

[lkaufman]

Hi Larry,

40/3 + ... (our limit) @ fast Intel + AMD.
At the moment it looks like 2830-2840 CEGT-ELO.

Best wishes,
G.S.
(CEGT team)

The average difference between your 40/4 list and the CCRL 40/40 list for single thread engines in the 2800 to 2900 range on your liset is 124 elo. So my 3000 CCRL 40/40 forecast would equate to 2876 on your 40/4 list. Since your test is almost at the minimum time we give for MCTS (4.5 seconds vs 4 seconds) I suppose a rating in the 2830 to 2840 range is not inconsistent with our estimate. Is there a substantial difference in the speed of the intel and AMD processors you use? If so I'll predict that the faster one (presumably Intel) will show better results for Komodo MCTS than the other one.

Best wishes,
Larry

Komodo 11.3.1 is 3342 elo at CEGT 40/4, and if it indeed finishes up at 2830-40 elo, that is like 510 elos weeker ... hmmm

Rating differences contract with more time, so a 510 elo gap at 40/4 might be consistent with 340 elo gap (2/3 of 510) at 40/40, ten times as long a TC, especially if the 40/4 is just a bit too fast for MCTS, as it might be on the AMD machines. We'll see.

[lkaufman]

Hi Larry,

40/3 + ... (our limit) @ fast Intel + AMD.
At the moment it looks like 2830-2840 CEGT-ELO.

Best wishes,
G.S.
(CEGT team)

The average difference between your 40/4 list and the CCRL 40/40 list for single thread engines in the 2800 to 2900 range on your liset is 124 elo. So my 3000 CCRL 40/40 forecast would equate to 2876 on your 40/4 list. Since your test is almost at the minimum time we give for MCTS (4.5 seconds vs 4 seconds) I suppose a rating in the 2830 to 2840 range is not inconsistent with our estimate. Is there a substantial difference in the speed of the intel and AMD processors you use? If so I'll predict that the faster one (presumably Intel) will show better results for Komodo MCTS than the other one.

So far CEGT reports a 9 out of 20 score at 40/20 for Komodo MCTS vs Ethereal 9.65, which although not yet on the CEGT list appears on the CCRL lists at 3044 for 40/40 and 3079 at 40/4. So this works out to a performance rating on the CCRL scale (which is significantly higher than the CEGT scale) of 3009 on the 40/40 list or 3044 on the 40/4 list. Since 40/20 is in between these two, neither is totally accurate (and 20 games is a small sample), but my 3000 CCRL estimate on one thread is looking pretty reasonable.

Best wishes,
Larry

Komodo 11.3.1 is 3342 elo at CEGT 40/4, and if it indeed finishes up at 2830-40 elo, that is like 510 elos weeker ... hmmm

[ThatsIt]

Hi !

Some facts based on CCRL-40/4-ratings.

finished matches

Code: Select all

vs Vajolet2 2.4 x64 1CPU [ELO 2998 @ CCRL 40/4] perf=3065

interim results

Code: Select all

vs Hakkapeliitta 3.0 x64 [ELO 2985 @ CCRL 40/4] perf=3009 (88 games)
vs Hiarcs 14 1CPU        [ELO 2986 @ CCRL 40/4] perf=3014 (50 games)
vs Senpai 1.0 x64 1CPU   [ELO 3007 @ CCRL 40/4] perf=3019 (30 games)
vs Naum 4.2 x64 1CPU     [ELO 2988 @ CCRL 40/4] perf=3118 (28 games)

http://cegt.forumieren.com/t989-testing ... -1cpu-mcts

Best wishes,
G.S.
(CEGT team)

[lkaufman]

These results are actually better than I expected at such a fast time control (really 40/3). Were they run on the faster intel or the slower AMD machines? Intel I'm guessing.

[ThatsIt]

These results are actually better than I expected at such a fast time control (really 40/3). Were they run on the faster intel or the slower AMD machines? Intel I'm guessing.

Half and half.
I will try to separate it soon.

Best wishes,
G.S.
(CEGT team)