Test with Stockfish and Komodo with different contempt

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Carlos777
Posts: 1737
Joined: Sun Dec 13, 2009 6:09 pm

Test with Stockfish and Komodo with different contempt

Post by Carlos777 »

I did a little experiment with contempt for Stockfish and Komodo. 1 thread and 20" + 0.02" games, same test suite of 100 starting positions for each match.
Stockfish with contempt 15 is 20 Elo stronger than SF with contempt 0 (default.) Komodo 10.1 with contempt 10 (default) is 8 Elo better than K with contempt 0.
These results are a surprise to me, because I thought that contempt=0 was better against stronger opponents. Maybe someone with better resources can do something similar at longer time controls.

Code: Select all

   # Player                  :      Elo  Games  Score%  won  draw  lost  Points  Draw%  Error   OppAvg   OppE   OppN
   1 Stockfish 131016 C15    :  3369.05   1000    76.0  595   329    76   759.5   32.9  22.12  3150.83  17.67     10
   2 Stockfish 131016 C0     :  3349.08   1000    74.0  550   379    71   739.5   37.9  15.18  3150.83  17.67     10
   3 Komodo 10.1 C10         :  3269.86   1000    60.0  427   345   228   599.5   34.5  16.79  3188.14  19.27     10
   4 Komodo 10.1 C0          :  3261.70   1000    59.0  393   394   213   590.0   39.4  12.53  3188.14  19.57      7
   5 Houdini 4               :  3141.43    800    27.9   89   268   443   223.0   33.5  21.74  3312.42  16.66      4
   6 Gull 3                  :  3050.00    800    18.8   32   237   531   150.5   29.6  18.88  3312.42  16.66      4
   7 Fizbo 1.8               :  3031.16    800    17.3   36   204   560   138.0   25.5  18.42  3312.42  16.66      4

White advantage = 35.62 +/- 4.29
Draw rate (equal opponents) = 46.07 % +/- 0.74

Code: Select all

Head to head statistics:

1) Stockfish 131016 C15 3369.05 :   1000 (+595,=329,-76),  76.0 %

   vs.                          :  games (   +,   =,  -),   (%) :     Diff,     SD, CFS (%)
   Komodo 10.1 C10              :    200 (  89,  82, 29),  65.0 :   +99.19,  16.53,  100.0
   Komodo 10.1 C0               :    200 (  90,  88, 22),  67.0 :  +107.34,  12.73,  100.0
   Houdini 4                    :    200 ( 115,  68, 17),  74.5 :  +227.61,  17.26,  100.0
   Gull 3                       :    200 ( 145,  50,  5),  85.0 :  +319.05,  16.40,  100.0
   Fizbo 1.8                    :    200 ( 156,  41,  3),  88.3 :  +337.89,  14.60,  100.0

2) Stockfish 131016 C0  3349.08 :   1000 (+550,=379,-71),  74.0 %

   vs.                          :  games (   +,   =,  -),   (%) :     Diff,     SD, CFS (%)
   Komodo 10.1 C10              :    200 (  80,  91, 29),  62.8 :   +79.23,   8.21,  100.0
   Komodo 10.1 C0               :    200 (  75, 108, 17),  64.5 :   +87.38,  10.39,  100.0
   Houdini 4                    :    200 ( 124,  60, 16),  77.0 :  +207.65,  16.71,  100.0
   Gull 3                       :    200 ( 134,  63,  3),  82.8 :  +299.08,  13.40,  100.0
   Fizbo 1.8                    :    200 ( 137,  57,  6),  82.8 :  +317.93,  10.46,  100.0

3) Komodo 10.1 C10      3269.86 :   1000 (+427,=345,-228),  60.0 %

   vs.                          :  games (   +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  29,  82,  89),  35.0 :   -99.19,  16.53,    0.0
   Stockfish 131016 C0          :    200 (  29,  91,  80),  37.3 :   -79.23,   8.21,    0.0
   Houdini 4                    :    200 (  98,  69,  33),  66.3 :  +128.42,  16.15,  100.0
   Gull 3                       :    200 ( 129,  59,  12),  79.3 :  +219.86,  13.14,  100.0
   Fizbo 1.8                    :    200 ( 142,  44,  14),  82.0 :  +238.70,  13.12,  100.0

4) Komodo 10.1 C0       3261.70 :   1000 (+393,=394,-213),  59.0 %

   vs.                          :  games (   +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  22,  88,  90),  33.0 :  -107.34,  12.73,    0.0
   Stockfish 131016 C0          :    200 (  17, 108,  75),  35.5 :   -87.38,  10.39,    0.0
   Houdini 4                    :    200 ( 106,  71,  23),  70.8 :  +120.27,  11.00,  100.0
   Gull 3                       :    200 ( 123,  65,  12),  77.8 :  +211.70,  12.62,  100.0
   Fizbo 1.8                    :    200 ( 125,  62,  13),  78.0 :  +230.55,  12.98,  100.0

5) Houdini 4            3141.43 :    800 (+89,=268,-443),  27.9 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 ( 17,  68, 115),  25.5 :  -227.61,  17.26,    0.0
   Stockfish 131016 C0          :    200 ( 16,  60, 124),  23.0 :  -207.65,  16.71,    0.0
   Komodo 10.1 C10              :    200 ( 33,  69,  98),  33.8 :  -128.42,  16.15,    0.0
   Komodo 10.1 C0               :    200 ( 23,  71, 106),  29.3 :  -120.27,  11.00,    0.0

6) Gull 3               3050.00 :    800 (+32,=237,-531),  18.8 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  5,  50, 145),  15.0 :  -319.05,  16.40,    0.0
   Stockfish 131016 C0          :    200 (  3,  63, 134),  17.3 :  -299.08,  13.40,    0.0
   Komodo 10.1 C10              :    200 ( 12,  59, 129),  20.8 :  -219.86,  13.14,    0.0
   Komodo 10.1 C0               :    200 ( 12,  65, 123),  22.3 :  -211.70,  12.62,    0.0

7) Fizbo 1.8            3031.16 :    800 (+36,=204,-560),  17.3 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  3,  41, 156),  11.8 :  -337.89,  14.60,    0.0
   Stockfish 131016 C0          :    200 (  6,  57, 137),  17.3 :  -317.93,  10.46,    0.0
   Komodo 10.1 C10              :    200 ( 14,  44, 142),  18.0 :  -238.70,  13.12,    0.0
   Komodo 10.1 C0               :    200 ( 13,  62, 125),  22.0 :  -230.55,  12.98,    0.0
Top
shrapnel
Posts: 1339
Joined: Fri Nov 02, 2012 9:43 am
Location: New Delhi, India

Re: Test with Stockfish and Komodo with different contempt

Post by shrapnel »

Carlos777 wrote: These results are a surprise to me, because I thought that contempt=0 was better against stronger opponents. Maybe someone with better resources can do something similar at longer time controls.

[
Good Test.
Not really a surprise for me as far as Komodo is concerned because I too have seen in my online matches that Contempt 0 is quite bad for Komodo, at least for Komodo 10.1.
Earlier versions of Komodo were quite strong with Contempt 0, but not this one.
I'm not so sure about Stockfish with Contempt 15; results are not consistent with Contempt > 0 ; sometimes it just weakens the engine with no positive gain.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis
Top
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Test with Stockfish and Komodo with different contempt

Post by Frank Quisinsky »

Hi,

worte that ...
In "all" cases I am thinking Contempt with 10 (I think enough, OK maybe more is better but I don't know) is better. The stats I posted last week are very clear.

With Komodo 9.42 I made also a smaller test in the past vs. 30 opp. and find out 12 Elo is Contempt = 10 better as 0.

Should be clear, because Stockfish is stronger in the beginng of the games after opening book moves. So the different is higher.

Nice test ...
Not a surprise!

Have fun in testing engines!

And to Komodo 10.1 ... after my stats also to see that Komodo 10.0 is stronger vs. all the opponents I have in my list. If I am looking in results by others maybe the same strenght vs. stronger opponents, not vs. all the opponents I have in my list. 10.0 is furthermore more interesting in playing style ... more aggressive.

Maybe with more as 1 core Komodo 10.1 is stronger, I don't know because more as 1 core is for me not interesting ... if I have interest to find out the pure playing strength of an engine!

Best
Frank
Top
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Test with Stockfish and Komodo with different contempt

Post by mwyoung »

Carlos777 wrote:I did a little experiment with contempt for Stockfish and Komodo. 1 thread and 20" + 0.02" games, same test suite of 100 starting positions for each match.
Stockfish with contempt 15 is 20 Elo stronger than SF with contempt 0 (default.) Komodo 10.1 with contempt 10 (default) is 8 Elo better than K with contempt 0.
These results are a surprise to me, because I thought that contempt=0 was better against stronger opponents. Maybe someone with better resources can do something similar at longer time controls.

Code: Select all

   # Player                  :      Elo  Games  Score%  won  draw  lost  Points  Draw%  Error   OppAvg   OppE   OppN
   1 Stockfish 131016 C15    :  3369.05   1000    76.0  595   329    76   759.5   32.9  22.12  3150.83  17.67     10
   2 Stockfish 131016 C0     :  3349.08   1000    74.0  550   379    71   739.5   37.9  15.18  3150.83  17.67     10
   3 Komodo 10.1 C10         :  3269.86   1000    60.0  427   345   228   599.5   34.5  16.79  3188.14  19.27     10
   4 Komodo 10.1 C0          :  3261.70   1000    59.0  393   394   213   590.0   39.4  12.53  3188.14  19.57      7
   5 Houdini 4               :  3141.43    800    27.9   89   268   443   223.0   33.5  21.74  3312.42  16.66      4
   6 Gull 3                  :  3050.00    800    18.8   32   237   531   150.5   29.6  18.88  3312.42  16.66      4
   7 Fizbo 1.8               :  3031.16    800    17.3   36   204   560   138.0   25.5  18.42  3312.42  16.66      4

White advantage = 35.62 +/- 4.29
Draw rate (equal opponents) = 46.07 % +/- 0.74

Code: Select all

Head to head statistics:

1) Stockfish 131016 C15 3369.05 :   1000 (+595,=329,-76),  76.0 %

   vs.                          :  games (   +,   =,  -),   (%) :     Diff,     SD, CFS (%)
   Komodo 10.1 C10              :    200 (  89,  82, 29),  65.0 :   +99.19,  16.53,  100.0
   Komodo 10.1 C0               :    200 (  90,  88, 22),  67.0 :  +107.34,  12.73,  100.0
   Houdini 4                    :    200 ( 115,  68, 17),  74.5 :  +227.61,  17.26,  100.0
   Gull 3                       :    200 ( 145,  50,  5),  85.0 :  +319.05,  16.40,  100.0
   Fizbo 1.8                    :    200 ( 156,  41,  3),  88.3 :  +337.89,  14.60,  100.0

2) Stockfish 131016 C0  3349.08 :   1000 (+550,=379,-71),  74.0 %

   vs.                          :  games (   +,   =,  -),   (%) :     Diff,     SD, CFS (%)
   Komodo 10.1 C10              :    200 (  80,  91, 29),  62.8 :   +79.23,   8.21,  100.0
   Komodo 10.1 C0               :    200 (  75, 108, 17),  64.5 :   +87.38,  10.39,  100.0
   Houdini 4                    :    200 ( 124,  60, 16),  77.0 :  +207.65,  16.71,  100.0
   Gull 3                       :    200 ( 134,  63,  3),  82.8 :  +299.08,  13.40,  100.0
   Fizbo 1.8                    :    200 ( 137,  57,  6),  82.8 :  +317.93,  10.46,  100.0

3) Komodo 10.1 C10      3269.86 :   1000 (+427,=345,-228),  60.0 %

   vs.                          :  games (   +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  29,  82,  89),  35.0 :   -99.19,  16.53,    0.0
   Stockfish 131016 C0          :    200 (  29,  91,  80),  37.3 :   -79.23,   8.21,    0.0
   Houdini 4                    :    200 (  98,  69,  33),  66.3 :  +128.42,  16.15,  100.0
   Gull 3                       :    200 ( 129,  59,  12),  79.3 :  +219.86,  13.14,  100.0
   Fizbo 1.8                    :    200 ( 142,  44,  14),  82.0 :  +238.70,  13.12,  100.0

4) Komodo 10.1 C0       3261.70 :   1000 (+393,=394,-213),  59.0 %

   vs.                          :  games (   +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  22,  88,  90),  33.0 :  -107.34,  12.73,    0.0
   Stockfish 131016 C0          :    200 (  17, 108,  75),  35.5 :   -87.38,  10.39,    0.0
   Houdini 4                    :    200 ( 106,  71,  23),  70.8 :  +120.27,  11.00,  100.0
   Gull 3                       :    200 ( 123,  65,  12),  77.8 :  +211.70,  12.62,  100.0
   Fizbo 1.8                    :    200 ( 125,  62,  13),  78.0 :  +230.55,  12.98,  100.0

5) Houdini 4            3141.43 :    800 (+89,=268,-443),  27.9 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 ( 17,  68, 115),  25.5 :  -227.61,  17.26,    0.0
   Stockfish 131016 C0          :    200 ( 16,  60, 124),  23.0 :  -207.65,  16.71,    0.0
   Komodo 10.1 C10              :    200 ( 33,  69,  98),  33.8 :  -128.42,  16.15,    0.0
   Komodo 10.1 C0               :    200 ( 23,  71, 106),  29.3 :  -120.27,  11.00,    0.0

6) Gull 3               3050.00 :    800 (+32,=237,-531),  18.8 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  5,  50, 145),  15.0 :  -319.05,  16.40,    0.0
   Stockfish 131016 C0          :    200 (  3,  63, 134),  17.3 :  -299.08,  13.40,    0.0
   Komodo 10.1 C10              :    200 ( 12,  59, 129),  20.8 :  -219.86,  13.14,    0.0
   Komodo 10.1 C0               :    200 ( 12,  65, 123),  22.3 :  -211.70,  12.62,    0.0

7) Fizbo 1.8            3031.16 :    800 (+36,=204,-560),  17.3 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  3,  41, 156),  11.8 :  -337.89,  14.60,    0.0
   Stockfish 131016 C0          :    200 (  6,  57, 137),  17.3 :  -317.93,  10.46,    0.0
   Komodo 10.1 C10              :    200 ( 14,  44, 142),  18.0 :  -238.70,  13.12,    0.0
   Komodo 10.1 C0               :    200 ( 13,  62, 125),  22.0 :  -230.55,  12.98,    0.0
What is interesting is that Stockfish is 100 elo ahead with best settings for both engines in your test. Stockfish progress has been insane of late.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
Top
Carlos777
Posts: 1737
Joined: Sun Dec 13, 2009 6:09 pm

Re: Test with Stockfish and Komodo with different contempt

Post by Carlos777 »

shrapnel wrote: Good Test.
Not really a surprise for me as far as Komodo is concerned because I too have seen in my online matches that Contempt 0 is quite bad for Komodo, at least for Komodo 10.1.
Earlier versions of Komodo were quite strong with Contempt 0, but not this one.
I'm not so sure about Stockfish with Contempt 15; results are not consistent with Contempt > 0 ; sometimes it just weakens the engine with no positive gain.
Thanks.

About Komodo, I have read Mark Lefler saying contempt=0 is best against Stockfish, and this was recently.

About Stockfish inconsistency with contempt>0, I saw another test where asmFish had an excellent result with contempt=15.
I guess more games and longer time control would be needed to confirm this.
Top
Carlos777
Posts: 1737
Joined: Sun Dec 13, 2009 6:09 pm

Re: Test with Stockfish and Komodo with different contempt

Post by Carlos777 »

Frank Quisinsky wrote:Hi,

worte that ...
In "all" cases I am thinking Contempt with 10 (I think enough, OK maybe more is better but I don't know) is better. The stats I posted last week are very clear.
Hi Frank,

Yes, your stats are great, but I needed to test with more games against the stronger engines.
Frank Quisinsky wrote: With Komodo 9.42 I made also a smaller test in the past vs. 30 opp. and find out 12 Elo is Contempt = 10 better as 0.

Nice test ...
Not a surprise!

Have fun in testing engines!

And to Komodo 10.1 ... after my stats also to see that Komodo 10.0 is stronger vs. all the opponents I have in my list. If I am looking in results by others maybe the same strenght vs. stronger opponents, not vs. all the opponents I have in my list. 10.0 is furthermore more interesting in playing style ... more aggressive.

Maybe with more as 1 core Komodo 10.1 is stronger, I don't know because more as 1 core is for me not interesting ... if I have interest to find out the pure playing strength of an engine!

Best
Frank
Thanks, interesting about K 9.42's contempt. I guess it would be worth to also test Komodo 10.

Best,
Carlos
Top
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Test with Stockfish and Komodo with different contempt

Post by mwyoung »

Carlos777 wrote:I did a little experiment with contempt for Stockfish and Komodo. 1 thread and 20" + 0.02" games, same test suite of 100 starting positions for each match.
Stockfish with contempt 15 is 20 Elo stronger than SF with contempt 0 (default.) Komodo 10.1 with contempt 10 (default) is 8 Elo better than K with contempt 0.
These results are a surprise to me, because I thought that contempt=0 was better against stronger opponents. Maybe someone with better resources can do something similar at longer time controls.

Code: Select all

   # Player                  :      Elo  Games  Score%  won  draw  lost  Points  Draw%  Error   OppAvg   OppE   OppN
   1 Stockfish 131016 C15    :  3369.05   1000    76.0  595   329    76   759.5   32.9  22.12  3150.83  17.67     10
   2 Stockfish 131016 C0     :  3349.08   1000    74.0  550   379    71   739.5   37.9  15.18  3150.83  17.67     10
   3 Komodo 10.1 C10         :  3269.86   1000    60.0  427   345   228   599.5   34.5  16.79  3188.14  19.27     10
   4 Komodo 10.1 C0          :  3261.70   1000    59.0  393   394   213   590.0   39.4  12.53  3188.14  19.57      7
   5 Houdini 4               :  3141.43    800    27.9   89   268   443   223.0   33.5  21.74  3312.42  16.66      4
   6 Gull 3                  :  3050.00    800    18.8   32   237   531   150.5   29.6  18.88  3312.42  16.66      4
   7 Fizbo 1.8               :  3031.16    800    17.3   36   204   560   138.0   25.5  18.42  3312.42  16.66      4

White advantage = 35.62 +/- 4.29
Draw rate (equal opponents) = 46.07 % +/- 0.74

Code: Select all

Head to head statistics:

1) Stockfish 131016 C15 3369.05 :   1000 (+595,=329,-76),  76.0 %

   vs.                          :  games (   +,   =,  -),   (%) :     Diff,     SD, CFS (%)
   Komodo 10.1 C10              :    200 (  89,  82, 29),  65.0 :   +99.19,  16.53,  100.0
   Komodo 10.1 C0               :    200 (  90,  88, 22),  67.0 :  +107.34,  12.73,  100.0
   Houdini 4                    :    200 ( 115,  68, 17),  74.5 :  +227.61,  17.26,  100.0
   Gull 3                       :    200 ( 145,  50,  5),  85.0 :  +319.05,  16.40,  100.0
   Fizbo 1.8                    :    200 ( 156,  41,  3),  88.3 :  +337.89,  14.60,  100.0

2) Stockfish 131016 C0  3349.08 :   1000 (+550,=379,-71),  74.0 %

   vs.                          :  games (   +,   =,  -),   (%) :     Diff,     SD, CFS (%)
   Komodo 10.1 C10              :    200 (  80,  91, 29),  62.8 :   +79.23,   8.21,  100.0
   Komodo 10.1 C0               :    200 (  75, 108, 17),  64.5 :   +87.38,  10.39,  100.0
   Houdini 4                    :    200 ( 124,  60, 16),  77.0 :  +207.65,  16.71,  100.0
   Gull 3                       :    200 ( 134,  63,  3),  82.8 :  +299.08,  13.40,  100.0
   Fizbo 1.8                    :    200 ( 137,  57,  6),  82.8 :  +317.93,  10.46,  100.0

3) Komodo 10.1 C10      3269.86 :   1000 (+427,=345,-228),  60.0 %

   vs.                          :  games (   +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  29,  82,  89),  35.0 :   -99.19,  16.53,    0.0
   Stockfish 131016 C0          :    200 (  29,  91,  80),  37.3 :   -79.23,   8.21,    0.0
   Houdini 4                    :    200 (  98,  69,  33),  66.3 :  +128.42,  16.15,  100.0
   Gull 3                       :    200 ( 129,  59,  12),  79.3 :  +219.86,  13.14,  100.0
   Fizbo 1.8                    :    200 ( 142,  44,  14),  82.0 :  +238.70,  13.12,  100.0

4) Komodo 10.1 C0       3261.70 :   1000 (+393,=394,-213),  59.0 %

   vs.                          :  games (   +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  22,  88,  90),  33.0 :  -107.34,  12.73,    0.0
   Stockfish 131016 C0          :    200 (  17, 108,  75),  35.5 :   -87.38,  10.39,    0.0
   Houdini 4                    :    200 ( 106,  71,  23),  70.8 :  +120.27,  11.00,  100.0
   Gull 3                       :    200 ( 123,  65,  12),  77.8 :  +211.70,  12.62,  100.0
   Fizbo 1.8                    :    200 ( 125,  62,  13),  78.0 :  +230.55,  12.98,  100.0

5) Houdini 4            3141.43 :    800 (+89,=268,-443),  27.9 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 ( 17,  68, 115),  25.5 :  -227.61,  17.26,    0.0
   Stockfish 131016 C0          :    200 ( 16,  60, 124),  23.0 :  -207.65,  16.71,    0.0
   Komodo 10.1 C10              :    200 ( 33,  69,  98),  33.8 :  -128.42,  16.15,    0.0
   Komodo 10.1 C0               :    200 ( 23,  71, 106),  29.3 :  -120.27,  11.00,    0.0

6) Gull 3               3050.00 :    800 (+32,=237,-531),  18.8 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  5,  50, 145),  15.0 :  -319.05,  16.40,    0.0
   Stockfish 131016 C0          :    200 (  3,  63, 134),  17.3 :  -299.08,  13.40,    0.0
   Komodo 10.1 C10              :    200 ( 12,  59, 129),  20.8 :  -219.86,  13.14,    0.0
   Komodo 10.1 C0               :    200 ( 12,  65, 123),  22.3 :  -211.70,  12.62,    0.0

7) Fizbo 1.8            3031.16 :    800 (+36,=204,-560),  17.3 %

   vs.                          :  games (  +,   =,   -),   (%) :     Diff,     SD, CFS (%)
   Stockfish 131016 C15         :    200 (  3,  41, 156),  11.8 :  -337.89,  14.60,    0.0
   Stockfish 131016 C0          :    200 (  6,  57, 137),  17.3 :  -317.93,  10.46,    0.0
   Komodo 10.1 C10              :    200 ( 14,  44, 142),  18.0 :  -238.70,  13.12,    0.0
   Komodo 10.1 C0               :    200 ( 13,  62, 125),  22.0 :  -230.55,  12.98,    0.0
Here is a test with your contempt setting.

https://www.youtube.com/watch?v=Rgv-jqcnUG8
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
Top
Carlos777
Posts: 1737
Joined: Sun Dec 13, 2009 6:09 pm

Re: Test with Stockfish and Komodo with different contempt

Post by Carlos777 »

mwyoung wrote: What is interesting is that Stockfish is 100 elo ahead with best settings for both engines in your test. Stockfish progress has been insane of late.
Yes, progress is amazing.
mwyoung wrote: Here is a test with your contempt setting.

https://www.youtube.com/watch?v=Rgv-jqcnUG8
Great! How many games?
Top
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Test with Stockfish and Komodo with different contempt

Post by mwyoung »

Carlos777 wrote:
mwyoung wrote: What is interesting is that Stockfish is 100 elo ahead with best settings for both engines in your test. Stockfish progress has been insane of late.
Yes, progress is amazing.
mwyoung wrote: Here is a test with your contempt setting.

https://www.youtube.com/watch?v=Rgv-jqcnUG8
Great! How many games?
I will go at minimum 100 games.

I have never tested Stockfish with such a high contempt before. At the moment Stockfish seems to like the setting at contempt 15. If this test turns out to show promise. I will do more testing to confirm your findings.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
Top
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Test with Stockfish and Komodo with different contempt

Post by Frank Quisinsky »

Hi Carlos,

more games vs. stronger opponents ...
The results will be the same!

Note:
With higher Contempt = 10 the result will be a clearly higher move average. Possible that Elo goes higher a very little bit but to many games will be boring in endgames. I think ... in my huble opinion ... the secret is to find out a good mix of it. I think for Stockfish in reality is Contempt = 8 enough and for Komodo is Contempt = 10 good.

I tested contempt 15 and 17 too in my list for older SF versions. After my first opinion it should be better yes, but I tested the version I have with Contempt 15 too in my list with smaler quantity of games with contempt 10 and find out only 2 Elo more but 6 moves more is the move average undo mate or draw.

This one is "Out of Proportion".

Please have a look what Contempt do with DiscoCheck (Contempt is 25) or Houdini 4 in cross table by Ed ... move average on the end of the table.

http://www.amateurschach.de/main/cross-tab/v436.htm

Because I will have fun if I am looking live in the games.

Interesting is the question ...

Andscacs with Contempt = 10 ... I am very sure engine goes high with more as 15 Elo and is in the near from Fire and GullChess. Should be absolutely clear if I am looking how many games Andscacs gave fast draw with 3-fold. Andscasc is strong in king attacks / tactic and have a very good eval.

:-)

Contempt = 0 for Komodo and Stockfish is one of the many computer chess myths different people like to build without to tested out in detail.

Computer chess history is full of such myths.

Most of it are building with a bad testing methods nobody can something do with it.

Best and thanks
Frank
Top

Return to “Computer Chess Club: Tournaments and Matches”