Komodo 10.2 released.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
h1a8
Posts: 399
Joined: Fri Jun 04, 2010 5:23 am

Re: Komodo 10.2 released.

Post by h1a8 » Mon Oct 31, 2016 11:50 am

Modern Times wrote:My understanding is that the only "official" versions are the ones here:

https://stockfishchess.org/

In terms of unofficial daily builds, take your pick. I get mine here:

http://chess.ultimaiq.net/stockfish.html
Which compile is best for Intel i7 processors (2nd generation or 3 generation)?

Thanks

Werewolf
Posts: 1197
Joined: Thu Sep 18, 2008 8:24 pm

Re: Komodo 10.2 released.

Post by Werewolf » Mon Oct 31, 2016 4:13 pm

lkaufman wrote:
Werewolf wrote:
lkaufman wrote:
S.Taylor wrote:
lkaufman wrote:
What _I_ want to see is how this new komodo 10.2 does vs the SF that played in stage 3 of TCEC, OR, the version of SF that played in Graham Banks matches between sf and komodo.

It is very frustrating to have two moving targets at the same time.

e.g. let's say komodo beats ASM, how will we know if the older SF version, which Graham used, wouldn't have crushed komodo 10.2 like it did to komodo 10.1?

We should first test a moving target with a standing target before we go on to the second moving target.
I think we'll do ok relative to stockfish in tests like Frank's where each opponent plays against many other opponents, but we will not look so good in direct matches with Stockfish. The reasons for this are not all clear, although "Contempt" is a factor. It's up to each person to decide whether a direct match or a rr tournament is a better way to decide which of two engines is stronger. There are arguments both ways.
Larry, for use in IDeA, should contempt be set to zero?
Yes.
Sorry - for IDeA should UCI analysis mode be checked or unchecked?

Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 2:15 am

Re: Komodo 10.2 released.

Post by Dan Cooper » Mon Oct 31, 2016 4:36 pm

h1a8 wrote:
Modern Times wrote:My understanding is that the only "official" versions are the ones here:

https://stockfishchess.org/

In terms of unofficial daily builds, take your pick. I get mine here:

http://chess.ultimaiq.net/stockfish.html
Which compile is best for Intel i7 processors (2nd generation or 3 generation)?

Thanks
Both would use modern. 4th gen minimum for Haswell+.

beram
Posts: 1187
Joined: Wed Jan 06, 2010 2:11 pm

Re: Komodo 10.2 released.

Post by beram » Mon Oct 31, 2016 7:11 pm

lkaufman wrote:
S.Taylor wrote:
lkaufman wrote:We have released komodo 10.2 at komodochess.com. We estimate based on our testing that it is about 22 elo stronger than komodo 10.1 and about 30 elo stronger than komodo 10 at three minutes plus one second increment, on one or many threads. This is the largest elo gain we have had between versions since going to the subscription model. There were so many changes that it is very difficult to say which ones should get most of the credit. Evaluation, search, and time management were all changed significantly. Features (UCI options) are the same as in komodo 10.1. As usual, it is free to subscribers, and available at a 20% discount to those who bought komodo 9 or a later version.
What _I_ want to see is how this new komodo 10.2 does vs the SF that played in stage 3 of TCEC, OR, the version of SF that played in Graham Banks matches between sf and komodo.

It is very frustrating to have two moving targets at the same time.

e.g. let's say komodo beats ASM, how will we know if the older SF version, which Graham used, wouldn't have crushed komodo 10.2 like it did to komodo 10.1?

We should first test a moving target with a standing target before we go on to the second moving target.
I think we'll do ok relative to stockfish in tests like Frank's where each opponent plays against many other opponents, but we will not look so good in direct matches with Stockfish. The reasons for this are not all clear, although "Contempt" is a factor. It's up to each person to decide whether a direct match or a rr tournament is a better way to decide which of two engines is stronger. There are arguments both ways.
Well perhaps you did relatively OK in the past but nowadays... this was latest result by Frank. A clear win for SF 180916 against all, Komodo 10 and 10.1 included
(and besides Komodo 10.1 was performing 11 ELO points below K10 in his Test)
http://www.amateurschach.de/main/cross-tab/v438.htm

Code: Select all

1	SF 18Sep2016 BMI2 x64 C10	xxxxx	33.0	39.0	38.5	36.0	40.0	37.0	36.0	39.5	42.0	39.0	42.5	41.5	42.5	42.5
2	Komodo 10 x64	17.0	xxxxx	32.0	37.0	40.0	36.5	37.0	37.0	40.5	41.0	40.5	36.5	42.0	39.5	39.0
3	Houdini 4 STD B x64	11.0	18.0	xxxxx	26.5	26.0	27.5	32.5	29.5	30.5	34.0	32.0	33.0	28.5	32.0	38.0
4	Fire 4 x64	11.5	13.0	23.5	xxxxx	23.5	25.0	25.0	26.5	32.0	28.5	31.0	29.0	32.5	31.5	32.0
5	GullChess 3.0 BMI2 x64	14.0	10.0	24.0	26.5	xxxxx	31.0	23.5	24.5	28.0	30.5	30.0	32.0	26.5	31.5	32.5
6	Andscacs 0.872 BMI2 x64	10.0	13.5	22.5	25.0	19.0	xxxxx	26.0	25.0	26.0	23.5	26.5	31.5	25.0	28.5	31.0
7	Equinox 3.30 x64	13.0	13.0	17.5	25.0	26.5	24.0	xxxxx	24.0	23.5	26.0	29.0	27.5	28.5	32.5	30.5
8	Fizbo 1.8 BMI2 x64	14.0	13.0	20.5	23.5	25.5	25.0	26.0	xxxxx	29.5	23.5	25.5	27.0	28.0	30.5	29.0
9	Critter 1.6a x64	10.5	9.5	19.5	18.0	22.0	24.0	26.5	20.5	xxxxx	24.0	27.5	26.5	28.0	29.5	28.5
10	Fritz 15 x64	8.0	9.0	16.0	21.5	19.5	26.5	24.0	26.5	26.0	xxxxx	26.5	22.5	25.0	26.5	24.0
11	Nirvanachess 2.3 POP x64	11.0	9.5	18.0	19.0	20.0	23.5	21.0	24.5	22.5	23.5	xxxxx	26.5	27.0	28.5	22.5
12	Hannibal 1.7 x64	7.5	13.5	17.0	21.0	18.0	18.5	22.5	23.0	23.5	27.5	23.5	xxxxx	24.0	24.0	29.5
13	Chiron 3 x64	8.5	8.0	21.5	17.5	23.5	25.0	21.5	22.0	22.0	25.0	23.0	26.0	xxxxx	21.5	22.0
14	Protector 1.9.0 x64	7.5	10.5	18.0	18.5	18.5	21.5	17.5	19.5	20.5	23.5	21.5	26.0	28.5	xxxxx	24.0
15	Texel 1.06 x64	7.5	11.0	12.0	18.0	17.5	19.0	19.5	21.0	21.5	26.0	27.5	20.5	28.0	26.0	xxxxx

beram
Posts: 1187
Joined: Wed Jan 06, 2010 2:11 pm

Re: Komodo 10.2 released.

Post by beram » Mon Oct 31, 2016 7:42 pm

MikeB wrote:
beram wrote:
MikeB wrote:ok, this is my last update for the night:

Code: Select all

84 of 600 games completed...
time control: 18000+300
Date: 10/30/16 : 03:12:46
84 game(s) loaded
Rank Name                      Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 Stockfish 102916-14y       3146   0.0   47   47    42   25.0  59.5   12    4   26  28.6  61.9  3085 
   2 Komodo 10.2 64-bit         3089  56.0   49   49    41   19.5  47.6    9   11   21  22.0  51.2  3104 
   3 Komodo 10.1 64-bit         3088   1.5   47   47    43   20.5  47.7    7    9   27  16.3  62.8  3103 
   4 Stockfish 281016 64 POPC   3077  11.0   49   49    42   19.0  45.2    8   12   22  19.0  52.4  3108 
---------------------------------------------------------------------------------------------------------
84 game(s) loaded
you can check yourself for the latest updates here - about every 25 minutes or so: https://www.dropbox.com/s/q30k3kvul6qsa ... a.txt?dl=1
pgn file: https://www.dropbox.com/s/njswzxuo2u88gp2/all.pgn?dl=1
Thx for testing and download
After 120 games, half point difference between 10.1 and 10.2

Code: Select all

Computer chess game

                              1                                        2                                        3                                        4                                        
1   Stockfish 281016 64 POPC  21.0 - 19.021.5 - 18.524.0 - 16.0**     66.5/120
2   Stockfish 102916-14y      19.0 - 21.024.0 - 16.022.0 - 18.0 **    65.0/120
3   Komodo 10.2 64-bit        18.5 - 21.516.0 - 24.020.0 - 20.0  **   54.5/120
4   Komodo 10.1 64-bit        16.0 - 24.018.0 - 22.020.0 - 20.0   **  54.0/120


after 444 games @ 3 min/3 sec/, the K10.2-K11.1 spread has widen to 17.3 ELO - certainly within the expected range for a limited number of game - will be stopping this run as it appears my last mod did not have a desired affect on SF ...also I misread Larry's comment about contempt and had it set to 15 for this run and I believe it may have impacted K's result against SF

Code: Select all

444 of 600 games completed...
time control: 18000+300
Date: 10/30/16 : 15:39:37
444 game(s) loaded
Rank Name                      Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 Stockfish 281016 64 POPC   3145   0.0   20   20   223  131.5  59.0   64   24  135  28.7  60.5  3085 
   2 Stockfish 102916-14y       3129  16.3   20   20   222  124.0  55.9   56   30  136  25.2  61.3  3090 
   3 Komodo 10.2 64-bit         3072  57.3   20   20   221   98.0  44.3   34   59  128  15.4  57.9  3110 
   4 Komodo 10.1 64-bit         3054  17.3   20   20   222   90.5  40.8   29   70  123  13.1  55.4  3116 
---------------------------------------------------------------------------------------------------------
444 game(s) loaded
I have downloaded the full 600 games by your provided downloadlink
And this is what I get from your all.pgn when I ran it through ELOstat_13

-5 ELO for Komodo 10.2

Code: Select all

Individual statistics:

1 Stockfish 301016 64 POPC  : 3342  300 (+ 67,=214,- 19), 58.0 %

Komodo 10.2 64-bit            : 100 (+ 33,= 64,-  3), 65.0 %
Komodo 10.1 64-bit            : 100 (+ 29,= 64,-  7), 61.0 %
Stockfish 103016-14y          : 100 (+  5,= 86,-  9), 48.0 %

2 Stockfish 103016-14y      : 3335  300 (+ 60,=220,- 20), 56.7 %

Komodo 10.2 64-bit            : 100 (+ 25,= 66,-  9), 58.0 %
Komodo 10.1 64-bit            : 100 (+ 26,= 68,-  6), 60.0 %
Stockfish 301016 64 POPC      : 100 (+  9,= 86,-  5), 52.0 %

3 Komodo 10.1 64-bit        : 3264  300 (+ 25,=209,- 66), 43.2 %

Komodo 10.2 64-bit            : 100 (+ 12,= 77,- 11), 50.5 %
Stockfish 103016-14y          : 100 (+  6,= 68,- 26), 40.0 %
Stockfish 301016 64 POPC      : 100 (+  7,= 64,- 29), 39.0 %

4 Komodo 10.2 64-bit        : 3259  300 (+ 23,=207,- 70), 42.2 %

Komodo 10.1 64-bit            : 100 (+ 11,= 77,- 12), 49.5 %
Stockfish 103016-14y          : 100 (+  9,= 66,- 25), 42.0 %
Stockfish 301016 64 POPC      : 100 (+  3,= 64,- 33), 35.0 %

mjlef
Posts: 1427
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Komodo 10.2 released.

Post by mjlef » Mon Oct 31, 2016 9:19 pm

beram wrote:
lkaufman wrote:
S.Taylor wrote:
lkaufman wrote:We have released komodo 10.2 at komodochess.com. We estimate based on our testing that it is about 22 elo stronger than komodo 10.1 and about 30 elo stronger than komodo 10 at three minutes plus one second increment, on one or many threads. This is the largest elo gain we have had between versions since going to the subscription model. There were so many changes that it is very difficult to say which ones should get most of the credit. Evaluation, search, and time management were all changed significantly. Features (UCI options) are the same as in komodo 10.1. As usual, it is free to subscribers, and available at a 20% discount to those who bought komodo 9 or a later version.
What _I_ want to see is how this new komodo 10.2 does vs the SF that played in stage 3 of TCEC, OR, the version of SF that played in Graham Banks matches between sf and komodo.

It is very frustrating to have two moving targets at the same time.

e.g. let's say komodo beats ASM, how will we know if the older SF version, which Graham used, wouldn't have crushed komodo 10.2 like it did to komodo 10.1?

We should first test a moving target with a standing target before we go on to the second moving target.
I think we'll do ok relative to stockfish in tests like Frank's where each opponent plays against many other opponents, but we will not look so good in direct matches with Stockfish. The reasons for this are not all clear, although "Contempt" is a factor. It's up to each person to decide whether a direct match or a rr tournament is a better way to decide which of two engines is stronger. There are arguments both ways.
Well perhaps you did relatively OK in the past but nowadays... this was latest result by Frank. A clear win for SF 180916 against all, Komodo 10 and 10.1 included
(and besides Komodo 10.1 was performing 11 ELO points below K10 in his Test)
http://www.amateurschach.de/main/cross-tab/v438.htm

Code: Select all

1	SF 18Sep2016 BMI2 x64 C10	xxxxx	33.0	39.0	38.5	36.0	40.0	37.0	36.0	39.5	42.0	39.0	42.5	41.5	42.5	42.5
2	Komodo 10 x64	17.0	xxxxx	32.0	37.0	40.0	36.5	37.0	37.0	40.5	41.0	40.5	36.5	42.0	39.5	39.0
3	Houdini 4 STD B x64	11.0	18.0	xxxxx	26.5	26.0	27.5	32.5	29.5	30.5	34.0	32.0	33.0	28.5	32.0	38.0
4	Fire 4 x64	11.5	13.0	23.5	xxxxx	23.5	25.0	25.0	26.5	32.0	28.5	31.0	29.0	32.5	31.5	32.0
5	GullChess 3.0 BMI2 x64	14.0	10.0	24.0	26.5	xxxxx	31.0	23.5	24.5	28.0	30.5	30.0	32.0	26.5	31.5	32.5
6	Andscacs 0.872 BMI2 x64	10.0	13.5	22.5	25.0	19.0	xxxxx	26.0	25.0	26.0	23.5	26.5	31.5	25.0	28.5	31.0
7	Equinox 3.30 x64	13.0	13.0	17.5	25.0	26.5	24.0	xxxxx	24.0	23.5	26.0	29.0	27.5	28.5	32.5	30.5
8	Fizbo 1.8 BMI2 x64	14.0	13.0	20.5	23.5	25.5	25.0	26.0	xxxxx	29.5	23.5	25.5	27.0	28.0	30.5	29.0
9	Critter 1.6a x64	10.5	9.5	19.5	18.0	22.0	24.0	26.5	20.5	xxxxx	24.0	27.5	26.5	28.0	29.5	28.5
10	Fritz 15 x64	8.0	9.0	16.0	21.5	19.5	26.5	24.0	26.5	26.0	xxxxx	26.5	22.5	25.0	26.5	24.0
11	Nirvanachess 2.3 POP x64	11.0	9.5	18.0	19.0	20.0	23.5	21.0	24.5	22.5	23.5	xxxxx	26.5	27.0	28.5	22.5
12	Hannibal 1.7 x64	7.5	13.5	17.0	21.0	18.0	18.5	22.5	23.0	23.5	27.5	23.5	xxxxx	24.0	24.0	29.5
13	Chiron 3 x64	8.5	8.0	21.5	17.5	23.5	25.0	21.5	22.0	22.0	25.0	23.0	26.0	xxxxx	21.5	22.0
14	Protector 1.9.0 x64	7.5	10.5	18.0	18.5	18.5	21.5	17.5	19.5	20.5	23.5	21.5	26.0	28.5	xxxxx	24.0
15	Texel 1.06 x64	7.5	11.0	12.0	18.0	17.5	19.0	19.5	21.0	21.5	26.0	27.5	20.5	28.0	26.0	xxxxx
You are using as evident a run with a May 2016 version of Komodo (Komodo 10) versus a September 2016 version of Stockfish?

Also, as I understand, in the recent run with K 10.2, Contempt was improperly set to 15. 15 is a reasonable value for running against a larger number of mostly weaker opponents. A match against Stockfish or other Komodo versions should use a Contempt of 0.

beram
Posts: 1187
Joined: Wed Jan 06, 2010 2:11 pm

Re: Komodo 10.2 released.

Post by beram » Mon Oct 31, 2016 9:42 pm

mjlef wrote:
beram wrote:
lkaufman wrote:
S.Taylor wrote:
lkaufman wrote:We have released komodo 10.2 at komodochess.com. We estimate based on our testing that it is about 22 elo stronger than komodo 10.1 and about 30 elo stronger than komodo 10 at three minutes plus one second increment, on one or many threads. This is the largest elo gain we have had between versions since going to the subscription model. There were so many changes that it is very difficult to say which ones should get most of the credit. Evaluation, search, and time management were all changed significantly. Features (UCI options) are the same as in komodo 10.1. As usual, it is free to subscribers, and available at a 20% discount to those who bought komodo 9 or a later version.
What _I_ want to see is how this new komodo 10.2 does vs the SF that played in stage 3 of TCEC, OR, the version of SF that played in Graham Banks matches between sf and komodo.

It is very frustrating to have two moving targets at the same time.

e.g. let's say komodo beats ASM, how will we know if the older SF version, which Graham used, wouldn't have crushed komodo 10.2 like it did to komodo 10.1?

We should first test a moving target with a standing target before we go on to the second moving target.
I think we'll do ok relative to stockfish in tests like Frank's where each opponent plays against many other opponents, but we will not look so good in direct matches with Stockfish. The reasons for this are not all clear, although "Contempt" is a factor. It's up to each person to decide whether a direct match or a rr tournament is a better way to decide which of two engines is stronger. There are arguments both ways.
Well perhaps you did relatively OK in the past but nowadays... this was latest result by Frank. A clear win for SF 180916 against all, Komodo 10 and 10.1 included
(and besides Komodo 10.1 was performing 11 ELO points below K10 in his Test)
http://www.amateurschach.de/main/cross-tab/v438.htm

Code: Select all

1	SF 18Sep2016 BMI2 x64 C10	xxxxx	33.0	39.0	38.5	36.0	40.0	37.0	36.0	39.5	42.0	39.0	42.5	41.5	42.5	42.5
2	Komodo 10 x64	17.0	xxxxx	32.0	37.0	40.0	36.5	37.0	37.0	40.5	41.0	40.5	36.5	42.0	39.5	39.0
3	Houdini 4 STD B x64	11.0	18.0	xxxxx	26.5	26.0	27.5	32.5	29.5	30.5	34.0	32.0	33.0	28.5	32.0	38.0
4	Fire 4 x64	11.5	13.0	23.5	xxxxx	23.5	25.0	25.0	26.5	32.0	28.5	31.0	29.0	32.5	31.5	32.0
5	GullChess 3.0 BMI2 x64	14.0	10.0	24.0	26.5	xxxxx	31.0	23.5	24.5	28.0	30.5	30.0	32.0	26.5	31.5	32.5
6	Andscacs 0.872 BMI2 x64	10.0	13.5	22.5	25.0	19.0	xxxxx	26.0	25.0	26.0	23.5	26.5	31.5	25.0	28.5	31.0
7	Equinox 3.30 x64	13.0	13.0	17.5	25.0	26.5	24.0	xxxxx	24.0	23.5	26.0	29.0	27.5	28.5	32.5	30.5
8	Fizbo 1.8 BMI2 x64	14.0	13.0	20.5	23.5	25.5	25.0	26.0	xxxxx	29.5	23.5	25.5	27.0	28.0	30.5	29.0
9	Critter 1.6a x64	10.5	9.5	19.5	18.0	22.0	24.0	26.5	20.5	xxxxx	24.0	27.5	26.5	28.0	29.5	28.5
10	Fritz 15 x64	8.0	9.0	16.0	21.5	19.5	26.5	24.0	26.5	26.0	xxxxx	26.5	22.5	25.0	26.5	24.0
11	Nirvanachess 2.3 POP x64	11.0	9.5	18.0	19.0	20.0	23.5	21.0	24.5	22.5	23.5	xxxxx	26.5	27.0	28.5	22.5
12	Hannibal 1.7 x64	7.5	13.5	17.0	21.0	18.0	18.5	22.5	23.0	23.5	27.5	23.5	xxxxx	24.0	24.0	29.5
13	Chiron 3 x64	8.5	8.0	21.5	17.5	23.5	25.0	21.5	22.0	22.0	25.0	23.0	26.0	xxxxx	21.5	22.0
14	Protector 1.9.0 x64	7.5	10.5	18.0	18.5	18.5	21.5	17.5	19.5	20.5	23.5	21.5	26.0	28.5	xxxxx	24.0
15	Texel 1.06 x64	7.5	11.0	12.0	18.0	17.5	19.0	19.5	21.0	21.5	26.0	27.5	20.5	28.0	26.0	xxxxx
You are using as evident a run with a May 2016 version of Komodo (Komodo 10) versus a September 2016 version of Stockfish?

Also, as I understand, in the recent run with K 10.2, Contempt was improperly set to 15. 15 is a reasonable value for running against a larger number of mostly weaker opponents. A match against Stockfish or other Komodo versions should use a Contempt of 0.
I am using as evidence that in Franks ultimate testing Komodo 10.1 mid july version is -10 ELO to Komodo 10 and the result in the matches against SF where about equal, SF won these with over 60%
http://www.amateurschach.de/fcp-rating- ... -games.txt
As for your contempt setting. There is but little difference in outcome
My testings from recent year where most of the time with Komodo contempt set to zero. But even with that setting Komodo 10.1 looses with around 60% against the Stockfish dev's from the latest months

APassionForCriminalJustic
Posts: 415
Joined: Sat May 24, 2014 7:16 am

Re: Komodo 10.2 released.

Post by APassionForCriminalJustic » Tue Nov 01, 2016 3:05 am

mjlef wrote:
beram wrote:
lkaufman wrote:
S.Taylor wrote:
lkaufman wrote:We have released komodo 10.2 at komodochess.com. We estimate based on our testing that it is about 22 elo stronger than komodo 10.1 and about 30 elo stronger than komodo 10 at three minutes plus one second increment, on one or many threads. This is the largest elo gain we have had between versions since going to the subscription model. There were so many changes that it is very difficult to say which ones should get most of the credit. Evaluation, search, and time management were all changed significantly. Features (UCI options) are the same as in komodo 10.1. As usual, it is free to subscribers, and available at a 20% discount to those who bought komodo 9 or a later version.
What _I_ want to see is how this new komodo 10.2 does vs the SF that played in stage 3 of TCEC, OR, the version of SF that played in Graham Banks matches between sf and komodo.

It is very frustrating to have two moving targets at the same time.

e.g. let's say komodo beats ASM, how will we know if the older SF version, which Graham used, wouldn't have crushed komodo 10.2 like it did to komodo 10.1?

We should first test a moving target with a standing target before we go on to the second moving target.
I think we'll do ok relative to stockfish in tests like Frank's where each opponent plays against many other opponents, but we will not look so good in direct matches with Stockfish. The reasons for this are not all clear, although "Contempt" is a factor. It's up to each person to decide whether a direct match or a rr tournament is a better way to decide which of two engines is stronger. There are arguments both ways.
Well perhaps you did relatively OK in the past but nowadays... this was latest result by Frank. A clear win for SF 180916 against all, Komodo 10 and 10.1 included
(and besides Komodo 10.1 was performing 11 ELO points below K10 in his Test)
http://www.amateurschach.de/main/cross-tab/v438.htm

Code: Select all

1	SF 18Sep2016 BMI2 x64 C10	xxxxx	33.0	39.0	38.5	36.0	40.0	37.0	36.0	39.5	42.0	39.0	42.5	41.5	42.5	42.5
2	Komodo 10 x64	17.0	xxxxx	32.0	37.0	40.0	36.5	37.0	37.0	40.5	41.0	40.5	36.5	42.0	39.5	39.0
3	Houdini 4 STD B x64	11.0	18.0	xxxxx	26.5	26.0	27.5	32.5	29.5	30.5	34.0	32.0	33.0	28.5	32.0	38.0
4	Fire 4 x64	11.5	13.0	23.5	xxxxx	23.5	25.0	25.0	26.5	32.0	28.5	31.0	29.0	32.5	31.5	32.0
5	GullChess 3.0 BMI2 x64	14.0	10.0	24.0	26.5	xxxxx	31.0	23.5	24.5	28.0	30.5	30.0	32.0	26.5	31.5	32.5
6	Andscacs 0.872 BMI2 x64	10.0	13.5	22.5	25.0	19.0	xxxxx	26.0	25.0	26.0	23.5	26.5	31.5	25.0	28.5	31.0
7	Equinox 3.30 x64	13.0	13.0	17.5	25.0	26.5	24.0	xxxxx	24.0	23.5	26.0	29.0	27.5	28.5	32.5	30.5
8	Fizbo 1.8 BMI2 x64	14.0	13.0	20.5	23.5	25.5	25.0	26.0	xxxxx	29.5	23.5	25.5	27.0	28.0	30.5	29.0
9	Critter 1.6a x64	10.5	9.5	19.5	18.0	22.0	24.0	26.5	20.5	xxxxx	24.0	27.5	26.5	28.0	29.5	28.5
10	Fritz 15 x64	8.0	9.0	16.0	21.5	19.5	26.5	24.0	26.5	26.0	xxxxx	26.5	22.5	25.0	26.5	24.0
11	Nirvanachess 2.3 POP x64	11.0	9.5	18.0	19.0	20.0	23.5	21.0	24.5	22.5	23.5	xxxxx	26.5	27.0	28.5	22.5
12	Hannibal 1.7 x64	7.5	13.5	17.0	21.0	18.0	18.5	22.5	23.0	23.5	27.5	23.5	xxxxx	24.0	24.0	29.5
13	Chiron 3 x64	8.5	8.0	21.5	17.5	23.5	25.0	21.5	22.0	22.0	25.0	23.0	26.0	xxxxx	21.5	22.0
14	Protector 1.9.0 x64	7.5	10.5	18.0	18.5	18.5	21.5	17.5	19.5	20.5	23.5	21.5	26.0	28.5	xxxxx	24.0
15	Texel 1.06 x64	7.5	11.0	12.0	18.0	17.5	19.0	19.5	21.0	21.5	26.0	27.5	20.5	28.0	26.0	xxxxx
You are using as evident a run with a May 2016 version of Komodo (Komodo 10) versus a September 2016 version of Stockfish?

Also, as I understand, in the recent run with K 10.2, Contempt was improperly set to 15. 15 is a reasonable value for running against a larger number of mostly weaker opponents. A match against Stockfish or other Komodo versions should use a Contempt of 0.
The issue Mark is that every single tester keeps coming out with results that clearly show that Stockfish is stronger than Komodo. I personally know of a private match between the latest asmFish and Komodo 10.1 - and after about 700 games asmFish was up +90 Elo. I can confirm based on what the tester told me that it was a single thread, Nunn-match tournament at four minute\two second increment. Then you have reputable testers such as Ipman and SPCC clearly showing that Stockfish is stronger. Error margins and the amount of games played may be important; however, there are also things like common sense. The PATTERN of Stockfish's dominance has not changed for months. And that's exactly why Stockfish is now in the superfinal, and it's still undefeated in the TCEC tournament. Stockfish has gained massive strength this year. You should not just shrug off peoples' findings just because they may not be what you want to hear. The patterns speak for themselves.

Look at some of what Ipman wrote concerning his Komodo development versions tested:

28-10-2016

"Getting here Komodo 1730.00 and after first 100games he has it difficult..need for sure a good result in next 100games..
Stockfish 221016 has in mean time 300games and still clear first engine!"

23-10-2016

"Komodo 1714.17 did it not so well and came at same level as K1702 (is removed) and is equal with K10.1
Komodo 1687 is still best version but has played with Dynamism=117
It's amazing but Komodo is already 50Elo behind best Stockfish! a lot improvement is needed!"

Then most recently: 30-10-2016

"Testing Komodo 1730.00 is stopped because not so good results and Komodo 10.2 is released!"

I love Komodo man. In effect, I hope that both you and Larry keep improving it. I love its positional style - and its excellent endgame play. That said, I think even Larry himself sees that Stockfish is on top now. But really, who cares? Just keep adding Elo. That is all that should matter. Stockfish has slowed down over the last few days; it is not impossible to catch Stockfish development over time.

lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 10.2 released.

Post by lkaufman » Tue Nov 01, 2016 4:06 am

APassionForCriminalJustic wrote:
mjlef wrote:
beram wrote:
lkaufman wrote:
S.Taylor wrote:
lkaufman wrote:We have released komodo 10.2 at komodochess.com. We estimate based on our testing that it is about 22 elo stronger than komodo 10.1 and about 30 elo stronger than komodo 10 at three minutes plus one second increment, on one or many threads. This is the largest elo gain we have had between versions since going to the subscription model. There were so many changes that it is very difficult to say which ones should get most of the credit. Evaluation, search, and time management were all changed significantly. Features (UCI options) are the same as in komodo 10.1. As usual, it is free to subscribers, and available at a 20% discount to those who bought komodo 9 or a later version.
What _I_ want to see is how this new komodo 10.2 does vs the SF that played in stage 3 of TCEC, OR, the version of SF that played in Graham Banks matches between sf and komodo.

It is very frustrating to have two moving targets at the same time.

e.g. let's say komodo beats ASM, how will we know if the older SF version, which Graham used, wouldn't have crushed komodo 10.2 like it did to komodo 10.1?

We should first test a moving target with a standing target before we go on to the second moving target.
I think we'll do ok relative to stockfish in tests like Frank's where each opponent plays against many other opponents, but we will not look so good in direct matches with Stockfish. The reasons for this are not all clear, although "Contempt" is a factor. It's up to each person to decide whether a direct match or a rr tournament is a better way to decide which of two engines is stronger. There are arguments both ways.
Well perhaps you did relatively OK in the past but nowadays... this was latest result by Frank. A clear win for SF 180916 against all, Komodo 10 and 10.1 included
(and besides Komodo 10.1 was performing 11 ELO points below K10 in his Test)
http://www.amateurschach.de/main/cross-tab/v438.htm

Code: Select all

1	SF 18Sep2016 BMI2 x64 C10	xxxxx	33.0	39.0	38.5	36.0	40.0	37.0	36.0	39.5	42.0	39.0	42.5	41.5	42.5	42.5
2	Komodo 10 x64	17.0	xxxxx	32.0	37.0	40.0	36.5	37.0	37.0	40.5	41.0	40.5	36.5	42.0	39.5	39.0
3	Houdini 4 STD B x64	11.0	18.0	xxxxx	26.5	26.0	27.5	32.5	29.5	30.5	34.0	32.0	33.0	28.5	32.0	38.0
4	Fire 4 x64	11.5	13.0	23.5	xxxxx	23.5	25.0	25.0	26.5	32.0	28.5	31.0	29.0	32.5	31.5	32.0
5	GullChess 3.0 BMI2 x64	14.0	10.0	24.0	26.5	xxxxx	31.0	23.5	24.5	28.0	30.5	30.0	32.0	26.5	31.5	32.5
6	Andscacs 0.872 BMI2 x64	10.0	13.5	22.5	25.0	19.0	xxxxx	26.0	25.0	26.0	23.5	26.5	31.5	25.0	28.5	31.0
7	Equinox 3.30 x64	13.0	13.0	17.5	25.0	26.5	24.0	xxxxx	24.0	23.5	26.0	29.0	27.5	28.5	32.5	30.5
8	Fizbo 1.8 BMI2 x64	14.0	13.0	20.5	23.5	25.5	25.0	26.0	xxxxx	29.5	23.5	25.5	27.0	28.0	30.5	29.0
9	Critter 1.6a x64	10.5	9.5	19.5	18.0	22.0	24.0	26.5	20.5	xxxxx	24.0	27.5	26.5	28.0	29.5	28.5
10	Fritz 15 x64	8.0	9.0	16.0	21.5	19.5	26.5	24.0	26.5	26.0	xxxxx	26.5	22.5	25.0	26.5	24.0
11	Nirvanachess 2.3 POP x64	11.0	9.5	18.0	19.0	20.0	23.5	21.0	24.5	22.5	23.5	xxxxx	26.5	27.0	28.5	22.5
12	Hannibal 1.7 x64	7.5	13.5	17.0	21.0	18.0	18.5	22.5	23.0	23.5	27.5	23.5	xxxxx	24.0	24.0	29.5
13	Chiron 3 x64	8.5	8.0	21.5	17.5	23.5	25.0	21.5	22.0	22.0	25.0	23.0	26.0	xxxxx	21.5	22.0
14	Protector 1.9.0 x64	7.5	10.5	18.0	18.5	18.5	21.5	17.5	19.5	20.5	23.5	21.5	26.0	28.5	xxxxx	24.0
15	Texel 1.06 x64	7.5	11.0	12.0	18.0	17.5	19.0	19.5	21.0	21.5	26.0	27.5	20.5	28.0	26.0	xxxxx
You are using as evident a run with a May 2016 version of Komodo (Komodo 10) versus a September 2016 version of Stockfish?

Also, as I understand, in the recent run with K 10.2, Contempt was improperly set to 15. 15 is a reasonable value for running against a larger number of mostly weaker opponents. A match against Stockfish or other Komodo versions should use a Contempt of 0.
The issue Mark is that every single tester keeps coming out with results that clearly show that Stockfish is stronger than Komodo. I personally know of a private match between the latest asmFish and Komodo 10.1 - and after about 700 games asmFish was up +90 Elo. I can confirm based on what the tester told me that it was a single thread, Nunn-match tournament at four minute\two second increment. Then you have reputable testers such as Ipman and SPCC clearly showing that Stockfish is stronger. Error margins and the amount of games played may be important; however, there are also things like common sense. The PATTERN of Stockfish's dominance has not changed for months. And that's exactly why Stockfish is now in the superfinal, and it's still undefeated in the TCEC tournament. Stockfish has gained massive strength this year. You should not just shrug off peoples' findings just because they may not be what you want to hear. The patterns speak for themselves.

Look at some of what Ipman wrote concerning his Komodo development versions tested:

28-10-2016

"Getting here Komodo 1730.00 and after first 100games he has it difficult..need for sure a good result in next 100games..
Stockfish 221016 has in mean time 300games and still clear first engine!"

23-10-2016

"Komodo 1714.17 did it not so well and came at same level as K1702 (is removed) and is equal with K10.1
Komodo 1687 is still best version but has played with Dynamism=117
It's amazing but Komodo is already 50Elo behind best Stockfish! a lot improvement is needed!"

Then most recently: 30-10-2016

"Testing Komodo 1730.00 is stopped because not so good results and Komodo 10.2 is released!"

I love Komodo man. In effect, I hope that both you and Larry keep improving it. I love its positional style - and its excellent endgame play. That said, I think even Larry himself sees that Stockfish is on top now. But really, who cares? Just keep adding Elo. That is all that should matter. Stockfish has slowed down over the last few days; it is not impossible to catch Stockfish development over time.
The IPON list now reports Komodo 10.2 gained 22 elo over Komodo 10.1 (exactly matching our estimate in the readme and in our announcement here) with a 43 elo lead over Stockfish 7 at 5' + 3" with ponder on. I know that SF 8 will probably be something like 80 elo above SF 7 in bullet chess, but at IPON levels this may drop to somewhere near the 43 elo SF needs to top the list. In other words, it's too close to call whether Stockfish can regain the top spot if they release now. The Fast GM list has Komodo 10 2 at + 26 elo over Komodo 10.1 in bullet chess, and will probably show a lead of 50 elo or so over SF 7 on the ten minute list once it is available, so Stockfish 8 will have no easy time topping that list either. I readily admit that a new Stockfish will defeat Komodo 10.2 in direct matches (at least at fairly fast time controls), but this will not necessarily put Stockfish at the top of rating lists. Most of the testers seem to feel that performance against a gauntlet of dissimilar strong engines is more relevant than direct matches for judging strength, but of course this is a matter of opinion, not one of fact. Someday we'll figure out why Komodo underperforms against Stockfish, but it may just be something we can't change without hurting Komodo in overall performance. This is also the reason we do poorly on the IPMAN list, which has Komodo playing a zillion different Stockfish versions. It's useful for us to measure our own improvement this way, but it's not useful for predicting whether STockfish or Komodo will do better on lists like IPON, fastgm, Frank's list, CCRL, and CEGT. Doing well on lists like those is our primarily goal.
Komodo rules!

Werewolf
Posts: 1197
Joined: Thu Sep 18, 2008 8:24 pm

Re: Komodo 10.2 released.

Post by Werewolf » Tue Nov 01, 2016 8:37 am

lkaufman wrote:
I readily admit that a new Stockfish will defeat Komodo 10.2 in direct matches (at least at fairly fast time controls), but this will not necessarily put Stockfish at the top of rating lists. Most of the testers seem to feel that performance against a gauntlet of dissimilar strong engines is more relevant than direct matches for judging strength, but of course this is a matter of opinion, not one of fact. Someday we'll figure out why Komodo underperforms against Stockfish.
I'm not trying to be provocative, but why isn't it as simple as this:

i) Stockfish is currently stronger than Komodo and therefore wins in head to head matches.

ii) But Komodo uses contempt as default and so doesn't yield draws as easily to weak opponents, thus gaining some elo in multi-opponent match ups.

Post Reply