See http://www.computerchess.org.uk/ccrl/40 ... t_all.html
9.3 is trailing 45 ELO on 1 core / 64bits and 32 ELO on 4 cores / 64 bits vs 9.2
Granted, there are only 176 and 124 games so far, but the tendency for both platforms is negative. Something is wrong here, default contempt?
Komodo 9.3 well below 9.2 in CCRL/40
Moderators: hgm, Rebel, chrisw
-
- Posts: 213
- Joined: Thu Dec 16, 2010 4:39 pm
Komodo 9.3 well below 9.2 in CCRL/40
Per ardua ad astra
-
- Posts: 2071
- Joined: Thu May 04, 2006 3:40 am
- Location: Dune
Re: Komodo 9.3 well below 9.2 in CCRL/40
I'm testing Komodo 9.3 x64 12CPU for CEGT 40/4 and I have 750 games so far and it's scoring 64.9%, I think it will end up slightly higher than K9.2 x64 C0 12CPU but I'm just guessing by the score.
It scored 97.5 out of 200 (48.7%) against Stockfish TCEC8 12CPU, so there's a chance that Komodo 9.3 x64 12CPU will start at second place on the list. In comparison Komodo 9.2 x64 C0 12CPU scored only 46%, and Komodo 9.2 x64 12CPU (the default) scored only 44.7% against the TCEC 8 Stockfish.
So at least at 40/4 everything seems fine to me.
40/4 ratinglist: http://www.husvankempen.de/nunn/40_4_Ra ... liste.html
It scored 97.5 out of 200 (48.7%) against Stockfish TCEC8 12CPU, so there's a chance that Komodo 9.3 x64 12CPU will start at second place on the list. In comparison Komodo 9.2 x64 C0 12CPU scored only 46%, and Komodo 9.2 x64 12CPU (the default) scored only 44.7% against the TCEC 8 Stockfish.
So at least at 40/4 everything seems fine to me.
40/4 ratinglist: http://www.husvankempen.de/nunn/40_4_Ra ... liste.html
-
- Posts: 10121
- Joined: Thu Mar 09, 2006 12:57 am
- Location: van buren,missouri
Re: Komodo 9.3 well below 9.2 in CCRL/40
There is sure a lot of versions of Komodo in there.
-
- Posts: 2071
- Joined: Thu May 04, 2006 3:40 am
- Location: Dune
Re: Komodo 9.3 well below 9.2 in CCRL/40
And Wolfgang at CEGT 40/4 seems to be getting normal results as well with the 1CPU version, it seems like it will go on the list with +20 elo over K9.2:
http://cegt.forumieren.com/t474-testing-komodo-9-3
He's now working on the 4CPU version. Unfortunately it seems that K9.2 x64 4CPU hasn't been tested yet for that list so I'll see if I have enough time to test that for a bit.
http://cegt.forumieren.com/t474-testing-komodo-9-3
He's now working on the 4CPU version. Unfortunately it seems that K9.2 x64 4CPU hasn't been tested yet for that list so I'll see if I have enough time to test that for a bit.
-
- Posts: 2071
- Joined: Thu May 04, 2006 3:40 am
- Location: Dune
Re: Komodo 9.3 well below 9.2 in CCRL/40
Looking at the CCRL results I think what's causing it on the 4CPU version is Bouquet 1.8. K93's performance there is -60 elo. On the 1CPU version you see -68 against Equinox and -65 against Sting and -84 against BlackMamba. But all these matches have about 10 games each, so I think with more games K9.3 should pull ahead of K92.
-
- Posts: 213
- Joined: Thu Dec 16, 2010 4:39 pm
Re: Komodo 9.3 well below 9.2 in CCRL/40
Thanks for your detailed answers with sensible comments,
statistical fluke then
statistical fluke then
Per ardua ad astra
-
- Posts: 9773
- Joined: Wed Mar 08, 2006 8:44 pm
- Location: Amman,Jordan
Re: Komodo 9.3 well below 9.2 in CCRL/40
Every chess engine & it's subversions has it's own nemesis....Leto wrote:Looking at the CCRL results I think what's causing it on the 4CPU version is Bouquet 1.8. K93's performance there is -60 elo. On the 1CPU version you see -68 against Equinox and -65 against Sting and -84 against BlackMamba. But all these matches have about 10 games each, so I think with more games K9.3 should pull ahead of K92.
I see this on daily basis building my rating list regards,
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
-
- Posts: 1056
- Joined: Thu Mar 09, 2006 4:15 pm
- Location: Long Island, NY, USA
Re: Komodo 9.3 well below 9.2 in CCRL/40
I had assumed that the strong victory by Komodo in tcec8 was mainly due to a big improvement from 9.2 to 9.3. It seems now any improvement was minimal.
I now assume the big victory was mainly due to some previously undetected bugs in SF as well as some decline in performance resulting from the new search.
I now assume the big victory was mainly due to some previously undetected bugs in SF as well as some decline in performance resulting from the new search.
-
- Posts: 2071
- Joined: Thu May 04, 2006 3:40 am
- Location: Dune
Re: Komodo 9.3 well below 9.2 in CCRL/40
I have another theory. Perhaps Komodo 9.3 is about 25 elo stronger than Stockfish TCEC8 at TCEC8 time control, but either equal or slightly weaker than Stockfish TCEC8 at faster time controls.Norm Pollock wrote:I had assumed that the strong victory by Komodo in tcec8 was mainly due to a big improvement from 9.2 to 9.3. It seems now any improvement was minimal.
I now assume the big victory was mainly due to some previously undetected bugs in SF as well as some decline in performance resulting from the new search.
-
- Posts: 417
- Joined: Sat May 24, 2014 9:16 am
Re: Komodo 9.3 well below 9.2 in CCRL/40
Komodo 9.2 is already super strong. People who know their stuff, like myself, already knew that Komodo 9.3x was only about 10 ELO stronger versus 9.2 (clearly you didn't get the memo). What you're basically saying is Komodo won because Stockfish got worse; Komodo won as a result of it being stronger. Stockfish is actually super strong at least on my 28 core. Bugs and all that are excuses...Norm Pollock wrote:I had assumed that the strong victory by Komodo in tcec8 was mainly due to a big improvement from 9.2 to 9.3. It seems now any improvement was minimal.
I now assume the big victory was mainly due to some previously undetected bugs in SF as well as some decline in performance resulting from the new search.