Will a Komodo 9 catch up with Stockfish ?

[beram]

SPCC, FGRL, IPON, Tom Casanovas, CEGT, CCRL, all have SF 6 better than K8 by 56-62 %
It doesn’t matter what machine or time control, SF 6 is 40-70 ELO better than K8 in their matches
But what is the real difference between latest Komodo dev and latest SF dev ?
Last indication for distance between K8 dev and SFdev was a match which Ray Banks ran 1 month ago,
where difference was 52,25% - 52,5 better for SF dev 221214 against K8 dev 1339 (with drawscore default and zero) over 2x 400 games: http://www.talkchess.com/forum/viewtopi ... 45&t=47512
Meaning that K8 was about 15-20 ELO behind at that time
Since tests have shown that SF 6 has gained 20 ELO over SF dev211214(the one Ray used) it will be very unlikely that K8 has improved more than that in such a short time.
Therefore when looking at the time needed for improving Komodo in the past I guess it will take about at least half a year for Komodo team to catch up with SF 6 on the most common ratinglists
But within that half year SF dev at that time will again be ahead Komodo 9 by at least 15- 30 ELO
So I doubt Komodo 9 will catch up at least not this year

[notyetagm]

SPCC, FGRL, IPON, Tom Casanovas, CEGT, CCRL, all have SF 6 better than K8 by 56-62 %
It doesn’t matter what machine or time control, SF 6 is 40-70 ELO better than K8 in their matches
But what is the real difference between latest Komodo dev and latest SF dev ?

Egads!

Didn't Komodo just beat Stockfish in the last TCEC?

And now not even two months later Stockfish is 40-70 ELO stronger????

What the hell just happened???

[carldaman]

SPCC, FGRL, IPON, Tom Casanovas, CEGT, CCRL, all have SF 6 better than K8 by 56-62 %
It doesn’t matter what machine or time control, SF 6 is 40-70 ELO better than K8 in their matches
But what is the real difference between latest Komodo dev and latest SF dev ?

Egads!

Didn't Komodo just beat Stockfish in the last TCEC?

And now not even two months later Stockfish is 40-70 ELO stronger????

What the hell just happened???

C'mon man, TCEC is played with 16 cores, where Komodo scales much better than Stockfish. Relatively few people have 16 cores to test with.

Regards,
CL

[Sedat Canbaz]

SPCC, FGRL, IPON, Tom Casanovas, CEGT, CCRL, all have SF 6 better than K8 by 56-62 %
It doesn’t matter what machine or time control, SF 6 is 40-70 ELO better than K8 in their matches
But what is the real difference between latest Komodo dev and latest SF dev ?
Last indication for distance between K8 dev and SFdev was a match which Ray Banks ran 1 month ago,
where difference was 52,25% - 52,5 better for SF dev 221214 against K8 dev 1339 (with drawscore default and zero) over 2x 400 games: http://www.talkchess.com/forum/viewtopi ... 45&t=47512
Meaning that K8 was about 15-20 ELO behind at that time
Since tests have shown that SF 6 has gained 20 ELO over SF dev211214(the one Ray used) it will be very unlikely that K8 has improved more than that in such a short time.
Therefore when looking at the time needed for improving Komodo in the past I guess it will take about at least half a year for Komodo team to catch up with SF 6 on the most common ratinglists
But within that half year SF dev at that time will again be ahead Komodo 9 by at least 15- 30 ELO
So I doubt Komodo 9 will catch up at least not this year

Interesting thread, thanks )

Yes...the Elo difference between K8 and SF6 depends on our used conditions:
Hardwares, time control, cores, opening books etc...

For example,
According to my newest blitz rating list,
http://www.sedatcanbaz.com/chess/?page_id=1339

Stockfish 6 performed approx. 30 Elo over Komodo 8
Of course SF6 has several games lost on time, otherwise probably the Elo difference would be approx. 40 Elo

In case of comparing under Android platforms,
Then SF6 performed much better than Komodo 8, Elo difference: approx. 100

But however, I should admit too,
With slow time controls the situation is different, for example Komodo 1318 performed incredibly strong:
http://www.sedatcanbaz.com/chess/?page_id=1168

Note also that I resumed the Gladiators 2015 (60m+15s) and soon SF6 performance will appear...

Yes...after all,
It looks like 2015 will be very exiting....not sure how much K9 will be improved...but I guess K9 will catch SF6

Lets see...


Best,
Sedat

[Isaac]

I think Komodo dev may be already stronger than SF6 on 4 cores under CCRL conditions. At least that's the impression I got from reading the Komodo dev here on talkchess.

According to my newest blitz rating list,
http://www.sedatcanbaz.com/chess/?page_id=1339

Stockfish 6 performed approx. 30 Elo over Komodo 8
Of course SF6 has several games lost on time, otherwise probably the Elo difference would be approx. 40 Elo

Time losses are not normal at all and should be extremely rare. I am guessing it's the GUI's fault.
Stockfish does not lose on time even when the whole game lasts 15 seconds if you use cutechess. At least less than once every 10,000 games.

[Sedat Canbaz]

I think Komodo dev may be already stronger than SF6 on 4 cores under CCRL conditions. At least that's the impression I got from reading the Komodo dev here on talkchess.

According to my newest blitz rating list,
http://www.sedatcanbaz.com/chess/?page_id=1339

Stockfish 6 performed approx. 30 Elo over Komodo 8
Of course SF6 has several games lost on time, otherwise probably the Elo difference would be approx. 40 Elo

Time losses are not normal at all and should be extremely rare. I am guessing it's the GUI's fault.
Stockfish does not lose on time even when the whole game lasts 15 seconds if you use cutechess. At least less than once every 10,000 games.

I am not sure about this issue...

If all engines forfeit on time, then I can understand that Arena GUI is buggy...

But what I noticed only a few engines forfeit on time, for example:
Engines like Stockfish, Senpai, Protector, Fire forfeit on time...
Where mostly of the engines like Rybka, Houdini, Shredder do not forfeit on time...

It looks like those mentioned engines which forfeits on time have same buggy code ))

Best,
Sedat

[Isaac]

I am not sure about this issue...

If all engines forfeit on time, then I can understand that Arena GUI is buggy...

But what I noticed only a few engines forfeit on time, for example:
Engines like Stockfish, Senpai, Protector, Fire forfeit on time...
Where mostly of the engines like Rybka, Houdini, Shredder do not forfeit on time...

It looks like those mentioned engines which forfeits on time have same buggy code ))

Best,
Sedat

Maybe Arena threats different engines in a different way.
I did not know you were using Arena... Arena steals about 0.4 second per move to Stockfish. To see that I asked a friend to login into FICS with Arena and play a few games vs Stockfish on winboard. The fastest moves (when there was only 1 legal moves and the moves out of the opening) took more than 0.4 second to be played for Stockfish under Arena and about 0.01 second for Winboard's Stockfish.

There: http://tests.stockfishchess.org/tests you can see extremely quick games between many different versions of Stockfish that last a few seconds and most machines have 0 loss on time after thousands of games. So the fact that Arena cannot handle Stockfish as well as Rybka or other engine does not mean that Stockfish is buggy since cutechess demonstrate that there's no problem whatsoever.

[Sedat Canbaz]

I am not sure about this issue...

If all engines forfeit on time, then I can understand that Arena GUI is buggy...

But what I noticed only a few engines forfeit on time, for example:
Engines like Stockfish, Senpai, Protector, Fire forfeit on time...
Where mostly of the engines like Rybka, Houdini, Shredder do not forfeit on time...

It looks like those mentioned engines which forfeits on time have same buggy code ))

Best,
Sedat

Maybe Arena threats different engines in a different way.
I did not know you were using Arena... Arena steals about 0.4 second per move to Stockfish. To see that I asked a friend to login into FICS with Arena and play a few games vs Stockfish on winboard. The fastest moves (when there was only 1 legal moves and the moves out of the opening) took more than 0.4 second to be played for Stockfish under Arena and about 0.01 second for Winboard's Stockfish.

There: http://tests.stockfishchess.org/tests you can see extremely quick games between many different versions of Stockfish that last a few seconds and most machines have 0 loss on time after thousands of games. So the fact that Arena cannot handle Stockfish as well as Rybka or other engine does not mean that Stockfish is buggy since cutechess demonstrate that there's no problem whatsoever.

Thanks for the useful info...

Ok...now I am quite satisfied with Stockfish 6, because with 3m.+1s, I noticed only a few games lost on time
I can not remember exactly the number, but I think SF6 lost 3-4 games on time, based on more than 2000 games per Stockfish 6 engine

Those time forfeits were more, when the time control was 3 min per side

In other words, it is ok....


Best,
Sedat

[shrapnel]

SPCC, FGRL, IPON, Tom Casanovas, CEGT, CCRL, all have SF 6 better than K8 by 56-62 %
It doesn’t matter what machine or time control, SF 6 is 40-70 ELO better than K8 in their matches
But what is the real difference between latest Komodo dev and latest SF dev ?
Last indication for distance between K8 dev and SFdev was a match which Ray Banks ran 1 month ago,
where difference was 52,25% - 52,5 better for SF dev 221214 against K8 dev 1339 (with drawscore default and zero) over 2x 400 games: http://www.talkchess.com/forum/viewtopi ... 45&t=47512
Meaning that K8 was about 15-20 ELO behind at that time
Since tests have shown that SF 6 has gained 20 ELO over SF dev211214(the one Ray used) it will be very unlikely that K8 has improved more than that in such a short time.
Therefore when looking at the time needed for improving Komodo in the past I guess it will take about at least half a year for Komodo team to catch up with SF 6 on the most common ratinglists
But within that half year SF dev at that time will again be ahead Komodo 9 by at least 15- 30 ELO
So I doubt Komodo 9 will catch up at least not this year

I really like Komodo, but I'm afraid you are right.
Even if we suppose that Komodo 9 suddenly becomes stronger than Stockfish and is released to customers, in only a week or two SF will catch up and even become stronger than Komodo 9 !
So, the number of customers will drop rapidly. Even if people want it, all they have to do is to simply wait for a week or two until Komodo 9 becomes available on the illegal Networks.
This is the harsh TRUTH !
So, in this context, it makes even more sound business sense to start a Subscription service for Komodo development versions. Komodo team will make more money that way.
Another(hidden) benefit of a subscription service would be that it would discourage the Hackers, as even they would hesitate to keep giving for free something for which they have to pay a regular subscription !
This way, loyal Komodo fans would be rewarded and it would mean more Profit for the Komodo Team
Q.E.D.

[carldaman]

SPCC, FGRL, IPON, Tom Casanovas, CEGT, CCRL, all have SF 6 better than K8 by 56-62 %
It doesn’t matter what machine or time control, SF 6 is 40-70 ELO better than K8 in their matches
But what is the real difference between latest Komodo dev and latest SF dev ?
Last indication for distance between K8 dev and SFdev was a match which Ray Banks ran 1 month ago,
where difference was 52,25% - 52,5 better for SF dev 221214 against K8 dev 1339 (with drawscore default and zero) over 2x 400 games: http://www.talkchess.com/forum/viewtopi ... 45&t=47512
Meaning that K8 was about 15-20 ELO behind at that time
Since tests have shown that SF 6 has gained 20 ELO over SF dev211214(the one Ray used) it will be very unlikely that K8 has improved more than that in such a short time.
Therefore when looking at the time needed for improving Komodo in the past I guess it will take about at least half a year for Komodo team to catch up with SF 6 on the most common ratinglists
But within that half year SF dev at that time will again be ahead Komodo 9 by at least 15- 30 ELO
So I doubt Komodo 9 will catch up at least not this year

I really like Komodo, but I'm afraid you are right.
Even if we suppose that Komodo 9 suddenly becomes stronger than Stockfish and is released to customers, in only a week or two SF will catch up and even become stronger than Komodo 9 !
So, the number of customers will drop rapidly. Even if people want it, all they have to do is to simply wait for a week or two until Komodo 9 becomes available on the illegal Networks.
This is the harsh TRUTH !
So, in this context, it makes even more sound business sense to start a Subscription service for Komodo development versions. Komodo team will make more money that way.
Another(hidden) benefit of a subscription service would be that it would discourage the Hackers, as even they would hesitate to keep giving for free something for which they have to pay a regular subscription !
This way, loyal Komodo fans would be rewarded and it would mean more Profit for the Komodo Team
Q.E.D.

Yes, it makes too much sense. I would also suggest an anti-piracy scheme, where one has to have the base version (Komodo 9) installed already, before any updated versions from the subscription can be run.

Regards,
CL