Who will win the Duel: Crafty or Stockfish?

[Uri Blass]

Dear Uri,

I think you missed to see:

According to CCRL (40/40):

Stockfish 5 64-bit 4CPU 3283 Elo
Crafty 23.8 64-bit 4CPU 2919 Elo

Elo Difference: 364 Elo


Note: Crafty is playing at slow time control, where Stockfish is playing at blitz


Just saying....

I did not miss it and I simply prefered your list when you use more recent versions of Stockfish and Crafty.

I also think that even if we assume 364 elo difference than the time difference does not compensate for it.

100 elo for doubling certainly does not hold in CCRL testing conditions
and you even do not have 80 elo for doubling(at long time control).

The difference between 4 cpu and 1 cpu is more than 2:1
and CCRL has the following ratings for stockfish:

Stockfish 5 64-bit 4CPU 3283
Stockfish 5 64-bit 3208

For Crafty you have 104 elo difference between 1 cpu and 4 cpu but certainly the speed difference is again clearly more than 2:1 so I do not think that even Crafty earn more than 80 elo for doubling the speed.

Crafty 23.8 64-bit 4CPU 2919 +34 −34
Crafty 23.8 64-bit 2815 +23 −23

If 4 cpu give 2^1.5 speed advantage for Crafty then
104/1.5 is the estimate for the rating advantage for doubling
104/1.5=69.3333...

[Uri Blass]

BTW,

We don't know exactly 60 Elo or 80 Elo or 100 Elo or...per doubling

So just an example... please check the below calculations:

According to CCRL (40/40):

Stockfish 5 64-bit 4CPU 3283 Elo
Crafty 23.8 64-bit 4CPU 2919 Elo

Elo Difference: 364 Elo

Let's say ....In case of 80 Elo per doubling

3 min = 360 Elo
6 min = 280 Elo
12 min = 200 Elo
24 min = 120 Elo
48 min = 40 Elo
60 min = 00 Elo

Once more I'd like to point out,
Stockfish is playing at 3 minutes + 2 seconds, where Crafty is playing at 60 minutes +15 seconds




Best,
Sedat

I can add that comparing 3 minutes to 60 minutes is not fair because you ignore the fact that the increasement is not multiplied by 20.

The time difference even with ponder off is only something near 1:15 and not 1:20

If you want 1:20 then you need to use 60 minutes+40 seconds per move and not 60 minutes+15 seconds for Crafty

[Sedat Canbaz]

Ok let's say 70 Elo per doubling


3 min = 365 Elo
6 min = 295 Elo
12 min = 225 Elo
24 min = 155 Elo
48 min = 85 Elo
96 min = 15 Elo


60 minutes + 15 seconds = 100 minutes (average 80 moves per game)
3 minutes + 2 seconds = 6 minutes (average 80 moves per game)

Another calculation method:

6 min
+70 Elo
12 min
+140 Elo
24 min
+210 Elo
48 min
+280 Elo
96 min
+350 Elo



Note also this is not official match, its just a fun match...)!
And don't take too seriously the current played games...



Best,
Sedat

[Adam Hair]

Ok let's say 70 Elo per doubling


3 min = 365 Elo
6 min = 295 Elo
12 min = 225 Elo
24 min = 155 Elo
48 min = 85 Elo
96 min = 15 Elo


60 minutes + 15 seconds = 100 minutes (average 80 moves per game)
3 minutes + 2 seconds = 6 minutes (average 80 moves per game)

Another calculation method:

6 min
+70 Elo
12 min
+140 Elo
24 min
+210 Elo
48 min
+280 Elo
96 min
+350 Elo



Note also this is not official match, its just a fun match...)!
And don't take too seriously the current played games...



Best,
Sedat

The Elo increase per doubling is not constant. Several of us, but most prominently Kai Laskos, have shown that the Elo increase per doubling decreases as the depth of search increases. At a time control equivalent to CCRL 40/40, the increase per doubling is around 50 to 55 Elo IIRC.

Also, Uri has a point. Stockfish is getting much more of a boost when it has a ponder hit than Crafty does.

However, it will be fun to see the results.

[Sedat Canbaz]

Uri, Adam etc...

You are missing one important point:
-Crafty especially in the beginning of the game, takes to think approx. 2-3 minutes per move
Where Stockfish thinks 3-5 seconds per move

In other words:

Crafty thinks average 150 seconds per move

Where Stockfish thinks average 5 seconds per move

Do you know what this means ??? 30 times more....!!!

Simply, Stockfish is an amazing engine )!

[Sedat Canbaz]

1st game: Stockfish wins ! even in 24 moves !!!!

No words to say, exception: WONDERFUL game by Stockfish !

Well - Done to Mr. Marco Costalba !

[pgn]Event "Stockfish vs Crafty"]
[Site "SCCT"]
[Date "2014.10.23"]
[Round "1"]
[White "Stockfish 151014 64 SSE4.2"]
[Black "Crafty 24.1"]
[Result "1-0"]
[ECO "C68"]
[Annotator "15. 0.92"]
[PlyCount "47"]
[TimeControl "180+2"]

{512MB, empty.ctg, pc12} 1. e4 {[%eval 0,0] [%emt 0:00:00]} e5 {[%emt 0:00:00]}
2. Nf3 {[%eval 0,0] [%emt 0:00:00]} Nc6 {[%emt 0:00:00]} 3. Bb5 {[%eval 0,0]
[%emt 0:00:00]} a6 {[%emt 0:00:00]} 4. Bxc6 {[%eval 0,0] [%emt 0:00:00]} dxc6 {
[%emt 0:00:01]} 5. O-O {[%eval 0,0] [%emt 0:00:00]} Bg4 {[%emt 0:00:00]} 6. h3
{[%eval 0,0] [%emt 0:00:00]} Bh5 {[%emt 0:00:00]} 7. d3 {[%eval 0,0] [%emt 0:
00:00]} Qe7 {[%emt 0:00:01]} 8. Nbd2 {[%eval 0,0] [%emt 0:00:00]} O-O-O {[%emt
0:00:00]} 9. a4 {[%eval 0,0] [%emt 0:00:00]} f6 {[%emt 0:00:00]} 10. b4 {
[%eval 0,0] [%emt 0:00:00]} Qxb4 {[%emt 0:00:00]} 11. Nc4 {[%eval 0,0] [%emt 0:
00:00]} Qe7 {[%emt 0:00:00]} 12. Rb1 {[%eval 0,0] [%emt 0:00:00]} b6 {[%emt 0:
00:00]} 13. Be3 {[%eval 0,0] [%emt 0:00:00]} Qe6 {[%emt 0:00:00]} 14. Qe2 {
[%eval 0,0] [%emt 0:00:00] last book move} a5 {[%emt 0:01:37]} 15. Nxb6+ {
[%eval 92,21] [%emt 0:00:09]} cxb6 {[%emt 0:01:43]} 16. Bxb6 {[%eval 87,28]
[%emt 0:00:00]} Bxf3 {[%emt 0:01:13]} 17. Qxf3 {[%eval 84,29] [%emt 0:00:00]}
Rd7 {[%emt 0:00:00]} 18. d4 {[%eval 119,20] [%emt 0:00:07]} Qc4 {[%emt 0:04:41]
} 19. dxe5 {[%eval 160,32] [%emt 0:00:00]} Be7 {[%emt 0:09:02]} 20. e6 {[%eval
585,36] [%emt 0:00:00]} Rd6 {[%emt 0:06:18]} 21. Ba7 {[%eval 791,33] [%emt 0:
00:00]} Rxe6 {[%emt 0:02:56] (Qa6)} 22. Rfd1 {[%eval 796,22] [%emt 0:00:10]} g5
{[%emt 0:02:13] (Qxe4)} 23. Qg3 {[%eval 1496,28] [%emt 0:00:25]} Re5 {[%emt 0:
00:04]} 24. Qe3 {[%eval 1544,28] [%emt 0:00:01]} 1-0[/pgn]




Feel the difference in SCCT !)



Best,
Sedat

[Sedat Canbaz]

Note also after 14 moves,
There was an unexpected end...so after this position I resumed the game manually
It seems Auto232 mode does not work very well on Windows 8 64-bit

[pgn][Event "Stockfish vs Crafty"]
[Site "SCCT"]
[Date "2014.10.22"]
[Round "1"]
[White "Stockfish 151014"]
[Black "Crafty-24.1"]
[Result "*"]
[ECO "C68"]
[Annotator "Sedat Canbaz"]
[PlyCount "27"]
[EventDate "2014.??.??"]
[TimeControl "180+2"]

{512MB, empty.ctg, pc12} 1. e4 {[%eval 22,20] [%emt 0:00:05]} e5 {
[%emt 0:02:21]} 2. Nf3 {[%eval 21,19] [%emt 0:00:04]} Nc6 {[%emt 0:02:52]} 3.
Bb5 {[%eval 13,27] [%emt 0:00:00]} a6 {(Nf6) [%emt 0:01:49]} 4. Bxc6 {
[%eval 25,21] [%emt 0:00:05]} dxc6 {[%emt 0:01:46]} 5. O-O {
[%eval 24,29] [%emt 0:00:00]} Bg4 {(Bd6) [%emt 0:02:05]} 6. h3 {
[%eval 15,22] [%emt 0:00:14]} Bh5 {[%emt 0:01:25]} 7. d3 {
[%eval 4,26] [%emt 0:00:00]} Qe7 {(Bd6) [%emt 0:02:07]} 8. Nbd2 {
[%eval 43,19] [%emt 0:00:07]} O-O-O {(Nf6) [%emt 0:01:34]} 9. a4 {
[%eval 22,19] [%emt 0:00:08]} f6 {[%emt 0:01:32]} 10. b4 {
[%eval -32769,24] [%emt 0:00:00]} Qxb4 {(Qe8) [%emt 0:01:30]} 11. Nc4 {
[%eval 28,21] [%emt 0:00:10]} Qe7 {[%emt 0:03:14]} 12. Rb1 {
[%eval -32769,26] [%emt 0:00:00]} b6 {[%emt 0:03:19]} 13. Be3 {
[%eval 52,27] [%emt 0:00:00]} Qe6 {[%emt 0:01:38]} 14. Qe2 {
[%eval 57,26] [%emt 0:00:00]} *[/pgn]

[Sedat Canbaz]

Adam,


With respect to Andreas, Kai etc...
I had serious testings (about doubling) I run such testings a long time before...what about you, do you have such data ?)

And I am not in chess since yesterday, more than 35 years in chess, and nearly 20 years in computer chess
So I have a little bit experience...and that experience helps me in most cases ))

About the current issue, see here please:
http://i.hizliresim.com/8mv1GW.gif

Crafty 423 kns = 2410 Elo
Crafty 11796 kns = 2813 Elo

423 kns
846 kns
1692 kns
3384 kns
6768 kns
13536 kns


Conclusion:
Even less than 5 times doublings = 400 Elo
That means Crafty gains min 80 Elo per doubling

Another proof:
http://www.sedatcanbaz.com/chess/?page_id=1020

6 Deep Rybka 4.1 x64 6c 3358 Elo
15 Deep Rybka 4.1 x64 4c 3293 Elo

i7 980X is approx. 1.66 faster than i7 920
And if with 1.66 we get approx. 60 Elo that means in case of doubling minimum will be 80 Elo


Hopes helps....


If you need more help...just let me know...

[Matthias Gemuh]

It would be a much more interesting match with ponder off so both engines get full use of all the cores during their thinking time.

I agree
ponder on gives advantage to stockfish that can use more time on Crafty's time ...

Considering the fact that it is also ponder on I will be surprised if Crafty scores better than 25%.

I agree too.

BTW, how many games will they play ? ... and how will they avoid repeating games without an opening book ?

[Vinvin]

It would be a much more interesting match with ponder off so both engines get full use of all the cores during their thinking time.

I agree
ponder on gives advantage to stockfish that can use more time on Crafty's time ...

Considering the fact that it is also ponder on I will be surprised if Crafty scores better than 25%.

I agree too.

BTW, how many games will they play ? ... and how will they avoid repeating games without an opening book ?

Me too, "ponder=off" is the way to go for a time handicapped match !