New Duel: Who will win: Rybka or Stockfish ?

[Sedat Canbaz]

It looks like the current conditions are all right,
I mean Komodo 8 3m+1s does not suffer under Arena 3.5 GUI

So I think it is not required to be continued the current match any more...

But what is required, I really wonder a lot:
Which Blitz conditions are better for chess engines ?)

So the next duel: 3m+2s Ponder ON vs 40/2 Ponder OFF

Komodo 1 core and Stockfish 1 core will be used as participants
Note: both participants will be tested (as switched sides) under both GUIs

1) Let's see what will be the time control influence...
2) Let's what will be the influence of the pondering....

I hope my small test will be useful for the new generation...

Soon more updates coming soon ! )


Best,
Sedat

[Sedat Canbaz]

1st Update:



So far 3m+2s Ponder ON is performing incredibly strong over 40/2 Ponder OFF!
Note: 40/2 = 40 moves in 2 minutes, repeating...

[Sedat Canbaz]

Standing after 10 games:

[Sedat Canbaz]

And here is another example, as we see,
Not only Stockfish, even Komodo too is doing very good at 3m+2s Ponder ON


More updates coming soon...!

[Sedat Canbaz]

More Details:
For a better conclusion we need much more games, but however in my opinion:
*In case of doubling the time controls on same hardware,
There is no right formula, all it depends on used conditions
Probably it looks like average between 20 Elo - 60 Elo
Note: the mentioned formula often can lead the misunderstanding results
So we need at least thousands of games at Blitz and at Slow time controls...
Even after thousands of games.. the used formula can be lead to wrong calculations....

BTW, What is the importance of running these testings ?!
We noticed:
- The Top engines (Stockfish, Komodo, Rybka) are incredibly strong even at Bullet fast time control
- Top Chess Engine's Bullet, Blitz games are at level in strength to GMs's Slow time control games
- Ponder ON is much better than Ponder OFF and 3 min + 2 sec is not bad Blitz time control too !)

And I have a wish to all engine testers (who rates too low Elo vlues):
- You should increase your Engines low Elo values to up... !
- It's a funny that who rates the engines 200-300 Elo less than reality...

Hopes helps...


Download SCCT Duel Games:
www.sedatcanbaz.com/chess/games/SCCT_Duel.rar



Best,
Sedat

[lkaufman]

More Details:
For a better conclusion we need much more games, but however in my opinion:
*In case of doubling the time controls on same hardware,
There is no right formula, all it depends on used conditions
Probably it looks like average between 20 Elo - 60 Elo
Note: the mentioned formula often can lead the misunderstanding results
So we need at least thousands of games at Blitz and at Slow time controls...
Even after thousands of games.. the used formula can be lead to wrong calculations....

BTW, What is the importance of running these testings ?!
We noticed:
- The Top engines (Stockfish, Komodo, Rybka) are incredibly strong even at Bullet fast time control
- Top Chess Engine's Bullet, Blitz games are at level in strength to GMs's Slow time control games
- Ponder ON is much better than Ponder OFF and 3 min + 2 sec is not bad Blitz time control too !)

And I have a wish to all engine testers (who rates too low Elo vlues):
- You should increase your Engines low Elo values to up... !
- It's a funny that who rates the engines 200-300 Elo less than reality...

Hopes helps...


Download SCCT Duel Games:
www.sedatcanbaz.com/chess/games/SCCT_Duel.rar



Best,
Sedat

If anything, the blitz play of SF and Komodo is well beyond the level of GM tournament play, more like the level of Carlsen or Caruana at standard tournament levels (or even better). On the other hand, rating differences between engines based on engine vs engine games are greater than what they would be against humans, partly due to the short books used in engine testing, whereas actual human tournament games often follow preparation for many moves. So clearly the level of ratings for ordinary engines on these lists is too low, but not as clearly so at the top, although it varies greatly from list to list. In general I agree with you.
Of course the quality of play in a PONDER ON game is higher than in a PONDER OFF game at the same level, but it is even more clearly lower than the quality of play in a PONDER OFF game at double the level. So PONDER ON is a serious waste of resources. Also the results are a bit more random as they depend on the frequency of ponder hits, which are somewhat random.
I'm not at all sure what you were trying to show with these last two matches; obviously with more time and ponder on the machines will beat less time with ponder off. It should also be noted that increment play is far more efficient than 40/x play; given the same average time per game, the quality of increment play is much higher as the time is available when needed, before the result is decided. I guess you were just demonstrating these fairly obvious points.

[M ANSARI]

What is certain is that the SOFTWARE side of chess engines has improved incredibly in the last 5 years. Now when you add the hardware side improvements (maybe even more impressive) you can imagine how strong the combination is. My guess is that if you would take SF at 5m +15s it would easily defeat the DB that beat Kasparov at the time control they played.

[Sedat Canbaz]

More Details:
For a better conclusion we need much more games, but however in my opinion:
*In case of doubling the time controls on same hardware,
There is no right formula, all it depends on used conditions
Probably it looks like average between 20 Elo - 60 Elo
Note: the mentioned formula often can lead the misunderstanding results
So we need at least thousands of games at Blitz and at Slow time controls...
Even after thousands of games.. the used formula can be lead to wrong calculations....

BTW, What is the importance of running these testings ?!
We noticed:
- The Top engines (Stockfish, Komodo, Rybka) are incredibly strong even at Bullet fast time control
- Top Chess Engine's Bullet, Blitz games are at level in strength to GMs's Slow time control games
- Ponder ON is much better than Ponder OFF and 3 min + 2 sec is not bad Blitz time control too !)

And I have a wish to all engine testers (who rates too low Elo vlues):
- You should increase your Engines low Elo values to up... !
- It's a funny that who rates the engines 200-300 Elo less than reality...

Hopes helps...


Download SCCT Duel Games:
www.sedatcanbaz.com/chess/games/SCCT_Duel.rar



Best,
Sedat

If anything, the blitz play of SF and Komodo is well beyond the level of GM tournament play, more like the level of Carlsen or Caruana at standard tournament levels (or even better). On the other hand, rating differences between engines based on engine vs engine games are greater than what they would be against humans, partly due to the short books used in engine testing, whereas actual human tournament games often follow preparation for many moves. So clearly the level of ratings for ordinary engines on these lists is too low, but not as clearly so at the top, although it varies greatly from list to list. In general I agree with you.
Of course the quality of play in a PONDER ON game is higher than in a PONDER OFF game at the same level, but it is even more clearly lower than the quality of play in a PONDER OFF game at double the level. So PONDER ON is a serious waste of resources. Also the results are a bit more random as they depend on the frequency of ponder hits, which are somewhat random.
I'm not at all sure what you were trying to show with these last two matches; obviously with more time and ponder on the machines will beat less time with ponder off. It should also be noted that increment play is far more efficient than 40/x play; given the same average time per game, the quality of increment play is much higher as the time is available when needed, before the result is decided. I guess you were just demonstrating these fairly obvious points.

Yes...I am glad that you see the reality too!

About my latest testings,
I am just demonstrating that the current SCCT MP Rating List (3m+2s) Ponder ON:
- Should be counted as one of the best available Blitz rating lists!

Btw, the next planning duel (SCCT vs IPON):
3m+2s 3 cores vs 5m+3s 1 core
Note: the duel games will be with Ponder ON

Let's see which conditions are better, I mean which games are stronger...
Soon will be clear this unknown issue...

Best,
Sedat

[Sedat Canbaz]

What is certain is that the SOFTWARE side of chess engines has improved incredibly in the last 5 years. Now when you add the hardware side improvements (maybe even more impressive) you can imagine how strong the combination is. My guess is that if you would take SF at 5m +15s it would easily defeat the DB that beat Kasparov at the time control they played.

+1

I am not surprised that you see the reality too!


Best,
Sedat

[Uri Blass]

More Details:
For a better conclusion we need much more games, but however in my opinion:
*In case of doubling the time controls on same hardware,
There is no right formula, all it depends on used conditions
Probably it looks like average between 20 Elo - 60 Elo
Note: the mentioned formula often can lead the misunderstanding results
So we need at least thousands of games at Blitz and at Slow time controls...
Even after thousands of games.. the used formula can be lead to wrong calculations....

BTW, What is the importance of running these testings ?!
We noticed:
- The Top engines (Stockfish, Komodo, Rybka) are incredibly strong even at Bullet fast time control
- Top Chess Engine's Bullet, Blitz games are at level in strength to GMs's Slow time control games
- Ponder ON is much better than Ponder OFF and 3 min + 2 sec is not bad Blitz time control too !)

And I have a wish to all engine testers (who rates too low Elo vlues):
- You should increase your Engines low Elo values to up... !
- It's a funny that who rates the engines 200-300 Elo less than reality...

Hopes helps...


Download SCCT Duel Games:
www.sedatcanbaz.com/chess/games/SCCT_Duel.rar



Best,
Sedat

If anything, the blitz play of SF and Komodo is well beyond the level of GM tournament play, more like the level of Carlsen or Caruana at standard tournament levels (or even better). On the other hand, rating differences between engines based on engine vs engine games are greater than what they would be against humans, partly due to the short books used in engine testing, whereas actual human tournament games often follow preparation for many moves. So clearly the level of ratings for ordinary engines on these lists is too low, but not as clearly so at the top, although it varies greatly from list to list. In general I agree with you.
Of course the quality of play in a PONDER ON game is higher than in a PONDER OFF game at the same level, but it is even more clearly lower than the quality of play in a PONDER OFF game at double the level. So PONDER ON is a serious waste of resources. Also the results are a bit more random as they depend on the frequency of ponder hits, which are somewhat random.
I'm not at all sure what you were trying to show with these last two matches; obviously with more time and ponder on the machines will beat less time with ponder off. It should also be noted that increment play is far more efficient than 40/x play; given the same average time per game, the quality of increment play is much higher as the time is available when needed, before the result is decided. I guess you were just demonstrating these fairly obvious points.

Yes...I am glad that you see the reality too!

About my latest testings,
I am just demonstrating that the current SCCT MP Rating List (3m+2s) Ponder ON:
- Should be counted as one of the best available Blitz rating lists!

Btw, the next planning duel (SCCT vs IPON):
3m+2s 3 cores vs 5m+3s 1 core
Note: the duel games will be with Ponder ON

Let's see which conditions are better, I mean which games are stronger...
Soon will be clear this unknown issue...

Best,
Sedat

I am almost sure 3+2 3 cores is stronger even with ponder off
because I guess 3 cores is probably not less than twice faster than 1 core and 5+3 is less than twice faster than 3+2

ponder on gives an additional advantage for 3+2 that can use more time for pondering.