SCCT's New Project:Ponder ON/OFF

[Frank Quisinsky]

Hi Sedat,

after your posting to ponder on / off ... I had in the evening the idea to make after years again a test too. A very little test.

So I had the idea to test in my experimental SWCR-64 WB Crafty 23.3 JA x64 without ponder too. No problem in a automatically tournament. No possible with a UCI engine under Shredder I think.

But the question is really ... is this interesting with only one engine. Means later I can see only for Crafty 23.3 JA x64 ... OK x ELO with ponder = on and y ELO with ponder = off.

I know that the difference to ponder = on / off is around 20-25 ELO. But each time interesting to see it again

Best
Frank

[Sedat Canbaz]

Hi Sedat,

after your posting to ponder on / off ... I had in the evening the idea to make after years again a test too. A very little test.

So I had the idea to test in my experimental SWCR-64 WB Crafty 23.3 JA x64 without ponder too. No problem in a automatically tournament. No possible with a UCI engine under Shredder I think.

But the question is really ... is this interesting with only one engine. Means later I can see only for Crafty 23.3 JA x64 ... OK x ELO with ponder = on and y ELO with ponder = off.

I know that the difference to ponder = on / off is around 20-25 ELO. But each time interesting to see it again

Best
Frank

Hello Frank,

Indeed a good idea...with more testings under different conditions (even if you test only one engine) its will be more clear the influence of the pondering

Btw,i just wonder,how many games (with ponder off) do you plan to test Crafty 23.3 ?

Regards,
Sedat

[Frank Quisinsky]

Code: Select all

01. Rybka 4 x64
02. Stockfish 1.8.0 JA x64
03. Critter 0.80 x64
04. Naum 4.2 x64
05. Komodo 1.2 JA x64
06. Shredder 12 w32 (for ELO calculation in both tables, SWCR-32 / SWCR-64)
07. WB Thinker 5.4d Inert x64
08. Spark 0.5 x64 (private status)
09. Protector 1.3.4 JA x64 (best Protector version)
10. Zappa Mexico II x64
11. Fruit 09.07.05 x64 (private status)
12. Hannibal 1.0a x64
13. Junior 11.2 x64
14. Sjeng WC-2008 x64
15. Onno 1.2.70 x64
16. Umko 1.0 x64
17. Smarthink 1.20 x64
18. WB Crafty 23.3 JA x64

Each one vs. each one 40 games, this games played :-)

The following engines are new.

19. Houdini 1.03a x64 (best engine IPP. fam.)
20. Fire 1.31 x64 (second engine IPP. fam.)
21. GullChess 0.12a x64 (ponder not possible)
22. Cipollino 3.25 x64 (private status)
23. Loop M1-T x64 (longer time available, so far not tested, I should do that)
24. WB Crafty 23.3 JA x64 (without ponder, TEST)

This tournament started around September 06th, 2010 after the still running tournament.

Sedat,

so 920 for each engine are to play. 4.920 new games to play. Need around 17 days (288 games per day can be played).

New games:
Engine 19 = 920
Engine 20 = 880
Engine 21 = 840
Engine 22 = 800
Engine 23 = 760
Engine 24 = 720 = 4.920 new games.

With other words.
Crafty 23.3 JA x64 = 920 games with ponder
Crafty 23.3 JA x64 = 920 games without ponder

Later I will add the results from this test with an extra calculation in my german actual site.

So we know, how many ELO points is Crafty 23.3 x64 stronger with ponder = on.

Best
Frank

[Sedat Canbaz]

Hello Frank,

Thanks for the information...

One question more about testing Crafty 23.3 (with ponder off):
Crafty23.3's opponents will be played also with Ponder OFF or with ON ?

Note:as far as i know,normally Fritz/Shredder GUIs don't allow to use
separately pondering settings,i mean one engine with ponder off,the other engine with ponder on ?! (of course to be 100 % sure about this issue...i think is needed to be checked by modifying crafty.rc)

Btw,i think you will test Crafty 23.3 (with ponder off) using one thread (1 cpu) ?

Greetings,
Sedat

[Frank Quisinsky]

Hi Sedat,

all will played with ponder = on (not GullChess, ponder not posible).

Crafty 23.3 JA x64 played 2x in the same tournament. 1x with ponder = on and 1x with ponder = off. Easy to adjust because its a winboard engines. Can set this setting in crafty.rc



Best
Frank

[Sedat Canbaz]

Hello Frank,

Sorry...but this time i did not understand you

all will played with ponder = on
I am not sure and just wonder about the test of Crafty - ponder off ?
I know that Crafty will be tested with ponder off,but what about the opponents of Crafty,they will be tested against Crafty with pondering on ?


And i think you missed to read my previous question :
-Crafty 23.3 will be tested using one cpu (1 thread) or 2cpu 4cpu or... ?

Best,
Sedat

[Frank Quisinsky]

Hi Sedat,

22 Engines.
2x Crafty 23.3 x64
= 24 Engines

22 Engines played with ponder = on, 1x Crafty 23.3 x64 too
2 Engines played with ponder = off, 1x Crafty 23.3 x64 and GullChess 0.12a (ponder mode not possible).

Tournament under Shredder Classic 4 GUI.

For the two Crafty versions ...
1x in Crafty.rc with ponder = on, 1x in Crafty.rc with ponder = off

In SWCR all engines are played with 1 core only.
All games in SWCR are ponder = on games.

Only Chronos 1.99 and GullChess 0.12a don't have the ponder mode. All other played the games with ponder = on.

So, later we have an ELO for Crafty 23.3 x64 with ponder and without ponder.

Hope a little bit clearer!

Yes, the opponents played with ponder, not GullChess 0.12a and one of the two Crafty versions, this engines played with ponder = off.

[Sedat Canbaz]

Hi Sedat,

22 Engines.
2x Crafty 23.3 x64
= 24 Engines

22 Engines played with ponder = on, 1x Crafty 23.3 x64 too
2 Engines played with ponder = off, 1x Crafty 23.3 x64 and GullChess 0.12a (ponder mode not possible).

Tournament under Shredder Classic 4 GUI.

For the two Crafty versions ...
1x in Crafty.rc with ponder = on, 1x in Crafty.rc with ponder = off

In SWCR all engines are played with 1 core only.
All games in SWCR are ponder = on games.

Only Chronos 1.99 and GullChess 0.12a don't have the ponder mode. All other played the games with ponder = on.

So, later we have an ELO for Crafty 23.3 x64 with ponder and without ponder.

Hope a little bit clearer!

Yes, the opponents played with ponder, not GullChess 0.12a and one of the two Crafty versions, this engines played with ponder = off.

Hello Frank,

Now its all clear...thank you very much for your detailed information

And i have no patience to see your results

Best,
Sedat

[Frank Quisinsky]

Hi Sedat,

interesting is the still running SWCR-64 tournament too.
3 new engines (Tornado 3.6.7 x64, WB BugChess2 1.7 x64, Umko 1.0 x64) and 4 updates (Critter 0.80 x64, Stockfish 1.8.0 JA x64, Crafty 23.3 JA x64 and Hannibal 1.0a x64).

This tournament is still running up to 06.09.2010.
Look here ... (800 games for each engine to play).

5. ELO calculation after 17.357 games (SWCR-64, all games)

4.760 new games to play ...
Actual SWCR-64 rating list (all games):
The 7 new engines have now around 500 / 800 games!

Code: Select all

01. Rybka 4 x64                  2.945 21 20 1049 82% 2.694 28%
--. Rybka 3 x64                  2.908 22 22  840 78% 2.690 28%
02. Stockfish 1.8.0 JA x64       2.900 28 28  483 76% 2.705 32% + 1
--. Stockfish 1.7.1 JA x64       2.899 18 18 1120 76% 2.709 34%
03. Critter 0.80 x64             2.851 27 27  482 70% 2.708 33% +42
04. Naum 4.2 x64                 2.831 16 16 1289 67% 2.706 37%
--. Critter 0.70 x64             2.809 19 19  880 65% 2.704 38%
05. Komodo 1.2 JA x64            2.801 18 18 1049 64% 2.699 40%
06. Shredder 12 w32              2.800 17 16 1248 63% 2.705 36%
--. Komodo 1.0 JA x64            2.794 20 20  840 64% 2.696 40%
--. Shredder 12 x64              2.791 18 17 1080 61% 2.710 35%
07. Thinker 5.4d Inert x64       2.738 16 16 1290 54% 2.709 41%
08. Spark 0.5 x64                2.737 18 18 1049 55% 2.702 38%
09. Zappa Mexico II x64          2.725 16 16 1289 52% 2.709 40%
10. Protector 1.3.4 JA x64       2.721 16 16 1248 52% 2.710 37%
--. Spark 0.4 x64                2.721 19 19  840 53% 2.699 40%
11. Fruit 09_07_05 x64           2.717 16 16 1289 51% 2.710 34%
--. Critter 0.60 x64             2.703 20 20  840 50% 2.700 38%
12. Hannibal 1.0a x64            2.695 26 26  481 47% 2.715 36% +81
13. Junior 11.2 x64              2.692 18 18 1049 49% 2.703 32%
14. Sjeng WC-2008 x64            2.687 16 16 1289 47% 2.711 37%
--. Protector 1.3.5 x64          2.687 19 19  840 47% 2.709 39%
15. Onno 1.2.70 x64              2.678 17 17 1049 46% 2.704 38%
--. Onno 1.1.1 x64               2.674 19 19  840 46% 2.702 40%
--. Junior 11.1a x64             2.651 20 20  840 43% 2.703 32%
16. Loop 2007 x64                2.635 16 16 1290 39% 2.712 36%
--. Twisted Logic 20100131x x64  2.614 17 18 1120 35% 2.719 32%
17. Umko 1.0 x64                 2.614 26 27  481 35% 2.719 34% NEW
18. SmarThink 1.20 x64           2.602 16 17 1289 35% 2.714 33%
19. Crafty 23.3 JA x64           2.592 27 27  481 33% 2.720 31% +34
20. BugChess2 1.7 x64            2.571 26 27  499 29% 2.722 34% NEW
21. Scorpio 2.6 JA x64           2.558 18 18 1120 28% 2.721 32%
--. Crafty 23.2 JA x64           2.558 18 18 1120 28% 2.721 30%
22. Chronos 1.99 x64             2.555 18 18 1120 27% 2.721 33% (ponder not possible)
23. Daydreamer 1.75 JA x64       2.524 19 19 1120 24% 2.723 30%
24. Tornado 3.6.7 x64            2.473 30 31  481 19% 2.726 22% NEW

Frank's Chess Page (SWCR):
Details: http://www.amateurschach.de/swcr/_x64-actual-1.htm

Best
Frank

Not the right thread. But the reason I don't start the tournament I wrote before. This new tournament will be start if the still running tournament ended.

At the moment around 1.900 games to play ... 280 games per day ... the still running tournament need so far around 7 days up to end. Next week Monday I will start the next x64 tournament I wrote before.