SCCT's New Project:Ponder ON/OFF

[Sedat Canbaz]

Hello Chess Friends !

Probably SedatChess's site Visitors will remember that there are many different kind of chess testings,e.g:
-Engine Ratings/Tournaments,Hardware Testings,Opening Book Tourneys,GUIs TEST...

But what about for SCCT-Ponder ON/OFF,yes i have already started a new one
This test is totally different than the previous ones,the main goal of the test is to see the influence of the 'Pondering'
The TOP engines will be played exactly on same conditions,exception 'Ponder ON/OFF'


1.Main Tournament Conditions:
----------------------------------------

Code: Select all

Processors                       i7 920 @3.3GHz/QX9650 @3.80GHz
Operating System                 Windows XP Professional x64 (service pack 2)
Main Chess GUI                   ChessProgram12(Fritz 12 GUI-Aug 9 2010)
Time Control                     5 min + 5 sec
Engine's All Threads (CPUs)      Enabled
Engine / Position Learning       Default Settings
HashTable Size                   512 MB
Ponder                           ON/OFF
Opening Book                     Perfect 2010
Book Learning                    OFF
Tablebases                       DVD Endgame Turbo 3
TB Cache                         32 MB
Tournament Director              Sedat Canbaz

So there will be two kind of Tournaments:
----------------------------------------------------
1.Engines will be tested with 'Ponder ON' (engine matches between each other on two separate computers using Netchess utility)
2.Engines will be tested with 'Ponder OFF' (engines will be played on the same computers,not using netchess utility or auto232player)


Let's see what will be the standings and ELO difference between 'Ponder OFF' and 'Ponder ON'


More information coming soon !




Best Regards,
Sedat Canbaz

[Graham Banks]

Let's see what will be the standings and ELO difference between 'Ponder OFF' and 'Ponder ON'

Should be interesting.

Cheers,
Graham.

[Frank Quisinsky]

Hi Sedat,

I test it in ATL-4 for 4-5 years. But this test was not clear enough because I used not the same hardware.

I have in the brain that King of Kings and Zarkov are around 20-25 ELO better with ponder and Glaurung around 10-20 ELO better without ponder. For the others around +-10.

An interesting idea.

For around 10 years I test how many points I will get with ponder more. Same three engine with and without ponder. Here the results are very clear. It was around 22-26 points more.

I will follow your new event.
Thanks!

Best
Frank

[Sedat Canbaz]

More Details about SCCT (Ponder OFF/ON) Test


Participants of the both Tournaments:
-----------------------------------------------

Code: Select all

Critter 0.80 x64 T4
Deep Fritz 12 T4
Deep Onno-1-2-70 x64 T4
Deep Rybka 4 x64 T4
Deep Shredder 12 x64 T4
Hiarcs 13.1 T4
Naum 4.2 x64 T4
Protector 1.3.6 JA x64 T4
Spark 0.4 x64 T4
Stockfish 1.8 JA x64 T4
Zappa Mexico II x64 T4

Note:In the end of the both tests,each Engine will have 200 games per player,Total:1100 games

In my opinion,for a ideal conclusion is needed thousands of games per player,
but anyway (even with 1100 games) the test will provide quite a bit of useful information,
it will help us to compare the Elo difference between 'Ponder ON' and 'Ponder OFF' !

In my estimation,however the standings of the both tournaments will be approximately same or similar


Greetings,
Sedat

[Sedat Canbaz]

Let's see what will be the standings and ELO difference between 'Ponder OFF' and 'Ponder ON'

Should be interesting.

Cheers,
Graham.

Hello Graham,

Yes...i think so

Btw,the tournament with 'Ponder ON' will be over in 2-3 days and then the results will be published

Best,
Sedat

[Sedat Canbaz]

Hi Sedat,

I test it in ATL-4 for 4-5 years. But this test was not clear enough because I used not the same hardware.

I have in the brain that King of Kings and Zarkov are around 20-25 ELO better with ponder and Glaurung around 10-20 ELO better without ponder. For the others around +-10.

An interesting idea.

For around 10 years I test how many points I will get with ponder more. Same three engine with and without ponder. Here the results are very clear. It was around 22-26 points more.

I will follow your new event.
Thanks!

Best
Frank

Hello Frank,

Yes...thanks for your useful information and i know that you are one of the Testers who is testing the engines only with Ponder ON

Personally i get much more fun and exiting when Pondering is enabled,but due to not enough hardwares,the CPU Time don't allow me testing variety of engines with Ponder ON

About pondering...yes i agree with you,some engines benefit very well when is ponder on,where there are also such engines,which have ponder bugs and it makes a big influence to the engine's performance

BTW,i have no patience and i really wonder what will be the standings


Regards,
Sedat

[Sedat Canbaz]

Another interesting note for gaining elo points is the influence of using maximum engine threads

And i have just made some statistics about ELO differences between 1CPU and 4CPU

Code: Select all

SCCT 10 Minutes/Game
Deep Rybka 4 x64 4CPU  3309 Elo
Deep Rybka 4 x64 1CPU  3201 Elo
*Elo Difference:108 


Stockfish 1.7.1 JA x64 4CPU  3206 Elo
Stockfish 1.7.1 JA x64 1CPU  3137 Elo
*Elo Difference:69 

------------------------------------------
CEGT 40/20
Deep Rybka 4 x64 4CPU    3226 Elo
Deep Rybka 4 x64 1CPU    3134 Elo
*Elo Difference:92       


Stockfish 1.7.1 x64 4CPU    3164 Elo   
Stockfish 1.7.1 x64 1CPU    3084 Elo
*Elo Difference:80 

------------------------------------------
CEGT 40/4
Rybka 4.0 x64 4CPU    3260 Elo
Rybka 4.0 x64 1CPU    3179 Elo
*Elo Difference:81 

Stockfish 1.7.1 x64 4CPU    3206 Elo
Stockfish 1.7.1 x64 1CPU    3101 Elo
*Elo Difference:105

------------------------------------------
CCRL 40/40
Rybka 4 64-bit 4CPU     3269 Elo
Rybka 3 64-bit 1CPU     3152 Elo
*Elo Difference:117 Elo

Stockfish 1.7.1 64-bit 4CPU     3217 Elo
Stockfish 1.7.1 64-bit 1CPU     3119 Elo
*Elo Difference:98 

------------------------------------------
CCRL 40/4
Rybka 4 64-bit 4CPU   3293 Elo
Rybka 4 64-bit 1CPU   3201 Elo
*Elo Difference:92


Stockfish 1.7.1 64-bit 4CPU   3234  Elo
Stockfish 1.7.1 64-bit 1CPU   3142 Elo
*Elo Difference:92

*****************************************
Overall Results (including all rating lists and time controls):
Deep Rybka 4 x64 4CPU 3271 Elo
Deep Rybka 4 x64 1CPU 3173 Elo
*Elo Difference:98

Stockfish 1.7.1 JA x64 4CPU 3205 Elo
Stockfish 1.7.1 JA x64 4CPU 3116 Elo
*Elo Difference:89 

Best,
Sedat

[Vinvin]

Code: Select all

CCRL 40/40
Rybka 4 64-bit 4CPU     3269 Elo
Rybka 3 64-bit 1CPU     3152 Elo
*Elo Difference:117 Elo

Different version ?

[Sedat Canbaz]

Code: Select all

CCRL 40/40
Rybka 4 64-bit 4CPU     3269 Elo
Rybka 3 64-bit 1CPU     3152 Elo
*Elo Difference:117 Elo

Different version ?

Oh,yes,my finger mistake (i have calculated the older version of Rybka),so here is the right statistics

Code: Select all

CCRL 40/40
Rybka 4 64-bit 4CPU     3269 Elo
Rybka 4 64-bit 1CPU     3184 Elo
*Elo Difference:85 Elo 


Overall Results (including all rating lists and time controls):
Deep Rybka 4 x64 4CPU 3271 Elo
Deep Rybka 4 x64 1CPU 3179 Elo
*Elo Difference:92

Stockfish 1.7.1 JA x64 4CPU 3205 Elo
Stockfish 1.7.1 JA x64 1CPU 3116 Elo
*Elo Difference:89

[Sedat Canbaz]

The Tournament with Ponder ON (via Netchess utility) is completed:


Note:the test with 'Ponder OFF' is coming soon

Games:
http://sedatchess.110mb.com/games/scct_ponder_on.zip

Best,
Sedat