DJ13 Welcome Test "Reloaded"

[Houdini]

Yeah, Rybka stays unbeaten until now, but also Komodo hasn't lost a single game, yet. Maybe these two engines like the long TCs most?!

Strange conclusion.
Is the primary goal in chess to remain unbeaten, or is the goal to score as many points as possible?

Houdini 2 scores 17-5
Rybka 4.1 scores 18-9
Komodo 4 scores 18-7

Which is best?

[TimoK]

Hello Robert,

Is the primary goal in chess to remain unbeaten, or is the goal to score as many points as possible?

well that depends on your own point of view I would say. But anyways, I would agree that it's better to lose a few games, but to score better than the others in the end. Besides I like engines more that play a bit more risky - so that they are able to earn beautiful wins, but sometimes have to concede defeat. So from my point of view Houdini plays more interesting (and scores better!) - but there may be a few people who see it in a different way. When they use an engine for correspondence chess for example they'll try not to lose at whatever it costs - even if they pass over the possible win sometimes.

Best regards
Timo

P.S.: Did you follow Clemens' "Myth Buster" test (DJ13 vs. Houdini 1.5a)? In that test DJ13 scored much better than in my test. Would you say that DJ13 was lucky back then? Or is Houdini 2.0c better than 1.5a in long TC? Well, of course both tests have too few games to draw any conclusion of statistical relevance...

[FriedmannC]

Hi Timo, I guess when the authors of Deep Junior asserted that their chess program has a giant 3200 ELO, maybe they didn't specify that this could be reached only by their cluster, or by running Deep Junior on at least 16 cores, who knows? So far it didn't prove at all to be that strong l look forward to see how Zappa Mexico II behaves against Deep Junior 13.
All the best,
Catalin

[FriedmannC]

I would LIKE to see 3 matches, ONLY 3 matches on long time controls and on very big hardware:
DEEP RYBKA 4.1-HOUDINI 2.c
KOMODO 4 MP-HOUDINI 2.c
STOCKFISH 2.2.2-HOUDINI 2.c
If HOUDINI 2.c wins them all, no matter how tight, it would clearly mean it's the best on the conditions mentioned above, as we know that on LTC , DEEP RYBKA 4.1, KOMODO 4 MP (not yet on the market) and STOCKFISH 2.2.2 are very tough competitors! You may ask why I want to see these matches: Well, Robert will launch HOUDINI 3 in September and he stated it would be considerably stronger than its predecessor! So the final result of those matches are already known if instead of HOUDINI 2, we have HOUDINI 3!
Best regards,
Catalin

[TimoK]

I would LIKE to see 3 matches, ONLY 3 matches on long time controls and on very big hardware:
DEEP RYBKA 4.1-HOUDINI 2.c
KOMODO 4 MP-HOUDINI 2.c
STOCKFISH 2.2.2-HOUDINI 2.c

Hi Catalin,

I don't have very big hardware (only AMD Phenom II x6), so I'm afraid some other person has to do that test. But this would be very thrilling indeed. I think Houdini will prevail in the end and win all 3 matches. But maybe the Komodo-Team has made more improvement since Komodo 4 SP, so they can close up to Houdini 2.

Best regards
Timo

[IGarcia]

Best regards
Timo

Games completed today. Great tournament, thanks for running and sharing.
Some comments:

1) DJ13 against Stockish, the first 3 games are missing.
2) some games draw fast:
DJ13-Stockfish round 39 (9 moves)
DJ13-Critter round 45 (12 moves)
Both with ECO D80

An endgame found, testing tablebases, Chiron1.1a - DJ13, game 2 this position is reach at move 177 chiron played Bb7?.
Was a blunder? or game was already lost several moves before?

[d]8/1B6/8/p3p3/5k2/K1P5/8/3b4 b - - 87 177


DJ13 finds the correct reply very fast!

Critter 1.4 also plays 177. Bb7 and has big trouble to find 177...Bf3 wich is rejected in favor 177...Ke3.

But after Bf3 white has no options:
Bxf3 = mate 66 (tablebase) OR moving the bishop outside digonal: Ba6, Bc8 loses.
So Bf3 is the way to go.

[Jouni]

Thanks for great match! Final score didn't change after around 20 games: top engines score 31,5% (IPON 21,5%). You don't need 2000 games with LTC? My wish for next matches:

Houdini - Rybka
Critter - Stockfish

[IGarcia]

Thanks for great match! Final score didn't change after around 20 games: top engines score 31,5% (IPON 21,5%). You don't need 2000 games with LTC? My wish for next matches:

Houdini - Rybka
Critter - Stockfish

this are the IPON results DJ13

Code: Select all

46 Deep Junior 13           2749 2850.0 (1137.5 : 1712.5)
                                   150.0 ( 31.0 : 119.0) Houdini 2.0 STD          3017
                                   150.0 ( 26.5 : 123.5) Komodo 4                 2976
                                   150.0 ( 34.5 : 115.5) Critter 1.4a             2975
                                   150.0 ( 33.5 : 116.5) Deep Rybka 4.1           2953
                                   150.0 ( 36.5 : 113.5) Stockfish 2.2.2 JA       2951
                                   150.0 ( 59.0 :  91.0) Chiron 1.1a              2830
                                   150.0 ( 51.5 :  98.5) Naum 4.2                 2827
                                   150.0 ( 60.0 :  90.0) Fritz 13 32b             2819
                                   150.0 ( 73.5 :  76.5) Deep Shredder 12         2800
                                   150.0 ( 65.0 :  85.0) Gull 1.2                 2794
                                   150.0 ( 60.5 :  89.5) Deep Sjeng c't 2010 32b  2790
                                   150.0 ( 68.5 :  81.5) Spike 1.4 32b            2786
                                   150.0 ( 67.0 :  83.0) Protector 1.4.0          2761
                                   150.0 ( 69.0 :  81.0) spark-1.0                2760
                                   150.0 ( 72.0 :  78.0) Hannibal 1.1             2755
                                   150.0 ( 71.0 :  79.0) HIARCS 13.2 MP 32b       2751
                                   150.0 ( 82.5 :  67.5) Zappa Mexico II          2718
                                   150.0 ( 88.0 :  62.0) Deep Onno 1-2-70         2686
                                   150.0 ( 88.0 :  62.0) Strelka 2.0 B            2670

[FriedmannC]

Thank you very much for the provided games, Timo! You did a great work! After you finish testing Deep Junior 13, I hope you will set up these matches:
DEEP RYBKA 4.1 6 CPU - HOUDINI 2.c 6 CPU
STOCKFISH 2.2.2 6 CPU - HOUDINI 2.c 6 CPU
HOUDINI 2.c 1 CPU - KOMODO 4 1 CPU - At long time control too (so we could finally figure out who is really the best, as there is way too much discussion about this confrontation)
CRITTER 1.4 6 CPU - HOUDINI 2.c 6 CPU
Best regards,
Catalin

[IGarcia]

And this the LTC match results http://www.team-oh.de/Computerschach/dj13.htm

Code: Select all

Junior-Houdini
--------------------------------------------------------------------------------------------------
 1: Houdini 2.0c Pro x64  45.0 / 60   110=1=11111=1=01=1111=10==11=111111=111111==11==0===11==1===
 2: Deep Junior 13        15.0 / 60   001=0=00000=0=10=0000=01==00=000000=000000==00==1===00==0===
--------------------------------------------------------------------------------------------------
60 games: +34 =22 -4

Junior-Komodo
-----------------------------------------------------------------------------------------------------
 1: Komodo64 SSE Version 4  42.0 / 60   ====1=1=11==11=1=11111==1=====11===11=01=1===110=1=1==1=1==1
 2: Deep Junior 13          18.0 / 60   ====0=0=00==00=0=00000==0=====00===00=10=0===001=0=0==0=0==0
-----------------------------------------------------------------------------------------------------
60 games: +26 =32 -2

Junior-Rybka
-------------------------------------------------------------------------------------------------------
 1: Deep Rybka 4.1 SSE42 x64  42.0 / 60   =======1========1===11111=1==1==11=11=111=111=1==11=1=1=====
 2: Deep Junior 13            18.0 / 60   =======0========0===00000=0==0==00=00=000=000=0==00=0=0=====
-------------------------------------------------------------------------------------------------------
60 games: +24 =36 -0

Junior-Critter
-------------------------------------------------------------------------------------------------------
 1: Critter 1.4a 64-bit SSE4  39.5 / 60   ===1=====1==1=1==111==1=1=1=1==1==1=1=000==11111=11===1=====
 2: Deep Junior 13            20.5 / 60   ===0=====0==0=0==000==0=0=0=0==0==0=0=111==00000=00===0=====
-------------------------------------------------------------------------------------------------------
60 games: +22 =35 -3

Junior-Stockfish
-------------------------------------------------------------------------------------------------------
 1: Stockfish 2.2.2 JA SSE42  38.0 / 57   ==1===========1=======10==1=110=1===1111======1=111=====111=
 2: Deep Junior 13            22.0 / 57   ==0===========0=======01==0=001=0===0000======0=000=====000=
-------------------------------------------------------------------------------------------------------
60 games: +18 =40 -2

Junior-Naum
---------------------------------------------------------------------------------------------
 1: Naum 4.2        32.0 / 60   ========1===1=0=====1==10==0=101=0111==1=110==00=11=0====0==
 2: Deep Junior 13  28.0 / 60   ========0===0=1=====0==01==1=010=1000==0=001==11=00=1====1==
---------------------------------------------------------------------------------------------
60 games: +14 =36 -10

Junior-Chiron
------------------------------------------------------------------------------------------------
 1: Chiron 1.1a 64bit  32.0 / 60   =0=11==00==0==1==0=0==01101==1=11=11=111=0=11=01=0=0=1=0=100
 2: Deep Junior 13     28.0 / 60   =1=00==11==1==0==1=1==10010==0=00=00=000=1=00=10=1=1=0=1=011
------------------------------------------------------------------------------------------------
60 games: +19 =26 -15

Junior-Fritz
---------------------------------------------------------------------------------------------
 1: Fritz 13        30.5 / 60   0=1==1=0====01===1111===00==01==0==010=10=11=1=0==110=0==0==
 2: Deep Junior 13  29.5 / 60   1=0==0=1====10===0000===11==10==1==101=01=00=0=1==001=1==1==
---------------------------------------------------------------------------------------------
60 games: +15 =31 -14