DJ13 Welcome Test "Reloaded"

[TimoK]

Hi all,

after canceling the last test I setup a new one with slightly adjusted conditions. Junior has to play against 8 strong engines now, so after the test we'll have 8 x 60 games with very long time controls. This can't be of any stastical relevance, but it's good for reading some kind of trend. But the main goal is still to have fun with watching some (hopefully) interesting games!

All information to playing conditions, hyperlinks to live-broadcasts and game-downloads you can find on this page: http://www.team-oh.de/Computerschach/dj13.htm

I wish all much fun watching the games, maybe someone has time to give commentary to a some excellent games that might be played.

Best regards
Timo

P.S.: Test started already last night, but I had to restart the matches against the Top 4 because of some issue with the GUI. At that time Junior had already won a game against Rybka, lost one to Stockfish and drew against Houdini and Critter.

P.S.2: I'm not at home until sunday evening so I can't get a touch on the matches if something hangs. So don't wonder why it's not running (if an error occurs) - it will be fixed then after sunday.

[Jouni]

Very interesting and beats CCT14 6-0 in interest. Surprisingly Junior has not lost any game yet and even won against Houdini 2! BTW how many PC You use for this totally?

Jouni

[TimoK]

Hi Jouni,

thanks for your interest. Seems like Deep Junior likes the new conditions much more (4GB Hash, longer TC).

A total of 11 PCs are used for this match.

Best regards
Timo

[mclane]

amazing. the more games we see on "longer time controls" we recognize that junior plays very good under these conditions (hardware / time control). Lets call this environment A.

it seems junior has more problems on slower hardware with 1 thread and faster time controls. Lets call this environment B.

the results under B are not linear to the results on A if the computation time increases.

this is very interesting if we consider that MOST chess programs are tested under B conditions and the chess program teams believe that the elo strength they measure is linear on for faster hardware or longer computation time (conditions B).

junior shows that there are engines where this "law" is not working.
an exception to the law of linear chess strength increase with better hardware.


timo, why the hell do you have 11 pc's ??

[Uri Blass]

I do not know about a law of linear chess strength increase with better hardware so I see no exception to the rule.

I believe that there is diminishing returns and that programs do not earn the same from doubling the speed.

I believe that Junior earns more from doubling the speed and still I believe that Junior is going to lose in the 5 matches against the best programs

Expect loss against houdini(result now is 1-1)
Expect loss against Critter(result now is 1-1)
Expect loss against Stockfish(result now is 1-1)
Expect loss against Rybka(Rybka is leading 2-1)
Expect loss against Komodo(Result now is 2-2)

For results against Naum Chiron and Fritz then the advantage that they have at blitz is not enough for me to have confidence that they are better at long time control.

[Werewolf]

I do not know about a law of linear chess strength increase with better hardware so I see no exception to the rule.

I believe that there is diminishing returns and that programs do not earn the same from doubling the speed.

I believe that Junior earns more from doubling the speed and still I believe that Junior is going to lose in the 5 matches against the best programs

Expect loss against houdini(result now is 1-1)
Expect loss against Critter(result now is 1-1)
Expect loss against Stockfish(result now is 1-1)
Expect loss against Rybka(Rybka is leading 2-1)
Expect loss against Komodo(Result now is 2-2)

For results against Naum Chiron and Fritz then the advantage that they have at blitz is not enough for me to have confidence that they are better at long time control.

This is interesting. I wonder if Junior benefits excellently from double time / speed AND also makes good use of hardware with many cores.

Being able to support 40 cores might hint at that.

[TimoK]

Hi Thorsten,

you're totally right: I'm crazy using so much computers just for hobby purposes. My wife is telling me that all the time and is angry about the electricity bill...

But I can tell you it is not as expensive as you might think. The PCs are all small cubes (Micro-ATX) fitting into an IKEA Expedit rack. Since I'm working for an IT company I got an excellent source for cheap hardware prices. CPUs are undervolted so the whole project is "Green-IT" approved...

Let's see if Junior will do better than in its first try (first match where it got beaten very badly by Stockfish and Critter). At the moment I think we can see it has a good fight with its opponents and is far away from a humiliation.

Best regards
Timo

[TimoK]

Hi all,

one of the computers had a crash last night (maybe too much undervolted), so the match against Stockfish was hanging. The game that they were playing at that time was broken off at a position that was clearly won by Stockfish. I checked that with 6-men Tablebases and adjusted the game as a win for Stockfish. Now the match continues normally.

Best regards
Timo

[FriedmannC]

It's great you're doing this test! Thank you very much for providing us the links to download these high-quality games! It would be very interesting to see the following matches: Deep Junior 13-Deep Shredder 12, Deep Junior 13-Zappa Mexico II (on big hardware and at long time controls, Zappa is still very strong), Deep Junior 13- Vitruvius 1 and Deep Junior 13-Deep Hiarcs 13.2. BTW, do you have any idea when Deep Hiarcs 14 will be released? Keep up with your great work! Best regards!

[TimoK]

Hi Teodoriu,

thanks for your interest and for your warm words! I think I'll probably setup the match DJ13 vs. Deep Shredder 12 in the future. I'm not so sure about Zappa. Do you think it is still strong enough? Maybe you are right and it'll do well with 6 threads and long TC.

Vitruvius is a brand new engine. I haven't seen any results yet. And I'm not sure if I should buy it. Maybe when first match results in some rating lists are published and I see that it's clearly stronger than Ivanhoe.

Hiarcs 13.2 will soon be overthrown by Hiarcs 14. I guess it's only a matter of a few weeks when Hiarcs 14 will be released. Of course DJ13 will play a match against the new Hiarcs then.

Best regards
Timo