3Champs reloaded

[lkaufman]

Larry - as you know, I'm planning a "LTC" 16-core 2687W, no ponder, event between H4ProB, K1142 and SF DD (or TBD), and ask for your, Marco and Robert's suggestions on format, along with anyone else's.

First decision: 8 cores each with ponder or 16 cores each without ponder. I favor 16 cores without ponder, partly because Clemens is running the ponder on test, no need to duplicate that. Also, in my opinion using twice as many cores raises the quality more than having ponder on, and also ponder on somewhat increases the luck factor as guessing the opponent's move is mostly luck.
Second decision: Time limit. It's hard to improve on the TCEC time limit of 2 hours plus 30" which corresponds to a major time limit in human play and has about the ideal ratio of base time to increment. But then you are pretty much just rerunning TCEC for the top three finishers. If you want to have the highest quality games on record, go to 2 1/2 hours plus 30", which is fairly comparable witih the traditional forty moves in two hours on average.
Third decision: hash table size. Half of your total memory is reasonable for long games like this. using many cores.
Fourth decision: Tablebases. Well, since Komodo doesn't yet support them I would of course prefer that they not be used, but I'm not neutral. I understand that the Stocfish team volunarily voted not to use them in the TCEC final; I don't know their reasons. They probably improve the quality of endgame play a bit perhaps depending on which TBs are used and how fast is the access. In any case I think it's fine to have the GUI use TBs to adjudicate (or rapidly play out) the games.
Fifth decision: books or opening test suites. I would favor test suites preferably ending after around six moves or so to give the engines the most chances to outplay their opponents.
Sixth Decision: Color reversal for each opening. Do it!
Seventh Decision: Settings. Follow the recommendations of the respective teams as stated on the internet.

Probably I've forgotten something!

Larry

[ouachita]

1. 16 cores
2. no ponder
3. TC: 150+30
4. 16 or 32GB hash table
6. Tablebases: I currently use Nalimov TB when relevant and will have to purchase Deep Fritz 14 if this event is to use syzygy TB.
7. Opening test suites: Noomen - six moves?
8. Color reversal for each opening.
9. Settings: Will need some help from someone on this - we could meet at webex.com.
10. 100 games each?

[lkaufman]

1. 16 cores
2. no ponder
3. TC: 150+30
4. 16 or 32GB hash table
6. Tablebases: I currently use Nalimov TB when relevant and will have to purchase Deep Fritz 14 if this event is to use syzygy TB.
7. Opening test suites: Noomen - six moves?
8. Color reversal for each opening.
9. Settings: Will need some help from someone on this - we could meet at webex.com.
10. 100 games each?

How many openings in the Noomen six move set? It has to be at least fifty to run 100 games. Assuming it is, that all looks fine to me. Try to figure out from the manuals/readme files what settings each engine recommends for long time control games; if they don't say you would either use the defaults or ask a member of each team to make recommendations.

[Milos]

9. Settings: Will need some help from someone on this -

The biggest difference in settings probably among all 3 engines makes contempt factor of Houdini. If you want the highes quality use contempt=0.
Everything else like split depth, idle threads, large pages (which compile to use), etc. has less than 3Elo effect. Maybe draw rate of Komodo, but Larry should know that.

[lkaufman]

9. Settings: Will need some help from someone on this -

The biggest difference in settings probably among all 3 engines makes contempt factor of Houdini. If you want the highes quality use contempt=0.
Everything else like split depth, idle threads, large pages (which compile to use), etc. has less than 3Elo effect. Maybe draw rate of Komodo, but Larry should know that.

I agree; contempt should be set to 0 for Houdini and DrawScore to 0 for Komodo since the three engines are very close in strength. Large pages should not be used as they would have rather random effects on play, helping at times then hurting at other times. For Komodo the most important change from defaults is to use Table Memory of 256 for long time control games like this.

[ouachita]

Larry,
I now have the Syzygy 3-4-5 TB, but I'm not sure that it would make a lot of sense for me to stage a LTC event between the Top Guns (K1142, SF DD and H4Pro) with two using TB and one not. Seems to me that it should all or none.

[lkaufman]

Larry,
I now have the Syzygy 3-4-5 TB, but I'm not sure that it would make a lot of sense for me to stage a LTC event between the Top Guns (K1142, SF DD and H4Pro) with two using TB and one not. Seems to me that it should all or none.

That is of course your call; there are arguments both ways. Some testers use TBs, some do not. I note that Stockfish voluntarily refrained from using them for the TCEC final, but I don't know why.

[shrapnel]

I note that Stockfish voluntarily refrained from using them for the TCEC final, but I don't know why.

Yep ! BIG mistake by the Stockfish Team....good for you !

[Vinvin]

Hi all computerchess enthusiasts

for the honor to Don Dailey

4 days ago I started a new run of my 3Champs Tournament.
This time the 3 participants are running on a 24core machine, using 12cpu:

Round Robin Tournament 90min +15 sec, 50rounds, 150 games
Ponder = ON
engine = 12 cpu
hashtables = 4 GB
learning = OFF
hyperthreading = OFF
large pages = OFF
tablebases (engine) = OFF
tablebases GUI = ON ( small set 80GB)
opening book = Fritz 14 (5moves), mode "normal", no reversed color games
hardware = Intel E5-2697 v2 @ 3.00 GHz (turboMode)

Engines:

Houdini 4.0 pro, split depth 12
Stockfish 241113, sleeping threads idle ON, min split depth 7
Komodo 1142

livegames : http://www.clemens-keck.de/livegames/

ReplayZone : http://www.clemens-keck.de/ReplayZone/3champs2013.htm

This tour will take a while. I think 4 - 5 weeks.

regards, Clemens Keck

PS.: congratulations to Komodo 1142 for winning the grand TCEC !!

Stockfish just reached the 1st place ... but all are close ...

3Champs2013 2013

Code: Select all

1 	Stockfish 241113 64 SSE4.2  	13.5 / 26 	 
2 	Houdini 4 Pro x64 x12	       13.0 / 26 	 
3 	Komodo 1142.00 64-bit x12-2	 12.5 / 26

[ouachita]

Very impressive machine you have and nice event - can you also show us a conventional cross table so we can readily see how each engine is doing versus the other?