3Champs reloaded

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
lkaufman
Posts: 3873
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: 3Champs reloaded

Post by lkaufman » Mon Dec 02, 2013 8:09 pm

ouachita wrote:Larry - as you know, I'm planning a "LTC" 16-core 2687W, no ponder, event between H4ProB, K1142 and SF DD (or TBD), and ask for your, Marco and Robert's suggestions on format, along with anyone else's.
First decision: 8 cores each with ponder or 16 cores each without ponder. I favor 16 cores without ponder, partly because Clemens is running the ponder on test, no need to duplicate that. Also, in my opinion using twice as many cores raises the quality more than having ponder on, and also ponder on somewhat increases the luck factor as guessing the opponent's move is mostly luck.
Second decision: Time limit. It's hard to improve on the TCEC time limit of 2 hours plus 30" which corresponds to a major time limit in human play and has about the ideal ratio of base time to increment. But then you are pretty much just rerunning TCEC for the top three finishers. If you want to have the highest quality games on record, go to 2 1/2 hours plus 30", which is fairly comparable witih the traditional forty moves in two hours on average.
Third decision: hash table size. Half of your total memory is reasonable for long games like this. using many cores.
Fourth decision: Tablebases. Well, since Komodo doesn't yet support them I would of course prefer that they not be used, but I'm not neutral. I understand that the Stocfish team volunarily voted not to use them in the TCEC final; I don't know their reasons. They probably improve the quality of endgame play a bit perhaps depending on which TBs are used and how fast is the access. In any case I think it's fine to have the GUI use TBs to adjudicate (or rapidly play out) the games.
Fifth decision: books or opening test suites. I would favor test suites preferably ending after around six moves or so to give the engines the most chances to outplay their opponents.
Sixth Decision: Color reversal for each opening. Do it!
Seventh Decision: Settings. Follow the recommendations of the respective teams as stated on the internet.

Probably I've forgotten something!

Larry

ouachita
Posts: 454
Joined: Tue Jan 15, 2013 3:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: 3Champs reloaded

Post by ouachita » Mon Dec 02, 2013 9:56 pm

1. 16 cores
2. no ponder
3. TC: 150+30
4. 16 or 32GB hash table
6. Tablebases: I currently use Nalimov TB when relevant and will have to purchase Deep Fritz 14 if this event is to use syzygy TB.
7. Opening test suites: Noomen - six moves?
8. Color reversal for each opening.
9. Settings: Will need some help from someone on this - we could meet at webex.com.
10. 100 games each?
SIM, PhD, MBA, PE

lkaufman
Posts: 3873
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: 3Champs reloaded

Post by lkaufman » Mon Dec 02, 2013 10:37 pm

ouachita wrote:1. 16 cores
2. no ponder
3. TC: 150+30
4. 16 or 32GB hash table
6. Tablebases: I currently use Nalimov TB when relevant and will have to purchase Deep Fritz 14 if this event is to use syzygy TB.
7. Opening test suites: Noomen - six moves?
8. Color reversal for each opening.
9. Settings: Will need some help from someone on this - we could meet at webex.com.
10. 100 games each?
How many openings in the Noomen six move set? It has to be at least fifty to run 100 games. Assuming it is, that all looks fine to me. Try to figure out from the manuals/readme files what settings each engine recommends for long time control games; if they don't say you would either use the defaults or ask a member of each team to make recommendations.

Milos
Posts: 3403
Joined: Wed Nov 25, 2009 12:47 am

Re: 3Champs reloaded

Post by Milos » Mon Dec 02, 2013 11:35 pm

ouachita wrote:9. Settings: Will need some help from someone on this -
The biggest difference in settings probably among all 3 engines makes contempt factor of Houdini. If you want the highes quality use contempt=0.
Everything else like split depth, idle threads, large pages (which compile to use), etc. has less than 3Elo effect. Maybe draw rate of Komodo, but Larry should know that.

lkaufman
Posts: 3873
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: 3Champs reloaded

Post by lkaufman » Mon Dec 02, 2013 11:47 pm

Milos wrote:
ouachita wrote:9. Settings: Will need some help from someone on this -
The biggest difference in settings probably among all 3 engines makes contempt factor of Houdini. If you want the highes quality use contempt=0.
Everything else like split depth, idle threads, large pages (which compile to use), etc. has less than 3Elo effect. Maybe draw rate of Komodo, but Larry should know that.
I agree; contempt should be set to 0 for Houdini and DrawScore to 0 for Komodo since the three engines are very close in strength. Large pages should not be used as they would have rather random effects on play, helping at times then hurting at other times. For Komodo the most important change from defaults is to use Table Memory of 256 for long time control games like this.

ouachita
Posts: 454
Joined: Tue Jan 15, 2013 3:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: 3Champs reloaded

Post by ouachita » Tue Dec 03, 2013 3:47 pm

Larry,
I now have the Syzygy 3-4-5 TB, but I'm not sure that it would make a lot of sense for me to stage a LTC event between the Top Guns (K1142, SF DD and H4Pro) with two using TB and one not. Seems to me that it should all or none.
SIM, PhD, MBA, PE

lkaufman
Posts: 3873
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: 3Champs reloaded

Post by lkaufman » Tue Dec 03, 2013 3:59 pm

ouachita wrote:Larry,
I now have the Syzygy 3-4-5 TB, but I'm not sure that it would make a lot of sense for me to stage a LTC event between the Top Guns (K1142, SF DD and H4Pro) with two using TB and one not. Seems to me that it should all or none.
That is of course your call; there are arguments both ways. Some testers use TBs, some do not. I note that Stockfish voluntarily refrained from using them for the TCEC final, but I don't know why.

shrapnel
Posts: 1291
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: 3Champs reloaded

Post by shrapnel » Tue Dec 03, 2013 4:12 pm

lkaufman wrote: I note that Stockfish voluntarily refrained from using them for the TCEC final, but I don't know why.
Yep ! BIG mistake by the Stockfish Team....good for you ! :)
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Vinvin
Posts: 4472
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: 3Champs reloaded

Post by Vinvin » Wed Dec 04, 2013 1:56 am

Hugo wrote:Hi all computerchess enthusiasts

for the honor to Don Dailey

4 days ago I started a new run of my 3Champs Tournament.
This time the 3 participants are running on a 24core machine, using 12cpu:

Round Robin Tournament 90min +15 sec, 50rounds, 150 games
Ponder = ON
engine = 12 cpu
hashtables = 4 GB
learning = OFF
hyperthreading = OFF
large pages = OFF
tablebases (engine) = OFF
tablebases GUI = ON ( small set 80GB)
opening book = Fritz 14 (5moves), mode "normal", no reversed color games
hardware = Intel E5-2697 v2 @ 3.00 GHz (turboMode)

Engines:

Houdini 4.0 pro, split depth 12
Stockfish 241113, sleeping threads idle ON, min split depth 7
Komodo 1142

livegames : http://www.clemens-keck.de/livegames/

ReplayZone : http://www.clemens-keck.de/ReplayZone/3champs2013.htm

This tour will take a while. I think 4 - 5 weeks.

regards, Clemens Keck

PS.: congratulations to Komodo 1142 for winning the grand TCEC !!
Stockfish just reached the 1st place ... but all are close ...

3Champs2013 2013

Code: Select all

1 	Stockfish 241113 64 SSE4.2  	13.5 / 26 	 
2 	Houdini 4 Pro x64 x12	       13.0 / 26 	 
3 	Komodo 1142.00 64-bit x12-2	 12.5 / 26

ouachita
Posts: 454
Joined: Tue Jan 15, 2013 3:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: 3Champs reloaded

Post by ouachita » Wed Dec 04, 2013 2:21 am

Very impressive machine you have and nice event - can you also show us a conventional cross table so we can readily see how each engine is doing versus the other?
SIM, PhD, MBA, PE

Post Reply