Chess.com 2018 computer chess championship

[Werewolf]

If there are 2 SSD, access is not a problem, so should be fine.

Aren't the two SSDs in RAID 1 though, rather than RAID 0?

[zullil]

Note that those tests were run on a single 192-core machine, so the second test merely pitted Stockfish against itself with hyperthreading enabled in one instance, and disabled in the other. At the level of the top engines, I don't think anyone would consider that to be poor scaling at all. I'm actually astonished that Stockfish on 192 cores gets 22±10 Elo stronger simply by turning on hyperthreading.

http://tests.stockfishchess.org/tests/v ... 02b2e60162

Hyperthreading was enabled in BIOS, providing 384 logical cores. One Stockfish was limited to Threads=192; the other used Threads=384.

I find myself wanting to disbelieve the result, but the test seems proper---assuming that the 192 threads of the first Stockfish were generally scheduled on distinct physical cores (which any non-defective OS would do).

Testing time control was 60+0.6. I wonder how the results might have differed with a time control significantly longer than game-in-a-minute.

[jdart]

Stockfish is a beast. But I think these high core counts don't benefit all engines equally. Some scale better than others.

If it were up to me I'd run these tournaments on a modest core count (<=20) but a high-end CPU and disable hyperthreading.

You wouldn't get quite the NPS you see in TCEC but I think you'd get more than half of it, maybe 3/4 even. And maybe fewer crashes, because few engines can or do test on 43 cores.

I don't really have an issue with ponder on. But as said earlier in this thread, I'd turn off hyperthreading, and maybe give each engine its own TB set, although I am not sure w/o testing how much that matters (my engine at least caches TB hits in the hash table, so it would be fetching a lot from its own memory).

--Jon

[zullil]

You wouldn't get quite the NPS you see in TCEC but I think you'd get more than half of it, maybe 3/4 even.
--Jon

Well, since everyone has gone to lazy smp, those node counts simply reflect the same nodes being looked at with high multiplicity. Still waiting for someone to (re)implement a high-quality, non-lazy smp.

[Milos]

Well, since everyone has gone to lazy smp, those node counts simply reflect the same nodes being looked at with high multiplicity. Still waiting for someone to (re)implement a high-quality, non-lazy smp.

You never look the same nodes. You just look the nodes that were supposed to be cut .
But most often you actually look at nodes that would have been pruned in single core search. And more then expected you find something useful there .
Beside Bob's up to depth 12, 30 years old results, there is no actual proof that YBWC or DTS work any better.

[zullil]

Well, since everyone has gone to lazy smp, those node counts simply reflect the same nodes being looked at with high multiplicity. Still waiting for someone to (re)implement a high-quality, non-lazy smp.

there is no actual proof that YBWC or DTS work any better.

This lack of proof frustrates me tremendously. Hard to accept that lazy smp works.

[Milos]

Well, since everyone has gone to lazy smp, those node counts simply reflect the same nodes being looked at with high multiplicity. Still waiting for someone to (re)implement a high-quality, non-lazy smp.

there is no actual proof that YBWC or DTS work any better.

This lack of proof frustrates me tremendously. Hard to accept that lazy smp works.

One can prove that you reach higher depth faster with YBWC or DTS, but reaching higher depth faster (or slower in case of LazySMP) means actually nothing especially with LazySMPs weird tree shape.

[zullil]

Well, since everyone has gone to lazy smp, those node counts simply reflect the same nodes being looked at with high multiplicity. Still waiting for someone to (re)implement a high-quality, non-lazy smp.

there is no actual proof that YBWC or DTS work any better.

This lack of proof frustrates me tremendously. Hard to accept that lazy smp works.

One can prove that you reach higher depth faster with YBWC or DTS, but reaching higher depth faster (or slower in case of LazySMP) means actually nothing especially with LazySMPs weird tree shape.

Right. And this is what frustrates me. Higher depth should mean better performance. Provided you're searching the right line.

[kranium]

There's no indication that the CPU engines will be given full access to the anemic CPU machine during games against Leela. The official documentation simply states the CPU engines will be limited to 46 threads and 8192 MB of hash without mention of any exceptions.

The CPU engines will be the only engine running on the system, and will have full access to the processors (w/ 46 threads addressing 48 physical CPUs) while playing vs. Lc0
I recently ran Ethereal in this manner and the NPS was impressive, so it's clear that that the CPU engines will be running at full or near full strength, and should perform well in games against Lc0.

With facts like these it becomes hard not see an agenda, and the claim that "the CCCC will run on cutting-edge technology to generate the best possible chess" becomes laughable.

I can assure you there's no 'agenda'. We are simply trying to provide a fun and entertaining event that includes high-level and competitive chess. All of which I can guarantee. We are of course very interested in the Lc0 project, and are fascinated by it's unusual yet strong style of play (as are many), and have tried to provide it with a suitable platform in an effort to help it play to it's full and current potential. If funding allowed we would have gone with 8x Tesla v100s. We're not overly concerned if the CPU vs GPU hardware platforms equate perfectly or not. This is not an official (software) world championship event.

These events will be run 24/7/365, a new one each month, with a enormous variety of formats and configurations.
Time Control: blitz, rapid, and classical
Ponder: on or off
Opening book: no book, 2 moves, 4 moves, 6 move, and 8 moves, and on occasion: thematic openings
etc.

We've specifically chosen ponder 'on' for this 1st inaugural event for 1 reason only...the chess.com team feels very strongly about letting viewers see both engine's thinking simultaneously, in an effort to highlight the new web UI capabilities (which we've written from the ground up with the newest modern web technologies). This includes a groundbreaking "live PVs" web broadcast feature.

I do hope you can enjoy it instead of picking it apart, leveling accusations, and being so critical of the effort. I certainly realize there's a lot of 'haters', and egotistical people who believe they could do much better, but please don't forget: these events are being produced, funded, and broadcast for free by chess.com for the benefit of community...as the amount of new subscribers necessary to pay for it all is astronomical.

[jdart]

This lack of proof frustrates me tremendously. Hard to accept that lazy smp works.

I was skeptical about it too but I'm willing to accept the empirical evidence that it does work.

YBWC has some inherent limits and one of the main ones is that it restricts the threads available to search a node. So then you can wind up with threads that are idle because they are not meeting the YBWC conditions.

Bob put some interesting mods into the latest release of Crafty that attempt to increase thread usage. One is doing a pre-emptive split, where you create a split point in advance of actually having a thread to service it. I am sure he tested this and got some gain but I tried some of these mods and could not get to to work well for me.

I think also there are probably possible improvements to LazySMP. I still have support in my code for having two or more threads access the same move generator, each getting a subset of the moves. So you could, as you distribute the work across threads, group some of them so that they search the same depth but are made to search distinct moves. I am not sure this would help but it might.

--Jon