Chess.com 2018 computer chess championship

[AndrewGrant]

https://pastebin.com/raw/FxRnwhHF

Ethereal, Senpai, Nemorino, Shredder, Nirvanachess are shown to exhibit the same behavior ...

Random depth 1 searches taking thousand of times longer than they should.

Most interesting, is that games are hit or miss. There are a handful of cases where it happens to Ethereal and Shredder, but some games are perfectly clear.

EDIT : Furthermore, none of the games posted above contained Leela. Further proof that hyperthreads and pondering was a poor idea.

[zullil]

EDIT : Furthermore, none of the games posted above contained Leela. Further proof that hyperthreads and pondering was a poor idea.

As many of us pointed out in advance. So the fiasco could have been avoided.

[CMCanavessi]

Maybe the rules can be changed for the second stage and stop this ponder on hyperthreads madness. It's not like the top 8 would be any different without ponder anyways, or maybe 1 engine out of 8.

[AndrewGrant]

Maybe the rules can be changed for the second stage and stop this ponder on hyperthreads madness. It's not like the top 8 would be any different without ponder anyways, or maybe 1 engine out of 8.

I hope so. Dropping pondering will fix things, and be completely elo neutral. Dropping hyper threads will only cause a ~10 elo loss when playing vs Leela, which I think is acceptable.

[elcabesa]

I don't understand the problem,
CCCC has defined some rules, maybe they has something to fix on server side too, but it seems to me that some chess engine shown some problem under this kind of setup.
Vajolet has some problem too, but hey, they are problem of Vajolet, I have to dig and find the bug, and I hope to have fixed them.

Don't you think that those program can have problem too with pondering and games without adjudication since not too much tournament have been done with this setup?

why always finger pointing?

[elcabesa]

From the logs provided to me ...

info depth 1 seldepth 3 score cp 44 time 6093 nodes 202659512 nps 33255000 tbhits 2067 hashfull 53 pv b4d5

6093ms for a depth 1 search? It looks to me like the main thread got locked out of CPU time, and probably some of the helpers.

it looks like it has searched 202659512 nodes... probably a bug on ponder code?

[Guenther]

https://pastebin.com/raw/FxRnwhHF

Ethereal, Senpai, Nemorino, Shredder, Nirvanachess are shown to exhibit the same behavior ...

Random depth 1 searches taking thousand of times longer than they should.

Most interesting, is that games are hit or miss. There are a handful of cases where it happens to Ethereal and Shredder, but some games are perfectly clear.

EDIT : Furthermore, none of the games posted above contained Leela. Further proof that hyperthreads and pondering was a poor idea.

There is much more to find with different parameters.
e.g. I found over 360 moves with depth 2-6 and time >10s! in the same pgn from this afternoon...

[AndrewGrant]

From the logs provided to me ...

info depth 1 seldepth 3 score cp 44 time 6093 nodes 202659512 nps 33255000 tbhits 2067 hashfull 53 pv b4d5

6093ms for a depth 1 search? It looks to me like the main thread got locked out of CPU time, and probably some of the helpers.

it looks like it has searched 202659512 nodes... probably a bug on ponder code?

No ... that shows that some threads got CPU time, but others + main thread did not.

In this case, 33mnps is < 50% the expected NPS for the given position.

Also, seldepth=3. Max tree size is <= 256^3. Its not a ponder bug, its a hardware issue.

[elcabesa]

In my programming experience, the problem I had with my software were rarely hardware or compiler bug, lot of time they were software bug created by myself.

[AndrewGrant]

In my programming experience, the problem I had with my software were rarely hardware or compiler bug, lot of time they were software bug created by myself.

Okay. This is not such an example. This was expected by everyone in this thread. No one listened.

Unless you think that 5 engines all display the same bug, and for some reason CCCC is the first group to discover it.