Chess.com 2018 computer chess championship

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
AndrewGrant
Posts: 494
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: Chess.com 2018 computer chess championship

Post by AndrewGrant » Wed Sep 12, 2018 2:30 pm

https://pastebin.com/raw/FxRnwhHF

Ethereal, Senpai, Nemorino, Shredder, Nirvanachess are shown to exhibit the same behavior ...

Random depth 1 searches taking thousand of times longer than they should.

Most interesting, is that games are hit or miss. There are a handful of cases where it happens to Ethereal and Shredder, but some games are perfectly clear.

EDIT : Furthermore, none of the games posted above contained Leela. Further proof that hyperthreads and pondering was a poor idea.

zullil
Posts: 5684
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Chess.com 2018 computer chess championship

Post by zullil » Wed Sep 12, 2018 2:40 pm

AndrewGrant wrote:
Wed Sep 12, 2018 2:30 pm
EDIT : Furthermore, none of the games posted above contained Leela. Further proof that hyperthreads and pondering was a poor idea.
As many of us pointed out in advance. So the fiasco could have been avoided.

User avatar
CMCanavessi
Posts: 844
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: Chess.com 2018 computer chess championship

Post by CMCanavessi » Wed Sep 12, 2018 3:38 pm

Maybe the rules can be changed for the second stage and stop this ponder on hyperthreads madness. It's not like the top 8 would be any different without ponder anyways, or maybe 1 engine out of 8.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls

AndrewGrant
Posts: 494
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: Chess.com 2018 computer chess championship

Post by AndrewGrant » Wed Sep 12, 2018 3:50 pm

CMCanavessi wrote:
Wed Sep 12, 2018 3:38 pm
Maybe the rules can be changed for the second stage and stop this ponder on hyperthreads madness. It's not like the top 8 would be any different without ponder anyways, or maybe 1 engine out of 8.
I hope so. Dropping pondering will fix things, and be completely elo neutral. Dropping hyper threads will only cause a ~10 elo loss when playing vs Leela, which I think is acceptable.

elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: Chess.com 2018 computer chess championship

Post by elcabesa » Wed Sep 12, 2018 4:00 pm

I don't understand the problem,
CCCC has defined some rules, maybe they has something to fix on server side too, but it seems to me that some chess engine shown some problem under this kind of setup.
Vajolet has some problem too, but hey, they are problem of Vajolet, I have to dig and find the bug, and I hope to have fixed them.

Don't you think that those program can have problem too with pondering and games without adjudication since not too much tournament have been done with this setup?

why always finger pointing?

elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: Chess.com 2018 computer chess championship

Post by elcabesa » Wed Sep 12, 2018 4:09 pm

From the logs provided to me ...

info depth 1 seldepth 3 score cp 44 time 6093 nodes 202659512 nps 33255000 tbhits 2067 hashfull 53 pv b4d5

6093ms for a depth 1 search? It looks to me like the main thread got locked out of CPU time, and probably some of the helpers.
it looks like it has searched 202659512 nodes... probably a bug on ponder code?

User avatar
Guenther
Posts: 3116
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Chess.com 2018 computer chess championship

Post by Guenther » Wed Sep 12, 2018 4:37 pm

AndrewGrant wrote:
Wed Sep 12, 2018 2:30 pm
https://pastebin.com/raw/FxRnwhHF

Ethereal, Senpai, Nemorino, Shredder, Nirvanachess are shown to exhibit the same behavior ...

Random depth 1 searches taking thousand of times longer than they should.

Most interesting, is that games are hit or miss. There are a handful of cases where it happens to Ethereal and Shredder, but some games are perfectly clear.

EDIT : Furthermore, none of the games posted above contained Leela. Further proof that hyperthreads and pondering was a poor idea.
There is much more to find with different parameters.
e.g. I found over 360 moves with depth 2-6 and time >10s! in the same pgn from this afternoon...
Current foe list count : [101]
http://rwbc-chess.de/chronology.htm

AndrewGrant
Posts: 494
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: Chess.com 2018 computer chess championship

Post by AndrewGrant » Wed Sep 12, 2018 5:54 pm

elcabesa wrote:
Wed Sep 12, 2018 4:09 pm
From the logs provided to me ...

info depth 1 seldepth 3 score cp 44 time 6093 nodes 202659512 nps 33255000 tbhits 2067 hashfull 53 pv b4d5

6093ms for a depth 1 search? It looks to me like the main thread got locked out of CPU time, and probably some of the helpers.
it looks like it has searched 202659512 nodes... probably a bug on ponder code?
No ... that shows that some threads got CPU time, but others + main thread did not.

In this case, 33mnps is < 50% the expected NPS for the given position.

Also, seldepth=3. Max tree size is <= 256^3. Its not a ponder bug, its a hardware issue.

elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: Chess.com 2018 computer chess championship

Post by elcabesa » Wed Sep 12, 2018 6:29 pm

In my programming experience, the problem I had with my software were rarely hardware or compiler bug, lot of time they were software bug created by myself.

AndrewGrant
Posts: 494
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: Chess.com 2018 computer chess championship

Post by AndrewGrant » Wed Sep 12, 2018 6:31 pm

elcabesa wrote:
Wed Sep 12, 2018 6:29 pm
In my programming experience, the problem I had with my software were rarely hardware or compiler bug, lot of time they were software bug created by myself.
Okay. This is not such an example. This was expected by everyone in this thread. No one listened.

Unless you think that 5 engines all display the same bug, and for some reason CCCC is the first group to discover it.

Post Reply