New engine releases 2019

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Gabor Szots
Posts: 186
Joined: Sat Jul 21, 2018 5:43 am
Location: Szentendre, Hungary
Full name: Gabor Szots

Re: New engine releases 2019

Post by Gabor Szots » Fri May 31, 2019 4:22 pm

jorose wrote:
Fri May 31, 2019 4:11 pm
I am assuming this is on 1 core at 4/40 time control? How is the speed compared to vs 0.5? I assume there were no losses on time? What GUI are you using?
Yes, it is on 1 core and the time control is 40/2 (adjusted to the reference machine).
The speed was abot 836 knps for 0.5 and 788 knps for 0.6.
There have been no time losses. I am using the Shredder 13 GUI.
Gabor Szots
CCRL testing group

jorose
Posts: 228
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: New engine releases 2019

Post by jorose » Fri May 31, 2019 4:38 pm

Could you test the "unofficial" 0.5.5b to see if the behavior is different? That version has a large part of the (expected) strength improvement over 0.5 but it doesn't a lot of refactoring which could have had unexpected adverse effects. Also it drew a game against Komodo in TCEC so hopefully that is a sign that version is not defect :P

One point to note is I think 0.5 single core got lucky on the 40/4 list and is probably about 50 points higher than it should be, but 30% against 2980 is roughly where I would expect Winter 0.4a to be...
-Jonathan

Gabor Szots
Posts: 186
Joined: Sat Jul 21, 2018 5:43 am
Location: Szentendre, Hungary
Full name: Gabor Szots

Re: New engine releases 2019

Post by Gabor Szots » Fri May 31, 2019 6:18 pm

jorose wrote:
Fri May 31, 2019 4:38 pm
Could you test the "unofficial" 0.5.5b to see if the behavior is different? That version has a large part of the (expected) strength improvement over 0.5 but it doesn't a lot of refactoring which could have had unexpected adverse effects. Also it drew a game against Komodo in TCEC so hopefully that is a sign that version is not defect :P

One point to note is I think 0.5 single core got lucky on the 40/4 list and is probably about 50 points higher than it should be, but 30% against 2980 is roughly where I would expect Winter 0.4a to be...
I'm going to try 0.5.5b as soon as I'll have some time. Meanwhile, 0.6 is at almost 35 % after 88 games.
Gabor Szots
CCRL testing group

Gabor Szots
Posts: 186
Joined: Sat Jul 21, 2018 5:43 am
Location: Szentendre, Hungary
Full name: Gabor Szots

Re: New engine releases 2019

Post by Gabor Szots » Fri May 31, 2019 6:21 pm

Gabor Szots wrote:
Fri May 31, 2019 6:18 pm
jorose wrote:
Fri May 31, 2019 4:38 pm
Could you test the "unofficial" 0.5.5b to see if the behavior is different? That version has a large part of the (expected) strength improvement over 0.5 but it doesn't a lot of refactoring which could have had unexpected adverse effects. Also it drew a game against Komodo in TCEC so hopefully that is a sign that version is not defect :P

One point to note is I think 0.5 single core got lucky on the 40/4 list and is probably about 50 points higher than it should be, but 30% against 2980 is roughly where I would expect Winter 0.4a to be...
I'm going to try 0.5.5b as soon as I'll have some time. Meanwhile, 0.6 is at almost 35 % after 88 games.

EDIT: I ran 0.5.5b with go depth 15 and its nps was 850.
Gabor Szots
CCRL testing group

jorose
Posts: 228
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: New engine releases 2019

Post by jorose » Fri May 31, 2019 6:59 pm

Gabor Szots wrote:
Fri May 31, 2019 6:21 pm
Gabor Szots wrote:
Fri May 31, 2019 6:18 pm
jorose wrote:
Fri May 31, 2019 4:38 pm
Could you test the "unofficial" 0.5.5b to see if the behavior is different? That version has a large part of the (expected) strength improvement over 0.5 but it doesn't a lot of refactoring which could have had unexpected adverse effects. Also it drew a game against Komodo in TCEC so hopefully that is a sign that version is not defect :P

One point to note is I think 0.5 single core got lucky on the 40/4 list and is probably about 50 points higher than it should be, but 30% against 2980 is roughly where I would expect Winter 0.4a to be...
I'm going to try 0.5.5b as soon as I'll have some time. Meanwhile, 0.6 is at almost 35 % after 88 games.

EDIT: I ran 0.5.5b with go depth 15 and its nps was 850.
Could be that something in my refactoring (which added a lot of constexprs, can be a bad thing) resulted in a bit slower code when compiled with mingw, combined with a previous overperformance on 1 core for v0.5 (which would be consistent with my expectations for that version) combined with a bad start for v0.6.

If Winter 0.5 is 2880 (10 Elo weaker than on the 40/40 list, which would be consistent with history and within the margin of error the list says for the 4/40 rating) and Winter is 50 above that, then it would be roughly 2930 and expected to score around 43% against 2980 average opposition. My estimates are usually based on lower bounds from self play, but that could easily be overestimating things still, so maybe it is closer to 2900 in which case we are within margin or error of the expected performance at this TC.
-Jonathan

konsolas
Posts: 166
Joined: Sun Jun 12, 2016 3:44 pm
Location: London
Full name: Vincent
Contact:

Re: New engine releases 2019

Post by konsolas » Fri May 31, 2019 8:33 pm

Topple 0.6.1 is a mostly bugfix update that should resolve the illegal moves that have been plaguing endgames with 5-piece tablebases:
https://github.com/konsolas/ToppleChess ... tag/v0.6.1

jorose
Posts: 228
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: New engine releases 2019

Post by jorose » Fri May 31, 2019 11:47 pm

Gabor Szots wrote:
Fri May 31, 2019 4:22 pm
jorose wrote:
Fri May 31, 2019 4:11 pm
I am assuming this is on 1 core at 4/40 time control? How is the speed compared to vs 0.5? I assume there were no losses on time? What GUI are you using?
Yes, it is on 1 core and the time control is 40/2 (adjusted to the reference machine).
The speed was abot 836 knps for 0.5 and 788 knps for 0.6.
There have been no time losses. I am using the Shredder 13 GUI.
I got home half an hour ago and started running some tests. All I can say for sure is that at least the speed unde wine is attrocious compared to the speed I get under Linux.

Winter 0.5 using WINE:
depth: 7 perft: 3195901860 time: 41335 nps: 77315218

Winter 0.6 using WINE:
depth: 7 perft: 3195901860 time: 37805 nps: 84534250

Winter 0.6 on Linux:
depth: 7 perft: 3195901860 time: 16580 nps: 192744819

For reference:
SF8 Perft 7 output on Linux:
Nodes searched : 3195901860
Nodes/second : 192849496
-Jonathan

Gabor Szots
Posts: 186
Joined: Sat Jul 21, 2018 5:43 am
Location: Szentendre, Hungary
Full name: Gabor Szots

Re: New engine releases 2019

Post by Gabor Szots » Sat Jun 01, 2019 7:34 am

FYI, here is a short test match with 40/1 time control.
Score of Winter_0.5.5b-x64 vs Winter_0.6-x64: 3 - 6 - 17 [0.442]
Elo difference: -40.3 +/- 79.4, LOS: 15.9 %, DrawRatio: 65.4 %

27 of 100 games finished.
Gabor Szots
CCRL testing group

jorose
Posts: 228
Joined: Thu Jan 22, 2015 2:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: New engine releases 2019

Post by jorose » Sat Jun 01, 2019 5:12 pm

I ran a gauntlet against previous Winter versions at roughly CCRL 4/40 TC (though in cutechess the moves and time seem to be flipped, so the TC command was 40/240) and the results against the previous releases were roughly as expected. If there is an issue I am thinking it is either compiler or architecture specific.

Rank Name Elo +/- Games Score Draws
0 Winter v0.6 130 9 3000 67.9% 49.3%
1 Winter v0.5 -60 13 1000 41.5% 62.8%
2 Winter v0.4 -143 14 1000 30.6% 52.5%
3 Winter v0.3 -198 19 1000 24.2% 32.6%

0.5.5b should be weaker than 0.6, but much closer to v0.6 than to v0.5.
-Jonathan

Gabor Szots
Posts: 186
Joined: Sat Jul 21, 2018 5:43 am
Location: Szentendre, Hungary
Full name: Gabor Szots

Re: New engine releases 2019

Post by Gabor Szots » Sat Jun 01, 2019 5:43 pm

I am continuing my Winter 0.6 gauntlet, Winter has been slowly improving its results and now stands at 37 %. Still a long way to go so we'll see.
Gabor Szots
CCRL testing group

Post Reply