Ethereal Tuning - Data Dump

Discussion of chess software programming and technical issues.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Joost Buijs
Posts: 1176
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Ethereal Tuning - Data Dump

Post by Joost Buijs » Fri Oct 16, 2020 4:05 pm

D Sceviour wrote:
Fri Oct 16, 2020 12:09 pm
I ran a test and got no useful results. That is, the tuning was unable to produce any improvement over Schooners master PST's.

Code: Select all

   1 Schooner2.25-sse                2       8    3000     50%     60%
   2 Schooner-E12_2                  2       8    3000     50%     61%
   3 Schooner1_125_end_2nd           1       8    3000     50%     60%
   4 Schooner-E12_1                 -4       8    3000     49%     59%
The test was compared against my new 1 million epd position filtered training set which I might post. The Ethereal E12 sets above had 80,000 duplicate positions.
I'm running my tuner now with the E12.52 set, it shows some values while it is running, the piece values for the opening phase keep dropping, for the endgame they keep rising, this is usually a bad sign. Maybe it doesn't matter, but I don't like it when the mg/eg values get skewed like this.

Joost Buijs
Posts: 1176
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Ethereal Tuning - Data Dump

Post by Joost Buijs » Fri Oct 16, 2020 4:23 pm

chrisw wrote:
Fri Oct 16, 2020 12:18 pm
it was marked as rofchade, not rpirofchade, so I guess the full on version, 64x. Significant was not really the hardware it was on, but the absolute dominance of the eval function, they were evaluation crushes not search ones. mobility and positional. not fireworks, which I guess figures, those small nets are not going to be highly knowledge packed for difficult king safety/king tactical stuff, but it looks like the knowledge packing/tuning for standard structural features is more than enough.
Maybe Rofchade uses NNUE, I dont' know. Many engines suddenly have NNUE, probably by just copy-pasting code from Stockfish and adding a precooked network too. Writing optimized code for NNUE and training a network with it is not something you can do in a couple of days, these things take weeks or even months when you have to implement everything from scratch without prior knowledge or experience in this field.

Joost Buijs
Posts: 1176
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Ethereal Tuning - Data Dump

Post by Joost Buijs » Sat Oct 17, 2020 9:00 am

Joost Buijs wrote:
Fri Oct 16, 2020 4:05 pm
D Sceviour wrote:
Fri Oct 16, 2020 12:09 pm
I ran a test and got no useful results. That is, the tuning was unable to produce any improvement over Schooners master PST's.

Code: Select all

   1 Schooner2.25-sse                2       8    3000     50%     60%
   2 Schooner-E12_2                  2       8    3000     50%     61%
   3 Schooner1_125_end_2nd           1       8    3000     50%     60%
   4 Schooner-E12_1                 -4       8    3000     49%     59%
The test was compared against my new 1 million epd position filtered training set which I might post. The Ethereal E12 sets above had 80,000 duplicate positions.
I'm running my tuner now with the E12.52 set, it shows some values while it is running, the piece values for the opening phase keep dropping, for the endgame they keep rising, this is usually a bad sign. Maybe it doesn't matter, but I don't like it when the mg/eg values get skewed like this.
With the E12.33 set the weights that I get look very similar to the weights I get with my own 4M training set. I wonder what the difference between 12.33 and 12.52 is. I haven't checked the Elo differences yet.

chrisw
Posts: 3831
Joined: Tue Apr 03, 2012 2:28 pm

Re: Ethereal Tuning - Data Dump

Post by chrisw » Sat Oct 17, 2020 8:21 pm

Joost Buijs wrote:
Sat Oct 17, 2020 9:00 am
Joost Buijs wrote:
Fri Oct 16, 2020 4:05 pm
D Sceviour wrote:
Fri Oct 16, 2020 12:09 pm
I ran a test and got no useful results. That is, the tuning was unable to produce any improvement over Schooners master PST's.

Code: Select all

   1 Schooner2.25-sse                2       8    3000     50%     60%
   2 Schooner-E12_2                  2       8    3000     50%     61%
   3 Schooner1_125_end_2nd           1       8    3000     50%     60%
   4 Schooner-E12_1                 -4       8    3000     49%     59%
The test was compared against my new 1 million epd position filtered training set which I might post. The Ethereal E12 sets above had 80,000 duplicate positions.
I'm running my tuner now with the E12.52 set, it shows some values while it is running, the piece values for the opening phase keep dropping, for the endgame they keep rising, this is usually a bad sign. Maybe it doesn't matter, but I don't like it when the mg/eg values get skewed like this.
With the E12.33 set the weights that I get look very similar to the weights I get with my own 4M training set. I wonder what the difference between 12.33 and 12.52 is. I haven't checked the Elo differences yet.
Well, those sets zapped my carefully tuned king attack code down to not very much. Noticeable change in play style.

jdart
Posts: 4014
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Ethereal Tuning - Data Dump

Post by jdart » Tue Oct 20, 2020 5:18 pm

I also go no improvement with the 12.52 training set over my default set. In fact it was a regression. YMMV.

--Jon

Joost Buijs
Posts: 1176
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Ethereal Tuning - Data Dump

Post by Joost Buijs » Tue Oct 20, 2020 5:40 pm

I've got very strange weights with the E12.52 set, after this I re-tuned with the E12.33 set, now the weights look very similar to the weights I get with my own 4M set. I didn't check performance with self-play yet, but I used these weights during the last blitz tourney. Besides the loss against CoronaVirusChess (due to a questionable position that arose from my opening book) it performed reasonably well.

D Sceviour
Posts: 554
Joined: Mon Jul 20, 2015 3:06 pm
Contact:

Re: Ethereal Tuning - Data Dump

Post by D Sceviour » Tue Oct 20, 2020 7:21 pm

There are no guarantees that a tuning set can help at arriving at a unique minimum for the generation of PST's. Each test might be converging on a local minimum. There may be at least two major locals. There may be a local minimum for passed pawns, and another one for mating positions or king safety values. Another local might be a combination of these, meaning both are being threatened in a position. The local minimums still have to be defined uniquely for every position using an evaluation function. The evaluator will tell us whether pawn promotions, or mating threats are more important. However, there is no unique regression line that can cover both possibilities. Can NNUE layer design fix this?

Post Reply