I did a spreadsheet for all simplifications on that site. I then set June 1st ELO to zero. I then summed the ELO gain ( or loss) for all simplification from June First to December First. The net ELO loss was 24.54 ELO over a six month period. IMHO this is too much ELO loss. Six months is too much time to wait for a statistical correction. The easiest way to fix this is to slightly alter the simplification bounds on the the tests. E.g. changing the bounds from [-3.00, 1.00] to something like [-2.50, 1.50] would favor less ELO loss (or a shorter time for a correction to occur). It might make simplification slightly harder to pass but it reduces the chances of having long runs of simplifications that have a net ELO loss.Uri Blass wrote: ↑Thu Dec 12, 2019 6:09 pmThis is about the linkZenmastur wrote: ↑Thu Dec 12, 2019 12:42 pmI have no clue what all this is supposed to mean.Uri Blass wrote: ↑Thu Dec 12, 2019 2:56 am

There are not enough games to know if a simplification is a regression or an improvement but you can get an unbiased estimate for the average value of simplifications from stockfish10.

These are the first numbers and you need to get more numbers from the link and calculate average for that purpose.

At least when I look at the first numbers it seems to me that the average is positive.

209.73->207.78(-1.95 elo) 1.12.2018 simplification

208.88->206.03(-2.85 elo) 6.12.2018 simplification

208.75->211.58(2.83 elo) 16.12.2018 simplification

214.03->216.00(1.97 elo) 16.12.2018 simplification

214.25->213.08(-1.17 elo) 24.12.2018 simplification

212.66->213.98(1.32 elo) 27.12.2018 simplification

209.88->210.54(0.66 elo) 4.1.2019 simplification

211.45->215.12(3.67 elo) 10.1.2019 simplification

215.12->212.84(-2.28 elo) 14.1.2019 simplification

212.84->212.17(-0.67 elo) 14.1.2019 simplification

212.17->216.75(4.58 elo) 17.1.2019 simplification

215.25->217.07(1.82 elo) 22.1.2019 simplification

216.10->215.10(-1 elo) 29.1.2019 simplification

215.10->221.39(6.29 elo) 31.1.2019 simplification

217.75->219.64(1.89 elo) 8.2.2019 simplification

219.64->220.48(0.84 elo) 21.2.2019 simplification

220.48->220.45(-0.03 elo) 21.2.2019 simplification

218.64->219.64(1 elo) 27.2.2019 simplification

220.93->218.38->220.45(-0.48 elo) 5.3 simplifications

219.49->220.93(+1.44 elo) 10.3 simplification

219.87->218.20(-1.67 elo) 20.3 simplification

221.09->218.53(-2.56 elo) 24.3 simplification

217.85->218.81(0.96 elo) 4.4 simplification

223.36->220.86->221.82(-1.54 elo) 13.4 simplifications

219.64->219.14->218.61->219.15(-0.49 elo) 16.4 smplifications

219.15->220.30(1.15 elo) 17.4 simplification

218.51->218.61(0.1 elo) 19.4 simplification

221.37->220.81->225.70(4.33 elo) 9.5 simplifications

I will explain one line and you can understand the other lines based on the same logic

209.73->207.78(-1.95 elo) 1.12.2018 simplification

The following lines are from the link

https://nextchessmove.com/dev-builds

20181201-0929 20000 11146 433 8421 +207.78 +/- 3.64 Simplification

20181129-1517 20000 11271 478 8251 +209.73 +/- 3.69 Non Functional

209.73 is elo difference from stockfish7 before the simplification.

207.78 is elo difference from stockfish7 after the simplification.

-1.95 is the estimate for elo improvement from the simplification(note that the statistical mistake is above 3.6 elo).

1.12.2018 is the date of the simplification.

The idea is that you can get unbiased estimate for the elo that stockfish get from simplifications by the sum of all these numbers.

I did not calculate the sum of all these numbers but at least the sum of the numbers that I wrote that is only about part of the simplications is above 0.

Maybe somebody can continue to calculate the sum of all the numbers(there are many simplifications after the simplification of 9.5.2019 when I did not write the numbers but you can get it from the link).

Regards,

Zenmastur