A question about "win at chess (new)" test suite

[peter]

How are the numbers corresponding to different moves obtained? Is it by normalizing the evaluation of the top move to 100 and adjusting the evaluations of other moves in proportion?

You can decide for your own test in your own way, mine is the human chess- understanding of how much more one move does for the side to move compared to the alternative move, is one of the candidates or more than one to be called a real game changer, and how "hard" is the position hardware- time- wise for the engines to find the one or the other one solution. And of course the basis of such evaluation is interactive analysis with engines and their output- evals, mainly of SF dev. but not only with this one, in cases of blind spots of SF's I use branches like Crystal and others too, now and then Lc0...
Those comparisons of evaluation give to me the relation between the points to be earned by position and move, the calibration in numeric height has to consider, how many positions of which kind are in which one suite,and which hardware- TC is planned for which engine- pool to test with.

Ferdinand Mosca's way was, when I watched at last, this one:
https://github.com/fsmosca/STS-Rating

[chesskobra]

I have been experimenting with sts-rating script, and I like the idea. Do you know how to pass engine parameters (e.g., the weights file in case of lc0)?

[peter]

I have been experimenting with sts-rating script, and I like the idea. Do you know how to pass engine parameters (e.g., the weights file in case of lc0)?

E.g.
--eoption "NNCacheSize=20000"
Most of the UCI- commands work like that (just like in .uci- files of most of the GUIs) with most of the engines, I tried so far.
Mind the spaces and notice, that Lc0 takes only about half of the time defined in
set MT=xmsec
, so, if you want Lc0 to use 1000msec, set 2000, on the other hand, e.g. ShashChess needs much less to be set to use same as normally,
for TCs <1000msec, better not let ShashChess run with such VSTCs, from 1000msec upwards it starts working with about correct usage of time.
Pity, you have to check each and every single engine- version (or net as for Lc0) with each and every single hardware- TC and suite newly, especially Lc0 and Shashchess differ much in real time- consumption from suite to suite and TC to TC.
To see this one screen at the end of console- output

Code: Select all

Time allocation  : GOOD!!
at <= et + mt and at >= et - mt
ExpectedTime     : 512.0s
ActualTime       : 531.2s
MarginTime/pos   : 0.2s
MarginTime       : 102.4s

without having to write log- file, edit
Pause
at the end of batch- file, regards