testing the engine using CCRL pgns and Gauntlets ?

[MahmoudUthman]

Can I measure improvements of the engine by having both the current & previous version of the engine play a Gauntlet against a subset of the engines listed in CCRL or a similar list , merging the resulting PGN with the pure PGN of the CCRL ?

and if this could work would it be better to calculate the ELO by merging the 2 tournament PGN with the entire pure list or with the entries of the selected subsets only ?

and do I have to use the same time control or could I use something like 1 min/all with CCRL(Pure 40/4) ?

[hgm]

You can measure improvement of your engine that way already without using the CCRL games, if all versions are playing the same gauntlet. The CCRL games would only be helpful if you want to have an absolute rating.

[MahmoudUthman]

You can measure improvement of your engine that way already without using the CCRL games, if all versions are playing the same gauntlet. The CCRL games would only be helpful if you want to have an absolute rating.

But I meant as a higher quality reference instead of having the testing pool play each other first in a round robin tournament ?
I may be missing something but my understanding is that I need to choose a set of engines close to my engines rating for example {A,B,C,D} , have them play a round robin to measure their relative strength , then have each version of my engine play a gauntlet against them , So if my understanding is right can I use the matches from the CCRL of the testing set even though it was on different hardware ?

[elcabesa]

I think you have to find a subset of engine in ccrl rating list that are of the same strenght of your engine.

then you do a gauntlet of your old engine against them
and then you do a gauntlet of your new development engine against them

at the end you can see whether your new engine had a better or worst result compared against the same CCRL engines

[hgm]

But I meant as a higher quality reference instead of having the testing pool play each other first in a round robin tournament ?

I don't think it would do anything for the quality of the improvement measurement. It just improves the Elo determination of the gauntlet engines. (Which you are not interested in.)