Development version testing idea

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Ferdy
Posts: 4856
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Development version testing idea

Post by Ferdy »

So you have the results of your previous public released version.
CEGT 40/20.

Image

What I plan is get the games of those matches, extract the opening, then use those opening to test your development version. Openings, opponents, TC and color allocations should be similar as in the previous released version.
Has anyone tried this before?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Development version testing idea

Post by bob »

Ferdy wrote:So you have the results of your previous public released version.
CEGT 40/20.

Image

What I plan is get the games of those matches, extract the opening, then use those opening to test your development version. Openings, opponents, TC and color allocations should be similar as in the previous released version.
Has anyone tried this before?
Not using TCEC data. But the idea of using the same set of openings to test each version is standard testing methodology. The hard part is hardware. Not all games get played on the same hardware, which makes comparing your data where they do use exactly the same hardware a bit different.
Ferdy
Posts: 4856
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Development version testing idea

Post by Ferdy »

bob wrote: Not using TCEC data. But the idea of using the same set of openings to test each version is standard testing methodology. The hard part is hardware. Not all games get played on the same hardware, which makes comparing your data where they do use exactly the same hardware a bit different.
Right, the only thing that I can do here is to get the equivalent benchmark for my hardware.