Time control for testing

[cdani]

Hi.
Which time controls do you use for testing?
I have trouble deciding about this, because Andscacs plays a lot better with long time controls. For example against Cheng my current version is able to obtain ~30% of points with 500+4, but only ~21% in short time controls like 50+1.
Obviously it's a pain to use 500+4 because of the long time required to have serious results.
May be I have a problem of unbalance in Andscacs, and it's not normal to have such differences.
And maybe I simply must not complicate myself and use always 50+1 and forget about those long time controls.
For example, Stockfish with all the tests are using the same time controls?

A related theme also appears often. For example I added an improvement of King safety that counts the number of attacked places around it, regardless of they are defended or not. This was an improvement of ranking for short time controls, but a regression in 500+4, so I doubt if it's a good idea to don't use 500+4.

Thanks

[phenri]

Stockfish framework use Short and Long in seconds

if short 15+0.05 pass successfully then long 60+0.05
some time they use 30+0.05 and 120+0.05

[Isaac]

I think it is meaningless to talk about a time control without specifying the hardware.
In Stockfish testing framework, they mostly test at 15+0.05 and 60+0.05 like Paul said but that translates to 25+0.05 and about 100+0.05 on my computer (Intel duo core E6300 at 1.86 Ghz).

[velmarin]

The first is see as far as your engine depth.
See if in a short time reached that depth type.
If develops its all eval.
If then you have much time for testing. two hours, 6 hours, overnight.
You can also try to change the timing (time) of the engine. (For testing).

Sometimes only a few dozen games already see that something does not work ..

[cetormenter]

For Nirvana I use a similar setup to the stockfish testing framework. I have my engine run a test version against the current master in an SPRT test. Generally I test at 6" + 0.06" but if something requires a very high depth (such as tweaks to search parameters that only trigger at high depths) then I use 15" + 0.15".

So far this has worked for me. The elo gain that I see in my tests, except for time control tweaks, correlate about 1:.8 with the ccrl's 40/40 ratings list. Using these short time controls allows me to run thousands of games while I am asleep and generally the test runs its course by the time I wake up.

I don't believe that time controls longer than 30 seconds per game are required for most engines to see elo increases across all time controls.

Good luck with your engine!

Thomas

[cdani]

Thanks for all the answers!!

So it seems that I'm using too long time controls for what it's accustomed. But, at least for Andscacs, it's clear that many changes give very different results on long controls.

I suppose that it's another time the standard idea that the optimum is enemy of the good. So it's preferable to lose some strength and be able to test a lot more features, which will give more options to advance, to test fewer features that will be better but too costly.

So for example at Stockfish they accept patches that are bad for long time controls, and they don't know, but for the long trend it's ok.

Also another thing to think about is that Stockfish goes deeper a lot faster than Andscacs, so with less time it's able to analyze a lot better.

I suppose about the improvement of King safety that counts the number of attacked places around it, its better in short time controls because it hides some other weaknesses, which are only circumvented if the engine has enough time to analyze; so in long time controls this change is unnecessary.