What makes an engine better at long TC vs short TC?

[rbarreira]

There has been some talk lately of what engines gain more by increasing the time per game (i.e. longer time controls). I thought that some engine authors must have some insight at what kind of changes have more impact in long TC testing vs short TC.

Other than the fact that draw percentage goes up (compressing the elo range, which benefits the weaker engines), what factors are most determinant of whether an engine performs better or worse at long TC relative to other engines?

Is eval quality an important factor? Does the better eval gain more by increasing the search time, or is it the opposite? Or perhaps the eval quality does not have much impact here, for any reasonably bug-free eval.

What about branching factor? It would seem that the more aggressive pruning should help at long TCs as the tree grows exponentially with depth, making the impact of lowering the branching factor higher and higher as time increases.

Are there any other important factors in determining how well an engine scales to longer time controls?

[Ferdy]

There has been some talk lately of what engines gain more by increasing the time per game (i.e. longer time controls). I thought that some engine authors must have some insight at what kind of changes have more impact in long TC testing vs short TC.

Other than the fact that draw percentage goes up (compressing the elo range, which benefits the weaker engines), what factors are most determinant of whether an engine performs better or worse at long TC relative to other engines?

Is eval quality an important factor? Does the better eval gain more by increasing the search time, or is it the opposite? Or perhaps the eval quality does not have much impact here, for any reasonably bug-free eval.

What about branching factor? It would seem that the more aggressive pruning should help at long TCs as the tree grows exponentially with depth, making the impact of lowering the branching factor higher and higher as time increases.

Are there any other important factors in determining how well an engine scales to longer time controls?

Eval granularity is important, finer is better. More important eval features is also better - balancing these features is another story.
Intelligent self-learning pruning and reduction will be of help.
Important also is minimal bugs .

[Don]

There has been some talk lately of what engines gain more by increasing the time per game (i.e. longer time controls). I thought that some engine authors must have some insight at what kind of changes have more impact in long TC testing vs short TC.

Other than the fact that draw percentage goes up (compressing the elo range, which benefits the weaker engines), what factors are most determinant of whether an engine performs better or worse at long TC relative to other engines?

I can only give a partial answer and we are still trying to understand the answer to that question. However I think one very important factor is evaluation. Before we did serious work on Komodo's king safety we had poor scaling characteristics which immediately changed after doing some based work on king safety. King safety is pretty dynamic so it's my (current) belief that anything that is pretty dynamic can make a difference, such as passed pawn handling, mobility and things like that.

But there are also search issues. A program that is aggressive about extensions can be pretty strong at super fast time controls but that can backfire at deeper levels. I don't know of any logical reason that should be so, but it does seem to be a factor. It's almost as if shallow searching program are weak tactically and need the extension but at deep levels they don't need them as much - however that is not an explanation that is very satisfying. Extensions should help positional play as well and should scale in principle, at least it seems logical but that may not be so. On the other hand all programs need some extensions so I don't believe that turning them all off is an answer either.

There are many other search techniques that we have found to be unpredictable in their scaling characteristics. We will have convincing and indisputable results at fast levels for some great idea and then when we proceed to higher levels the results will be just as convincingly negative. This of course is not always the case but we always check because we have not been able to predict these things in advance. It could be certain types of pruning, LMR, extensions, move ordering or any other idea that seems plausible.

So if you come up with more than this, let me know! I want to know the answer to this too!

Is eval quality an important factor? Does the better eval gain more by increasing the search time, or is it the opposite? Or perhaps the eval quality does not have much impact here, for any reasonably bug-free eval.

I think eval is one of the keys. But all the top programs have awesome evaluation functions yet scale differently. I have not studied the weaker programs in any detail but I would not be surprised to see that the weaker programs scale more poorly and might even explain why there are not as strong. Perhaps a top 100 program scales well but not as well as a top 50 and so on in the general case.

If you think like a computer scientist, the ONLY thing that matters is how well a program scales because raw performance is a linear thing but how it scales is not. Of course for a practical program raw speed is important, you cannot give away 50% of the performance for no reason.

What about branching factor? It would seem that the more aggressive pruning should help at long TCs as the tree grows exponentially with depth, making the impact of lowering the branching factor higher and higher as time increases.

Are there any other important factors in determining how well an engine scales to longer time controls?

Branching factor is not important, it's what you get for it. As Komodo has improved so has the branching factor, but that is almost a coincidence, we did what worked best. The top programs all seem to have exceptional branching factors but it does not necessarily mean that is the way to go. Stockfish has a very low branching factor, one of the lowest for a program that is one of the very bests, but Komodo is in the same league but with a higher branching factor which does not hurt it.

So what matters is what you GET for each iteration vs what you pay for it in time.