Time control envelope in top engines could be improved?

[Laskos]

From the balanced opening suite 3moves_GM.epd I played the latest SF at 500ms/move versus SF at 250ms/move in 4000 games without adjudication:

Code: Select all

Score of SF500 vs SF250: 1748 - 216 - 2036  [0.692] 4000
ELO difference: 140.21 +/- 7.40
Finished match

The output PGN was the main database from which I started to play another series of matches, using Cutechess command "plies" (the depth to which is must go in the PGN as starting positions). Each match was now "sequential" with 4000 games and SF self-play at 250ms/move time control. "Plies" was {1,21,41,61,81,101,121,161,221}. The outcomes of these matches gives the Elo gain got by segments of 10 moves in the move intervals (starting with 3-mover opening suite): 3-13, 13-23, 23-33,33-43,..., 83-113. The last two segments are 20 and 30 movers, because Elo gains in these endgames are small.

The importance of these 10 moves segments in the overall 140 Elo points gain of SF 500ms/move versus SF 250ms/move is the following:



Spoiler: the database used is SF at 500ms/move versus 250ms/move, very short time control. The short time control used might affect somehow the shape of Elo gains and importance of these 10-movers. At LTC it might look a bit differently. Aside from this, the remarks would be:


1/ The opening phase of the game contributes with no more than 25% of the total. People often hype the importance of the opening phase of the game, while the bulk, about 65% of outcomes, is determined in the middlegame.




2/ Elo gain is logarithmic with used time: Elo ~ ln (t2/t1). In our case total Elo gain is about 140 Elo points, and the formula is Elo gain = 202 * ln(t2/t1), with t2 =500ms, t1=250ms. Therefore necessary Time Usage is proportional to exp(Elo gain / 202), and this is valid on 10-moves intervals too. As Elo gains per intervals is much smaller than 202 Elo points, the shape of the "ideal" Time Usage is very close to the plot above of Elo gains. This shape is the shape of the "ideal time usage", as an envelope of time usage (disregarding particularities of each move). Some three years ago Ferdinand Mosca computed an averaged over many games time usage envelope of SF at 4 different time controls: 1'+ 1'', 2s/move, 2' per game, and 40/1':

• a/ The "ideal" time usage is closest to 2' per game time control. If small increment is used to not lose on time, then it should be much smaller that 1/60 of the main time used by Ferdinand, closer to 1/600 or even 1/1000
  
  
  b/ The shape of time usage of "total time per game" (closest to ideal) seems to not be ideal too. It overstates the opening phase of game, while more time should be used in the middlegame. I often see engines using by move 25-30 about half of the time per move used in the opening, while it seems it should be fairly stable from move 1 to move 30.

[Laskos]

I checked at 20s / game the time control envelope of top 3 engines, they look similar, and a bit different from the "ideal" one as derived by me from fixed time per move games and Elo gains:



But there is a snag in procedure. I equaled the total time used at different kinds of time controls with Stockfish engine: 18.84 seconds per game time control and 10 seconds + 0.1 seconds increment time control (increment of 1/100 of the base time). The total time used in games is almost identical in both cases. The time control usage envelope looks like that:



Visually, the 18.84 seconds per game envelope looks much closer to the "ideal" envelope. So, for the same used total time in games, it should be stronger than 10 seconds + 0.1 seconds increment engine (the same Stockfish engine in both cases). But the opposite happens:

Code: Select all

 
Score of SF 18.84/game vs SF 10+0.1: 364 - 532 - 1504  [0.465] 2400
ELO difference: -24.36 +/- 8.47

The total time used in games is within 0.5% difference in the two cases, it has almost no effect on Elo difference.

So, the increment seems to be beneficial, contrary to the derived "ideal" shape. Probably, the games used at fixed time per move show very little Elo gain beyond move 60, but playing with no increment shows more disadvantages later in the game than it shows int the "ideal" shape. What is the "ideal" value of increment is hard to guess.

Maybe the conclusion that middlegame should receive more attention still stands, though.

[Ozymandias]

If I'm reading this correctly, doubling time only harvests about 7 Elo points in the endgame (past move 53). Did SF have access to tablebases?

[Laskos]

If I'm reading this correctly, doubling time only harvests about 7 Elo points in the endgame (past move 53). Did SF have access to tablebases?

Yes, 7-8. No tablebases, no adjudications.

[Ozymandias]

Thank you, that would've probably been even less with TBs, but barely measurable.

[Laskos]

The derived "ideal time usage" is somehow vindicated now, it seems that only no increment at all, X time / game is problematic. I managed to equal the total average time used in games for 3 different time controls within 0.5%, so it doesn't affect by more than 1 Elo point the result. The total time used was on average 17.3 seconds per game in three cases:

X base + X/20 increment
X base + X/100 increment
X base + X/500 increment

The engine used was Stockfish 9.

Here is the result in a Round-Robin of total 3000 games:

Code: Select all

Rank Name                                 ELO     +/-   Games   Score   Draws
   1 SF 1/500 increment                    19      10    2000     53%     61%
   2 SF 1/100 increment                     6       9    2000     51%     62%
   3 SF 1/20 increment                    -25      10    2000     46%     59%
Finished match

The shapes of the envelopes (averaged) time controls used are the following:



So, closer to "ideal shape", stronger SF9 plays for different sorts of increment (for equal total time used per game). On the other hand, no increment at all harms, it came at 24 Elo points weaker than using 1/100 increment (also using equal total time per game).

Conclusions would be: "ideal time usage" seems to have some ground. Closer to ideal (smaller increment) gives better results. But with no increment at all (even closer to "ideal") the performance is bad. Maybe the time usage heuristic can be improved for X time/game case (no increment). Also, more time allocation for the middlegame compared to the opening can be worth trying.