To Larry Kaufman

[cdani]

What is stc and ltc for you? How can you ever test anything at a level slower than blitz (say 3' + 2")? For Stockfish, ltc is still bullet chess.

Of course I don't have resources for testing at really long time control. Always I'm talking about long time control relative to short time control, not absolute short or absolute long. I supposed this was clear

My current stc is 15'' + .03, and my ltc 80'' + .05, and I must wait a lot of hours to have something significative with this.

My final guide is the same as yours, CCRL, CEGT, ... and the differences between versions and different time controls.

[lkaufman]

What is stc and ltc for you? How can you ever test anything at a level slower than blitz (say 3' + 2")? For Stockfish, ltc is still bullet chess.

Of course I don't have resources for testing at really long time control. Always I'm talking about long time control relative to short time control, not absolute short or absolute long. I supposed this was clear

My current stc is 15'' + .03, and my ltc 80'' + .05, and I must wait a lot of hours to have something significative with this.

My final guide is the same as yours, CCRL, CEGT, ... and the differences between versions and different time controls.

So we are all optimizing for something in the ballbark of game in a minute, so this doesn't seem to be a factor here. I'm quite surprised that you get a fifty elo swing between those two time controls relative to other engines by making eval changes. I've never observed anything close to that. What categories of eval seem to be most dependent on the time control?

[cdani]

So we are all optimizing for something in the ballbark of game in a minute, so this doesn't seem to be a factor here. I'm quite surprised that you get a fifty elo swing between those two time controls relative to other engines by making eval changes. I've never observed anything close to that. What categories of eval seem to be most dependent on the time control?

King safety and passed pawns the most. I saw also in space, I remember some good knight vs bishop things, and I should find others in the list.

One of the latest I found was for king safety, an extra big penalization for not having any pawn in front, that was like -5 elo at stc, and +5 at ltc. Was not necessary to do a lot of games, as it was so clearly superior...

[peter]

Thanks, Mark

[lkaufman]

So we are all optimizing for something in the ballbark of game in a minute, so this doesn't seem to be a factor here. I'm quite surprised that you get a fifty elo swing between those two time controls relative to other engines by making eval changes. I've never observed anything close to that. What categories of eval seem to be most dependent on the time control?

King safety and passed pawns the most. I saw also in space, I remember some good knight vs bishop things, and I should find others in the list.

One of the latest I found was for king safety, an extra big penalization for not having any pawn in front, that was like -5 elo at stc, and +5 at ltc. Was not necessary to do a lot of games, as it was so clearly superior...

Can you make a general statement as to whether longer time controls favored higher king safety and passed scores or lower ones? The one example you give suggests that longer tc favors higher king safety weights.

[cdani]

Can you make a general statement as to whether longer time controls favored higher king safety and passed scores or lower ones? The one example you give suggests that longer tc favors higher king safety weights.

For king safety seems more clear that ltc favoring higher king safety. For passed pawns I found some contradictory results, but maybe because where bad tuned before.

[Laskos]

Can you make a general statement as to whether longer time controls favored higher king safety and passed scores or lower ones? The one example you give suggests that longer tc favors higher king safety weights.

For king safety seems more clear that ltc favoring higher king safety. For passed pawns I found some contradictory results, but maybe because where bad tuned before.

It can be tentatively shown in an hour with Komodo. With LittleBlitzer it's not hard to infer the scaling using the same engine tweaked only for King Safety (KS). Komodo has such a parameter, and the assumption would be that the scaling from ultra-ultra-short to ultra-short time controls will show up at longer time controls too. Many engines will support a fixed time per move control with LitlleBlitzer even given time of 1 ms per move, on my Windows the time used by Komodo 9.3 is in reality some 3 ms per move, minimal time per move of Komodo on my PC. Komodo almost never loses on time with this short time control. So, I compared the following: what KS Komodo prefers at 1 ms per move to what Komodo prefers at 30 ms per move.

Komodo preferred KS=30 over KS=50 at 1 ms per move with a conclusive result:

Code: Select all

Games Completed = 10000 of 10000 (Avg game length = 1.422 sec)
Settings = Gauntlet/32MB/1ms per move/M 900cp for 5 moves, D 150 moves/EPD:2moves_v1.epd(32000)
Time = 2509 sec elapsed, 0 sec remaining
 1.  Komodo 9.3 KS=30         	5122.5/10000	4315-4070-1615  	(L: m=4045 t=0 i=0 a=25)	(D: r=709 i=237 f=523 s=8 a=138)	(tpm=10.9 d=6.03 nps=1746716)
 2.  Komodo 9.3 KS=50         	4877.5/10000	4070-4315-1615  	(L: m=4282 t=0 i=0 a=33)	(D: r=709 i=237 f=523 s=8 a=138)	(tpm=10.9 d=6.03 nps=1754385)

At 30 ms per move Komodo preferred KS=50 with a conclusive result:

Code: Select all

Games Completed = 10000 of 10000 (Avg game length = 4.854 sec)
Settings = Gauntlet/32MB/30ms per move/M 900cp for 5 moves, D 150 moves/EPD:2moves_v1.epd(32000)
Time = 7455 sec elapsed, 0 sec remaining
 1.  Komodo 9.3 KS=30         	4807.0/10000	3149-3535-3316  	(L: m=682 t=0 i=0 a=2853)	(D: r=1515 i=341 f=1092 s=21 a=347)	(tpm=33.6 d=12.44 nps=1516911)
 2.  Komodo 9.3 KS=50         	5193.0/10000	3535-3149-3316  	(L: m=477 t=0 i=0 a=2672)	(D: r=1515 i=341 f=1092 s=21 a=347)	(tpm=33.6 d=12.47 nps=1520083)

And the ELO gain from KS=30 to KS=50 for longer 30 ms per move is large, 14 ELO points. If extrapolating, the estimation is for blitz KS to be optimal at 60-70 and for long time control (say more than 60 min per game on one i7 core) about 80. The gain over default (65) at LTC might be in the range of 5-10 ELO points. These are something like educated guesses.

[Jesse Gersenson]

1ms per move on a windows must be an invalid test.

[S.Taylor]

I would still like to remind everyone that I still feel it would be a great shame if even one potential elo point would be taken down from long time controls, to make the blitz or bullet better.

[Jesse Gersenson]

Code: Select all

 (Avg game length = 1.422 sec)
Settings = Gauntlet/32MB/1ms per move/M 900cp for 5 moves, D 150 moves/EPD:2moves_v1.epd(32000)
Time = 2509 sec elapsed, 0 sec remaining
 1.  Komodo 9.3 KS=30         	5122.5/10000	4315-4070-1615  	(L: m=4045 t=0 i=0 a=25)	(D: r=709 i=237 f=523 s=8 a=138)	(tpm=10.9 d=6.03 nps=1746716)

Where are you getting 3ms? It says "Avg game length = 1.422 sec". Are the games averaging 237 moves per game? Assuming 120 ply per game it's be 11.85ms per move.

I am not familiar with engine testing but it seems an engine's using 10x it's alloted time is an invalid test.

Is there a guideline for the minimum time per move, on one core, for testing?