Time Management

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

jhellis3
Posts: 546
Joined: Sat Aug 17, 2013 12:36 am

Re: Time Management

Post by jhellis3 »

do you notice any difference between them on WHICH moves in a given phase of the game get more time?
No, I haven't noticed anything in particular that would lead me to believe one is making better choices than the other.

It is obvious Komodo doesn't have only move, but that is just aesthetics.
lkaufman
Posts: 6006
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Time Management

Post by lkaufman »

Thank you for your detailed response. In your opinion, about what percentage of Stockfish elo gains (since SFDD or since SF4, whichever you prefer as the baseline) are due to improved time management?
Uri Blass
Posts: 10460
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Time Management

Post by Uri Blass »

lkaufman wrote:Thank you for your detailed response. In your opinion, about what percentage of Stockfish elo gains (since SFDD or since SF4, whichever you prefer as the baseline) are due to improved time management?
I guess that 10-20% of the elo gains of stockfish since SFDD is thanks to better time management.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Time Management

Post by bob »

lkaufman wrote:I would like to ask the readers of this forum who have watched games between Stockfish and Komodo recently, whether on TCEC or anywhere else, to comment on which engine manages its time better, and whether the difference in this respect is large/obvious or not. I have an opinion on this, but I'd rather not influence your opinions by stating my own. Thank you in advance.
Larry:

Don and I had some LONG offline discussions about this very idea last year. I had disliked the old "easy move" approach (move quickly for obvious moves, although defining "obvious" was always tricky) and the idea of "move slow on hard moves (where we have failed low)" is really almost a case of too little, too late.

I don't know where Don ended up, but for me, I decided to use a simple idea that was not very effective back in the old days of 8 ply searches, but which works pretty well today, namely that after any iteration where the best move does not change, the time limit is reduced, down to some lower limit that I hand-tuned with testing. Each time an iteration sees a best move change, the time is incrementally ramped up to some upper bound also hand-tuned.

This has actually proven to be pretty workable. The gain was less than 10 Elo, but it was a gain. Only oddity is that Crafty is now very inconsistent in how long it takes to make a move, but it does seem to spend more time in more appropriate places...

There are other cases I handle, but all are in Crafty's source and easy to understand with the comments...
lkaufman
Posts: 6006
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Time Management

Post by lkaufman »

bob wrote:
lkaufman wrote:I would like to ask the readers of this forum who have watched games between Stockfish and Komodo recently, whether on TCEC or anywhere else, to comment on which engine manages its time better, and whether the difference in this respect is large/obvious or not. I have an opinion on this, but I'd rather not influence your opinions by stating my own. Thank you in advance.
Larry:

Don and I had some LONG offline discussions about this very idea last year. I had disliked the old "easy move" approach (move quickly for obvious moves, although defining "obvious" was always tricky) and the idea of "move slow on hard moves (where we have failed low)" is really almost a case of too little, too late.

I don't know where Don ended up, but for me, I decided to use a simple idea that was not very effective back in the old days of 8 ply searches, but which works pretty well today, namely that after any iteration where the best move does not change, the time limit is reduced, down to some lower limit that I hand-tuned with testing. Each time an iteration sees a best move change, the time is incrementally ramped up to some upper bound also hand-tuned.

This has actually proven to be pretty workable. The gain was less than 10 Elo, but it was a gain. Only oddity is that Crafty is now very inconsistent in how long it takes to make a move, but it does seem to spend more time in more appropriate places...

There are other cases I handle, but all are in Crafty's source and easy to understand with the comments...
We already have some variant of that idea in Komodo, for at least 2 years now. One question: When you speak of "the time" or "the time limit" do you mean goal time or maximum time?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Time Management

Post by bob »

lkaufman wrote:
bob wrote:
lkaufman wrote:I would like to ask the readers of this forum who have watched games between Stockfish and Komodo recently, whether on TCEC or anywhere else, to comment on which engine manages its time better, and whether the difference in this respect is large/obvious or not. I have an opinion on this, but I'd rather not influence your opinions by stating my own. Thank you in advance.
Larry:

Don and I had some LONG offline discussions about this very idea last year. I had disliked the old "easy move" approach (move quickly for obvious moves, although defining "obvious" was always tricky) and the idea of "move slow on hard moves (where we have failed low)" is really almost a case of too little, too late.

I don't know where Don ended up, but for me, I decided to use a simple idea that was not very effective back in the old days of 8 ply searches, but which works pretty well today, namely that after any iteration where the best move does not change, the time limit is reduced, down to some lower limit that I hand-tuned with testing. Each time an iteration sees a best move change, the time is incrementally ramped up to some upper bound also hand-tuned.

This has actually proven to be pretty workable. The gain was less than 10 Elo, but it was a gain. Only oddity is that Crafty is now very inconsistent in how long it takes to make a move, but it does seem to spend more time in more appropriate places...

There are other cases I handle, but all are in Crafty's source and easy to understand with the comments...
We already have some variant of that idea in Komodo, for at least 2 years now. One question: When you speak of "the time" or "the time limit" do you mean goal time or maximum time?
A simple explanation.

I set a target time about like I always have. remaining_time / constant (which can vary if ponder=on or off, or increment/no-increment, etc). This is the ultimate goal time.

I have a difficulty factor that starts at 100%. Each iteration where the best move does not change sees difficulty reduced by 10%. To some lower bound (I think 60%). A PV change (new best move) has two cases.

(1) current difficulty < 100%. Set it back to 100% and add a 20% increment to that. For each pv change from here on in this iteration, difficulty gets bumped by 20%, up to a max of 200%.

(2) current difficult >= 100%, just add 20% of current, up to 200%.

I have an absolute time limit of 6x the normal target time that can't be exceeded, ever. The 6x might be adjusted if that would take us beyond the actual remaining time on the clock, but that seems pretty obvious.

So something like this, algebraically:

time used = T:

60%target < target < 200%target < 600% target.

T fits somewhere in that range...

Once I get to target * difficulty time used, Crafty will stop when the current iteration ends, or if we are still searching the fist move at the current iteration, assuming we have not failed low...
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Time Management

Post by bob »

Uri Blass wrote:
lkaufman wrote:Thank you for your detailed response. In your opinion, about what percentage of Stockfish elo gains (since SFDD or since SF4, whichever you prefer as the baseline) are due to improved time management?
I guess that 10-20% of the elo gains of stockfish since SFDD is thanks to better time management.
I'd bet that is WAY too high.
jhellis3
Posts: 546
Joined: Sat Aug 17, 2013 12:36 am

Re: Time Management

Post by jhellis3 »

Well, the gain from SF DD is ~50 - 55 Elo. And the cumulative gains from better TM are ~5 Elo, IIRC. That is 10%.

But, yeah, writing it as 10% certainly makes it sound more impressive than "5 Elo."
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Time Management

Post by bob »

jhellis3 wrote:Well, the gain from SF DD is ~50 - 55 Elo. And the cumulative gains from better TM are ~5 Elo, IIRC. That is 10%.

But, yeah, writing it as 10% certainly makes it sound more impressive than "5 Elo."
I think 5 Elo is the UPPER bound on TM gains. When Don and I were talking, I was testing many ideas for both of us. And NONE of them varied by more than 5 Elo, unless I introduced a bug by accident.
syzygy
Posts: 5647
Joined: Tue Feb 28, 2012 11:56 pm

Re: Time Management

Post by syzygy »

bob wrote:
jhellis3 wrote:Well, the gain from SF DD is ~50 - 55 Elo. And the cumulative gains from better TM are ~5 Elo, IIRC. That is 10%.

But, yeah, writing it as 10% certainly makes it sound more impressive than "5 Elo."
I think 5 Elo is the UPPER bound on TM gains. When Don and I were talking, I was testing many ideas for both of us. And NONE of them varied by more than 5 Elo, unless I introduced a bug by accident.
The upper bound will depend on your starting point. There is certainly no lower bound on TM loss.

Stating that for an arbitrary engine there is at most 5 Elo to gain from improving TM makes little sense imho. But obviously for any engine there is a limit to what one can extract from TM improvements.