Unless you introduce yet another technique called "aspirations windows", the answer is yes, Reset them to the full window.
With aspiration windows you set them to some margin around the last value (alpha = last score - margin, beta = last score + margin) but must be prepared that if the returned score is not inside your assumed window you have to widen the window and do a research.
#define valWINDOW 50
alpha = -INFINITY;
beta = INFINITY;
for (depth = 1;;) {
val = AlphaBeta(depth, alpha, beta);
if (TimedOut())
break;
if ((val <= alpha) || (val >= beta)) {
alpha = -INFINITY; // We fell outside the window, so try again with a
beta = INFINITY; // full-width window (and the same depth).
continue;
}
alpha = val - valWINDOW; // Set up the window for the next iteration.
beta = val + valWINDOW;
depth++;
}
This allows you to get cut-offs quicker than with an open window, where beta-cutoffs are impossible.