The upcoming Y2038 catastrophe

[mvk]

Just do some research? Or just scroll up to where it was already spellt out. No need to have doubt on this point.

At this point of the "discussion" I'm expecting Sarah Palin to appear on FOX News to loudly acclaim the virtues of measuring time intervals assisted by interconnected non-realtime, package switched message transfer networks, because that is the Patriotic way and how the Founding Fathers did it. Then a brief moment of White Old Men cheering on the background, followed by street interviews in front of the Old Cowtown Museum in Kansas on the topic. Before going back to denying global warming, evolution and the President's place of birth of course.

[sje]

You are completely wrong. adjtime() does affect clock_gettime(CLOCK_MONOTONIC). clock_settime(CLOCK_MONOTONIC), on the other hand,does not. settimeofday() does change the time reported by gettimeofday(). Computers with intermittent internet connection do exist. I write now from one of them.
Just read the gettimeofday OFFICIAL manual page:
http://pubs.opengroup.org/onlinepubs/96 ... ofday.html

Applications should use the clock_gettime() function instead of the obsolescent gettimeofday() function.

.

If adjtime() does affect clock_gettime(CLOCK_MONOTONIC), then how does the latter differ from gettimeofday() other than having a different name? Both are monotonic, which seems to be the Holy Grail for which some are searching. If clock_gettime(CLOCK_MONOTONIC) is somehow immune to big jumps, well, so is gettimeofday() assuming a competent system administrator and use with programs launched after the boot completes.

As for gettimeofday() being obsolete, I'll believe it when I see the routine removed from any BSD or GNU C library. Until then, its obsolescence is just an opinion coming from POSIX 2008 and which hasn't caught on yet.

[sje]

We already knew that clock_gettime() is not present on any released version of Mac OS/X all the way up to 10.9.5, the latest.

And now after a extensive search, I can't find any reference to clock_gettime() in the unreleased OS/X 10.10 Yosemite.

There are a lot of coders writing applications for desktop OS/X and its pal, gadget iOS. If they felt that there was a need for clock_gettime(), then they'd have it.

[syzygy]

Why clock_gettime(CLOCK_MONOTONIC) is bad

The clock_gettime(CLOCK_MONOTONIC) call is bad, or at least not as good as using gettimeofday(), for measuring intervals. Why? Because on a properly configured system with decent hardware and an occasional net connection, gettimeofday() also produces monotonic output AND is not totally reliant on a single timebase as is clock_gettime(CLOCK_MONOTONIC).

The single and probably unconditioned time base used for clock_gettime(CLOCK_MONOTONIC) is just the local crystal driving the CPU by some timer interrupt. The interval measurements are no more accurate than that crystal, and that crystal could be off by a couple of seconds per day but still be within specification. Compare this to ntpd / adjtime() / gettimeofday() which use a networked ensemble of time servers plus conditioning of the local oscillator output based on historical performance.

You are completely wrong. adjtime() does affect clock_gettime(CLOCK_MONOTONIC). clock_settime(CLOCK_MONOTONIC), on the other hand,does not. settimeofday() does change the time reported by gettimeofday().

And, to help Steven's understanding a little bit, ntpd typically uses settimeofday() to correct for 125ms or more.

[syzygy]

If adjtime() does affect clock_gettime(CLOCK_MONOTONIC), then how does the latter differ from gettimeofday() other than having a different name? Both are monotonic, which seems to be the Holy Grail for which some are searching. If clock_gettime(CLOCK_MONOTONIC) is somehow immune to big jumps, well, so is gettimeofday() assuming a competent system administrator and use with programs launched after the boot completes.

settimeofday() makes gettimeofday() jump.

ntpd typically uses settimeofday() to correct for 125ms or more. This can easily happen on occasion, especially if the internet connection is not reliable.

Do you understand now?

If you're in disbelief, read this:
http://talkchess.com/forum/viewtopic.ph ... 820#589820

[Rein Halbersma]

As for gettimeofday() being obsolete, I'll believe it when I see the routine removed from any BSD or GNU C library. Until then, its obsolescence is just an opinion coming from POSIX 2008 and which hasn't caught on yet.

In standard setting lingo, "obsolete" means "does not work as previously advertised, use of an alternative is recommended". It is a precursor to "deprecated", which means "use at own risk, functionality may be removed in a future version of the standard". The final stage is "removed". In any case, "obsolete" is most assuredly not just an opinion but a technical matter of fact. The long purgatory stage (typically a decade or more) is to give sinners the chance to see the light, before sending them to hell.

[bob]

If adjtime() does affect clock_gettime(CLOCK_MONOTONIC), then how does the latter differ from gettimeofday() other than having a different name? Both are monotonic, which seems to be the Holy Grail for which some are searching. If clock_gettime(CLOCK_MONOTONIC) is somehow immune to big jumps, well, so is gettimeofday() assuming a competent system administrator and use with programs launched after the boot completes.

settimeofday() makes gettimeofday() jump.

ntpd typically uses settimeofday() to correct for 125ms or more. This can easily happen on occasion, especially if the internet connection is not reliable.

Do you understand now?

If you're in disbelief, read this:
http://talkchess.com/forum/viewtopic.ph ... 820#589820

First, let's get correct numbers. This is directly from the current production ntpd source code:

* Important note: The kernel discipline is used only if the
* step threshold is less than 0.5 s, as anything higher can
* lead to overflow problems. This might occur if some misguided
* lad set the step threshold to something ridiculous.

So point 1. NTPD uses the adjtime() if the correction is no more than 0.5 seconds. 1/2 a second. The flavors of unix that work like this include Linux, Solaris, freeBSD, Digital unix, HPUX, etc.

Point 2. In order to get that far "off" what did one do? Closing the lid on the laptop? The normal unsuspend code procedure is to use ntpdate to update the system clock immediately on un-suspending. What else could it do? But this is done immediately. Clearly a lot of time has expired, so the clock needs correction. In the FORWARD direction ONLY. Unless you visit a time-warp somewhere.

gettimeofday() works as it should. ntpd works as it should. So long as someone doesn't intentionally try to break it. Once ntpd syncs the system time when it is first set up, clock drift is trained to be minimal unless there is interference. Killing the network connection will NOT cause the clock to drift seconds or minutes. The only thing that causes that kind of drift is a suspend/sleep/hibernate/etc and ntpd fixes that as soon as the system recovers.

So what on earth are you continuously whining about here? I did my best to break this yesterday, everything except manually changing the time backward, and absolutely nothing caused it to become non-monotonic. It takes a LARGE backward jump in time, only possible when a super-user changes the time, to break monotonicity. As I have repeatedly said.

One CAN look at ntpd source. I hacked mine running on a linux box, to display any case where the clock was changed in a backward direction. Could not make it happen. Did not happen naturally. ntpd typically logs what it is doing in whatever log file your specific system uses. Older linux: /var/log/message. Newer Linux: /var/log/syslog, osx: /var/log/system.log. Etc.

One CAN actually investigate this with source, and actually run it. gettimeofday() is monotonic if there is no interference. If there IS interference, no clock is monotonic. Which is what I have been saying from the get-go.

Once ntpd is started, and allowed to remain running UNMOLESTED, time works, period. Always monotonic, always with accuracy within a few ms, always with TINY corrections that are applied not by just slamming a value into the system time, but by slowly skewing the time back to the right value with adjtime().

This stuff about suspend/resume is pure nonsense. Only an idiot would close the lid on a laptop and then NOT expect a large forward jump in time when the lid is re-opened. Only an idiot would start a chess game, close the lid, open the lid later, and NOT expect major trouble in the chess engine since the program would think that either the opponent or it had used hours on the last move, which will break any time control.

Can we stop this now? It DOES work. Monotonically. Accurately.

[syzygy]

* Important note: The kernel discipline is used only if the
* step threshold is less than 0.5 s, as anything higher can
* lead to overflow problems. This might occur if some misguided
* lad set the step threshold to something ridiculous.

Funny, the author put in a warning just for you.

clock_gettime() with CLOCK_MONOTONIC guarantees that any discontinuities due to invocations of settimeofday() are ignored. It does exactly what you would want (unless you prefer to ignore adjtime() invocations as well in which case you need CLOCK_MONOTONIC_RAW).

[bob]

If adjtime() does affect clock_gettime(CLOCK_MONOTONIC), then how does the latter differ from gettimeofday() other than having a different name? Both are monotonic, which seems to be the Holy Grail for which some are searching. If clock_gettime(CLOCK_MONOTONIC) is somehow immune to big jumps, well, so is gettimeofday() assuming a competent system administrator and use with programs launched after the boot completes.

settimeofday() makes gettimeofday() jump.

ntpd typically uses settimeofday() to correct for 125ms or more. This can easily happen on occasion, especially if the internet connection is not reliable.

Do you understand now?

If you're in disbelief, read this:
http://talkchess.com/forum/viewtopic.ph ... 820#589820

First, let's get correct numbers. This is directly from the current production ntpd source code:

* Important note: The kernel discipline is used only if the
* step threshold is less than 0.5 s, as anything higher can
* lead to overflow problems. This might occur if some misguided
* lad set the step threshold to something ridiculous.

So point 1. NTPD uses the adjtime() if the correction is no more than 0.5 seconds. 1/2 a second. The flavors of unix that work like this include Linux, Solaris, freeBSD, Digital unix, HPUX, etc.

Point 2. In order to get that far "off" what did one do? Closing the lid on the laptop? The normal unsuspend code procedure is to use ntpdate to update the system clock immediately on un-suspending. What else could it do? But this is done immediately. Clearly a lot of time has expired, so the clock needs correction. In the FORWARD direction ONLY. Unless you visit a time-warp somewhere.

gettimeofday() works as it should. ntpd works as it should. So long as someone doesn't intentionally try to break it. Once ntpd syncs the system time when it is first set up, clock drift is trained to be minimal unless there is interference. Killing the network connection will NOT cause the clock to drift seconds or minutes. The only thing that causes that kind of drift is a suspend/sleep/hibernate/etc and ntpd fixes that as soon as the system recovers.

So what on earth are you continuously whining about here? I did my best to break this yesterday, everything except manually changing the time backward, and absolutely nothing caused it to become non-monotonic. It takes a LARGE backward jump in time, only possible when a super-user changes the time, to break monotonicity. As I have repeatedly said.

One CAN look at ntpd source. I hacked mine running on a linux box, to display any case where the clock was changed in a backward direction. Could not make it happen. Did not happen naturally. ntpd typically logs what it is doing in whatever log file your specific system uses. Older linux: /var/log/message. Newer Linux: /var/log/syslog, osx: /var/log/system.log. Etc.

One CAN actually investigate this with source, and actually run it. gettimeofday() is monotonic if there is no interference. If there IS interference, no clock is monotonic. Which is what I have been saying from the get-go.

Once ntpd is started, and allowed to remain running UNMOLESTED, time works, period. Always monotonic, always with accuracy within a few ms, always with TINY corrections that are applied not by just slamming a value into the system time, but by slowly skewing the time back to the right value with adjtime().

This stuff about suspend/resume is pure nonsense. Only an idiot would close the lid on a laptop and then NOT expect a large forward jump in time when the lid is re-opened. Only an idiot would start a chess game, close the lid, open the lid later, and NOT expect major trouble in the chess engine since the program would think that either the opponent or it had used hours on the last move, which will break any time control.

You keep writing something like "if the network connection is lost then..." If the network connection is lost, NTP keeps right on running. It has already trained the current clock to remain VERY close. That keeps right on working. Disconnect the network cable and you might see clock drift of a few ms. I disabled ntpd on my office linux box last night using a firewall trick to just block ntpd packets to the local time server. As of right now, after almost 12 hours of running with no contact with the timeserver, I started the little sampler program I ran yesterday. I grabbed the time (gettimeofday()) as fast as I could and stuck the values in a huge array. I prepared to enable the firewall, and started the sampler and instantly enabled the firewall and kicked ntpd with a restart. I could find ZERO jump in time after 12 hours.

The output looked something like this starting before I opened the firewall, and continuing after:

sec=1412435272 usec=721213
sec=1412435272 usec=721214
sec=1412435272 usec=721216
sec=1412435272 usec=721222
sec=1412435272 usec=721224
sec=1412435272 usec=721226
sec=1412435272 usec=721228
sec=1412435272 usec=721230
sec=1412435272 usec=722626
sec=1412435272 usec=722632

I don't trust the rightmost 3 digits of used. But the leftmost 3 are ms, and they behaved exactly as expected (that 721 to 722 was NOT at the point of firewall enable. Clocks do NOT drift significantly while the network is down, once NTPD has had time to determine how much the clock slips and get the adjustment programmed in.

Can we stop this now? It DOES work. Monotonically. Accurately.

[bob]

Why clock_gettime(CLOCK_MONOTONIC) is bad

The clock_gettime(CLOCK_MONOTONIC) call is bad, or at least not as good as using gettimeofday(), for measuring intervals. Why? Because on a properly configured system with decent hardware and an occasional net connection, gettimeofday() also produces monotonic output AND is not totally reliant on a single timebase as is clock_gettime(CLOCK_MONOTONIC).

The single and probably unconditioned time base used for clock_gettime(CLOCK_MONOTONIC) is just the local crystal driving the CPU by some timer interrupt. The interval measurements are no more accurate than that crystal, and that crystal could be off by a couple of seconds per day but still be within specification. Compare this to ntpd / adjtime() / gettimeofday() which use a networked ensemble of time servers plus conditioning of the local oscillator output based on historical performance.

You are completely wrong. adjtime() does affect clock_gettime(CLOCK_MONOTONIC). clock_settime(CLOCK_MONOTONIC), on the other hand,does not. settimeofday() does change the time reported by gettimeofday().

And, to help Steven's understanding a little bit, ntpd typically uses settimeofday() to correct for 125ms or more.

Which never happens once it has been running for a bit.

(and ntpd source says 500ms or more in the comments in one place, 125ms elsewhere.)