The upcoming Y2038 catastrophe

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

abulmo
Posts: 151
Joined: Thu Nov 12, 2009 6:31 pm

Re: The upcoming Y2038 catastrophe

Post by abulmo »

bob wrote:
abulmo wrote:
bob wrote:
By far, a properly configured ntpd is the most commonly used and least problematic time fix that doesn't wreck things.

And for the record, ANYONE running with root access is absolutely a security risk. I don't think you can find a single person on the planet that has not broken something while running as root. Which is why most of us with any experience do not use root as our working account.
Not everybody is running a cluster of servers. On tablets, laptops, etc., with intermittent network connections, ntpd is not working as you except. If the time deviation is big enough (1000s), ntpd is not going to correct anything, and you have to change the time manually. For smaller time deviation, ntpd can call settimeofday and produce big time jump if you are using the obsolescent settimeofday function. This is only for small time deviation that ntpd works as you describe it. On a permanently connected computer, only small deviations may occur, but in real life, not all computer are permanently connected.
OK, HOW does the time deviation reach 1000s? Only by human interference. Which means nothing will work if a human is in the middle of the process. How else does the clock get off that far?

(1) shutdown and a later reboot? an ntpdate during boot solves that, before ntpd decides to give up and quit. That's automatic on boot up.

(2) a suspend/wake on a laptop (closing the lid)? The wakeup will correct the clock if things are set up correctly. Again, no problem.

Are you going to do EITHER of those while playing a chess game where a monotonic clock is important? If you WANT to break anything, you can. But if you let ntpd run, and leave it alone, you get exactly the effect most want. Which is an accurate clock across all machines you are using so that Make does not get confused, a monotonic clock so that you don't have to deal with negative intervals, etc.
OK, an example from my real life of a computer with intermittent internet connection.
When I travel, I bring my laptop to places without wifi connection nor ethernet plugs, but I can connect it from time to time to the internet through my 4G mobile phone. Usually I let my computer working all days, doing some timed computations, and I will only connect it to the internet when I am present. If my computer stays a few days, or even weeks without internet connection it's internal clock can be off time by a few seconds. Once connected to internet, my computer real time clock can then be abruptly set to the network time, going forward or backward.
I wonder what is wrong with using clock_gettime. It 's a POSIX standard. It solves a potentially existing problem.
Richard
AlvaroBegue
Posts: 931
Joined: Tue Mar 09, 2010 3:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: The upcoming Y2038 catastrophe

Post by AlvaroBegue »

abulmo wrote: OK, an example from my real life of a computer with intermittent internet connection.
When I travel, I bring my laptop to places without wifi connection nor ethernet plugs, but I can connect it from time to time to the internet through my 4G mobile phone. Usually I let my computer working all days, doing some timed computations, and I will only connect it to the internet when I am present. If my computer stays a few days, or even weeks without internet connection it's internal clock can be off time by a few seconds. Once connected to internet, my computer real time clock can then be abruptly set to the network time, going forward or backward.
I wonder what is wrong with using clock_gettime. It 's a POSIX standard. It solves a potentially existing problem.
This is going to be good. I can't wait to see what Bob has to say about a perfectly reasonable real-world scenario where his "GUARANTEED" method actually fails miserably. :)

Perhaps one should heed the advice of the Linux man page.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: The upcoming Y2038 catastrophe

Post by bob »

AlvaroBegue wrote:
abulmo wrote: OK, an example from my real life of a computer with intermittent internet connection.
When I travel, I bring my laptop to places without wifi connection nor ethernet plugs, but I can connect it from time to time to the internet through my 4G mobile phone. Usually I let my computer working all days, doing some timed computations, and I will only connect it to the internet when I am present. If my computer stays a few days, or even weeks without internet connection it's internal clock can be off time by a few seconds. Once connected to internet, my computer real time clock can then be abruptly set to the network time, going forward or backward.
I wonder what is wrong with using clock_gettime. It 's a POSIX standard. It solves a potentially existing problem.
This is going to be good. I can't wait to see what Bob has to say about a perfectly reasonable real-world scenario where his "GUARANTEED" method actually fails miserably. :)

Perhaps one should heed the advice of the Linux man page.
OK.

(1) what is the issue? NTP is designed to keep a GROUP of machines sync'ed to the same time standard so that you don't run into timestamp problems. That is, two different machines can't modify files at the same time yet get different modification times for the files, which will break make and other things.

(2) you KNOWINGLY are inviting time sync problems by disconnecting and re-connecting. And if you recall, I specifically pointed out that an unsophisticated user, given root access or the ability to set the system time, can easily screw up the time. This falls directly under that umbrella.

If time is important, then a logical person would make every effort to provide the services needed to make it correct. Namely an internet connection. And a logical person would realize that if they disconnect from the net, it is a near-certainty that they will see some "time slip" where their time slowly falls behind the network time standard.

So this is NOT a "perfectly reasonable real-world scenario" for anyone that is interested in accurate timing. A human can break ANYTHING. But if you come to my office, you will see a machine that doesn't suffer from random timing problems such as the time suddenly jumping backward.

If you want to follow your scenario where humans can do anything they want... manually set the time.. manually break the network synchronization link... etc. Then clock_gettime() won't work for you either because you can break that JUST as easily. If you want to break it, you can do so. If you leave ntp alone, it works flawlessly, however.

BTW, you might have a small problem with clock_gettime() on freebsd-based Mac OS X. I'll leave it to others to point out just what the problem is, to avoid making this too easy for you.

I will happily remain one of those that do not intentionally try to break the system time-keeping, and continue cheerfully using gettimeofday() with absolutely no problems.

For chess, the main problem is to not have a negative elapsed time when playing games. I'd be hard-pressed to imagine how/why you would be playing a game and decide to turn the network on and off to break ntp's synchronization algorithm. If you are playing online, why would you break the net connection and lose contact with your opponent? If you are playing locally, why are you turning the network on and off?

For the record, ntp also fails if you press and hold the power button on a laptop, or if you close the lid, or if you type "reboot" as root, etc. You can break anything you want. Or you can do like most of us, run ntp, suffer ZERO problems, and use gettimeofday() which is standard across ALL Unix platforms.
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

adjtime()

Post by sje »

The man pages for adjtime() in OpenBSD and Linux are pretty much the same; both unequivocally state that the system clock is adjusted only in a monotonic fashion. This has been the case for more than twenty years. The system clock's rate may be decreased or increased, but the clock's value itself is never decreased.
This call may be used by time servers that synchronize the clocks of computers in a local area network. Such time servers would slow down the clocks of some machines and speed up the clocks of others to bring them to the average network time.
Use of the call is restricted to root (BSD), or a root designate (Linux).

The call will not fiddle with the clock if the adjustment is way out of range, something that would happen only if the clock was busted or had never been set.
AlvaroBegue
Posts: 931
Joined: Tue Mar 09, 2010 3:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: The upcoming Y2038 catastrophe

Post by AlvaroBegue »

Imagine I close my laptop's lid for a couple of days, then open it up and launch a chess engine. I am happily playing against it and then an ntpd update happens, which makes it think it has been searching for some negative amount of time. I would say that's a bug in the engine because it is using the wrong clock.

Users with root access can mess up your clock_gettime readings if they want, and people can close the lid of their laptops. The former never happens and the latter happens all the time. But if you are incapable of admitting that you simply didn't think of people closing their laptops or disconnecting their computers from the Internet... well, that's between you and your therapist.
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Here in the real world

Post by sje »

Here in the real world, most electronic timekeeping in modern consumer products is driven by crystal controlled oscillators. The crystals used have a frequency of at least 32,768 Hz and the frequency is guaranteed to be within 0.5 Hz of the true value for temperatures in the 0° to 100° range. So, a crystal controlled clock will stay correct to within two seconds per day. That's the worst case; the oscillator in my $12 Casio wristwatch is off by at most two seconds per month.

The adjtime() routine will handle adjustments up to about 2^11 (2,048) seconds, so it will do the right thing for a machine which has been unsynchronized for up to three years.
mvk
Posts: 589
Joined: Tue Jun 04, 2013 10:15 pm

Re: The upcoming Y2038 catastrophe

Post by mvk »

AlvaroBegue wrote:
abulmo wrote: OK, an example from my real life of a computer with intermittent internet connection.
When I travel, I bring my laptop to places without wifi connection nor ethernet plugs, but I can connect it from time to time to the internet through my 4G mobile phone. Usually I let my computer working all days, doing some timed computations, and I will only connect it to the internet when I am present. If my computer stays a few days, or even weeks without internet connection it's internal clock can be off time by a few seconds. Once connected to internet, my computer real time clock can then be abruptly set to the network time, going forward or backward.
I wonder what is wrong with using clock_gettime. It 's a POSIX standard. It solves a potentially existing problem.
This is going to be good. I can't wait to see what Bob has to say about a perfectly reasonable real-world scenario where his "GUARANTEED" method actually fails miserably. :)

Perhaps one should heed the advice of the Linux man page.
I have negative steps all the time on my laptop at home, just as explained above. See screenshot. And that is without travelling, and not all of these are human interference. Geographically distributed systems are not reliable. Maybe in the context of wired mainframes stuffed in an server center it works better. But who wants to go back to that? I also don't see why I would have to stop a computation when potentially losing signal. If I want to use a monotonic clock, then I should use one that guarantees me that, no strings attached. I don't want to depend on what I know (or in this case, what I think I know) of an implementation, when that property is neither mandated nor implied by the requirements. This is the strcpy story all over again. Piss-poor engineering to depend on something which is not there (or blaming the malfunction on the user, for that matter), while there is a standardised solution available.

Image
[Account deleted]
syzygy
Posts: 5563
Joined: Tue Feb 28, 2012 11:56 pm

Re: The upcoming Y2038 catastrophe

Post by syzygy »

bob wrote:(2) you KNOWINGLY are inviting time sync problems by disconnecting and re-connecting. And if you recall, I specifically pointed out that an unsophisticated user, given root access or the ability to set the system time, can easily screw up the time. This falls directly under that umbrella.
No, it was NOT YOU who pointed out that gettimeofday() can easily be screwed up. It was Richard:
http://talkchess.com/forum/viewtopic.ph ... 684#589684
abulmo wrote:gettimeofday should not be used to measure elapsed time. You should use clock_gettime(CLOCK_MONOTONIC) or an equivalent function of your OS instead. If the time of our system is adjusted (by a user or automatically), gettimeofday can return a wrong value.
bob wrote:If time is important, then a logical person would make every effort to provide the services needed to make it correct. Namely an internet connection. And a logical person would realize that if they disconnect from the net, it is a near-certainty that they will see some "time slip" where their time slowly falls behind the network time standard.
That is utter nonsense if you are talking about a Crafty user.
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Everything's positive

Post by sje »

On any properly configured system with decent hardware, the ntpd / adjtime() / gettimeofday() trio, and an occasional connection to the net, there will be no backward adjustments to the clock that a user program will see.

If by some strange reason a user program sees a backward adjustment of the clock (not the rate of the clock, a different thing), then something is wrong with the system and it needs to be fixed or tossed.
mvk
Posts: 589
Joined: Tue Jun 04, 2013 10:15 pm

Re: Everything's positive

Post by mvk »

No true Scotsman
Wikipedia wrote:No true Scotsman is an informal fallacy, an ad hoc attempt to retain an unreasoned assertion. When faced with a counterexample to a universal claim ("no Scotsman would do such a thing"), rather than denying the counterexample or rejecting the original universal claim, this fallacy modifies the subject of the assertion to exclude the specific case or others like it by rhetoric, without reference to any specific objective rule ("no true Scotsman would do such a thing").
[Account deleted]