North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Leap second reminder - Check your NTP

  • From: Kevin Day
  • Date: Sat Dec 31 19:58:59 2005


Last NTP spam:


I'm by no means an NTP expert, if anyone else is, please pipe up.

About 30 minutes before the leap second should have occurred, several of our systems reported "xntpd[13742]: time reset 0.958385 s", which was really strange. They moved the wrong direction, and they did it early. Shortly after, those systems lost ntp association and began drifting. About 10 minutes after midnight all have regained sync. I wasn't checking things that early to see why, it's possible some of our NTP sources started disagreeing on what the correct time was, and would also match what other people have reported off-list, going back as far as 18 hours before midnight.

Several public NTP sources are now indicating a "leap second alarm" (setting the leap bits to 11), which will cause most NTP clients to rule them out as a source. ntp-2.gw.uiuc.edu is an example:

130.126.24.44: Server dropped: Leap not in sync
server 130.126.24.44, port 123
stratum 2, precision -19, leap 11, trust 000
refid [128.174.38.133], delay 0.03357, dispersion 0.00049

According to ntpdate, its clock seems to have stopped about 5 minutes before midnight, and hasn't yet recovered.

Other NTP servers haven't cleared their "today is a leap second day" bit, which they should have by now. Some NTP implementations rule out servers that don't agree with what their "master" server thinks the leap second bits should be. My reading of the NTP spec says that at 00:00:00 the leap bits should have been returned to zero. Attempting to sync from one of these servers will produce a "Next leap second occurs at 00:00:00.000 UTC Sun Jan 01 2006" message, but that should be harmless as long as they correct themselves a while before midnight.

Still others have their clocks off by a significant amount(10+ minutes) and think they're still in sync, but since I started typing this email, they all have corrected themselves.


While I can't say anything broke on our network as a result of the leap second, a good percentage of our gear lost NTP sync or had some kind of NTP problem around midnight UTC. You may want to check your NTP status at some point, in case something drifted quite a way off and won't step itself back now because the difference is too great.

-- Kevin