Sortable, readable and standard time format for logs

StackOverflow https://stackoverflow.com/questions/17819834

Question

Timestamp format in logs

Most log lines contain a timestamp and event description:

[When] [What]

e.g.:

[23/Jul/2013:19:35:11 +0000] Processing started.
[23/Jul/2013:19:36:11 +0000] Processed 1000 items.
[23/Jul/2013:19:37:11 +0000] Processing finished successfully.

I am trying to find a standard timestamp for my log lines. My criteria is:

  1. Human readable. I want to easily understand when did the event happen.
  2. Alphabetically sortable. When I grep events from a few files and sort them using POSIX sort or even word/excel, I want the alphabetical sort to adhere to chronological sort. For example, [23/Jul/2012:19:35:11 +0000] and [22/Jul/2013:19:35:11 +0000] are not sortable - the 2013 line would appear before the 2012 line.
  3. Easily parsable by all common languages. The timestamp should be easily parsed using standard strptime if the log is processed by a script.

The only standard I've found so far is ISO_8601, which has many variants (e.g. 2007-04-05T14:30Z and 2007-03-01T13:00:00Z), and lacks a definite standard for log line events.

Could you recommend a standard timestamp format for log lines?

Was it helpful?

Solution

enter image description here

@J.F. Sebestian - Thanks for your comment.

After some research I chose RFC 3339 / ISO 8601 in UTC, e.g.:

date -u "+[%Y-%m-%d %H:%M:%S%z (%Z)]"       # Space separated with tz abbreviation
[2013-07-31 23:56:34+0000 (UTC)]                   

date -u "+[%Y-%m-%d %H:%M:%S.%N %z (%Z)]"  # Space separated with nanoseconds and tz abbreviation
[2013-07-31 23:56:34.812572000 +0000 (UTC)]

Features:

  • Sortable (Most significant date item is on the left)
  • Readable
  • Unambiguous, time zone clearly stated
  • Delimited by [,], useful for regexing the date away
  • Easily parsable
  • Accurate: uses nanoseconds (might be practically milliseconds on some machines, which is good enough)

I've also created a nice github project that helps with date formatting - feel free to take a look and suggest your own formats!

OTHER TIPS

The unix date command has such an option. Use

date -Iseconds

or

date -Ins

The manpage says:

   -I[FMT], --iso-8601[=FMT]
          output  date/time  in  ISO  8601  format.   FMT='date' for date only (the default),
          'hours', 'minutes', 'seconds', or 'ns' for date and time to  the  indicated  preci‐
          sion.  Example: 2006-08-14T02:34:56-0600
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top