How to implement format-complying logging system?

Question

We have a system that generates lots of logs and we have to somehow maintain logging workflow in a project.

The 'strict' (let's assume it's strict) requirement is that there should be a document describing format for each message and each message should comply to that format.

(By format I mean BNF-like things like 'some text {one|two} [three]' etc.

It's also desirable to see log messages themselves in the code, not some IDs referring to them in some 'message list'.

Currently the state is that there is a large file with 1000+ messages with IDs, severities and user actions described for each one, and in the main code we are just struggling to keep log messages exactly in the format we specified (and of course failing in that).

It isn't obviously a viable option to like parse the source code on build and try to ensure message format that way (messages can be written not literally etc.).

So, how to implement such a format-complying logging system?

I know it sounds a bit like a 'advise me some tool'-kind-of-question, but I still think it's more about conceptual approaches to the problem (yet still I'd be really glad to see example of such systems! :) language doesn't matter).

Answer 1

Your first and most important step is to develop an API. Create a library which when called will log the message. Yes, that's all for step one. The reason for this is to coordinate logging into a collective place for future development to occur. If the caller is providing his or her own format, offer a second method to call which is deprecated from the start, but adds no formatting.

Once you've done this step (and if you have many projects, it is no small step), then you can load a configuration and use it to determine the proper format of logging entries. I would highly recommend you have a means to hide logging levels so you can filter out messages according to severity. Better still if you can filter out specific namespaces or packages.

Each project/caller then removes his or her format from the message and calls the standard logger methods which automatically adds the desired format.

Ideally your logging format should be entirely configurable for future usage. I don't know the language you're using (you didn't specify), but a number of libraries already exist which do this functionality, for example Log4J in Java. If such a library exists, you'd probably be better off using that. However it is still recommended to create your own API so that future changes don't have to be performed across the board but only in one place.