Python: Warnings and logging verbose limit

Question

I want to unify the whole logging facility of my app. Any warning is raise an exception, next I catch it and pass it to the logger. But the question: Is there in logging any mute facility? Sometimes logger becomes too verbose. Sometimes for the reason of too noisy warnings, is there are any verbose limit in warnings?

http://docs.python.org/library/logging.html

http://docs.python.org/library/warnings.html

Answer 1

Not only are there log levels, but there is a really flexible way of configuring them. If you are using named logger objects (e.g., logger = logging.getLogger(...)) then you can configure them appropriately. That will let you configure verbosity on a subsystem-by-subsystem basis where a subsystem is defined by the logging hierarchy.

The other option is to use logging.Filter and Warning filters to limit the output. I haven't used this method before but it looks like it might be a better fit for your needs.

Give PEP-282 a read for a good prose description of the Python logging package. I think that it describes the functionality much better than the module documentation does.

Edit after Clarification

You might be able to handle the logging portion of this using a custom class based on logging.Logger and registered with logging.setLoggerClass(). It really sounds like you want something similar to syslog's "Last message repeated 9 times". Unfortunately I don't know of an implementation of this anywhere. You might want to see if twisted.python.log supports this functionality.

Answer 2

from the very source you mentioned. there are the log-levels, use the wisely ;-)

LEVELS = {'debug': logging.DEBUG,
          'info': logging.INFO,
          'warning': logging.WARNING,
          'error': logging.ERROR,
          'critical': logging.CRITICAL}

Answer 3

This will be a problem if you plan to make all logging calls from some blind error handler that doesn't know anything about the code that raised the error, which is what your question sounds like. How will you decide which logging calls get made and which don't?

The more standard practice is to use such blocks to recover if possible, and log an error (really, if it is an error that you weren't specifically prepared for, you want to know about it; use a high level). But don't rely on these blocks for all your state/debug information. Better to sprinkle your code with logging calls before it gets to the error-handler. That way, you can observe useful run-time information about a system when it is NOT failing and you can make logging calls of different severity. For example:

import logging
from traceback import format_exc
logger = logging.getLogger() # Gives the root logger.  Change this for better organization
# Add your appenders or what have you
def handle_error(e):
    logger.error("Unexpected error found")
    logger.warn(format_exc()) #put the traceback in the log at lower level
    ... #Your recovery code
def do_stuff():
    logger.info("Program started")
    ... #Your main code
    logger.info("Stuff done")
if __name__ == "__main__":
    try:
        do_stuff()
    except Exception,e:
        handle_error(e)