Best practice for python logging in module

Overview

When developing a python module/library, it is a good practice to use logging to log the information, warning, and error messages. This document provides a best practice for logging in a python module/library.

And there are some concepts for this practice:

1. You want to use logging in your module/library.
2. You don't want users of your module/library to see log messages from your module/library without their explicit configuration.

Usage

Consider there is a example package named parent_package which contains a module named sub_module. The structure of the package is as follows:

--- parent_package
    |--- __init__.py
    |--- parent_module.py
    |--- logger.py
    |--- sub_package
        |--- __init__.py
        |--- sub_module.py

1. Create a logger object in your module/library.
   Consider using the following code snippet to create a logger object in your module/library. It would get a logger named after the full module name. This makes you are able to use the logger in any module easily without naming it manually.

   # logger.py
   import inspect
   import logging
   from logging import Logger
   
   
   def get_logger(name: None) -> Logger:
       """Get logger for the calling module.
   
       Returns:
           Logger: Logger object for the calling module. For example, if this function is called from module `parent_package/sub_package/sub_module.py`, the logger will be named `parent_package.sub_package.sub_module`.
       """
       if name is None:
           return logging.getLogger(inspect.currentframe().f_back.f_globals["__name__"])
       else:
           return logging.getLogger(name)
   

2. Add a NullHandler to your loggers.

   # parent_package/__init__.py
   import logger
   
   _logger = logger.get_logger()   # this logger will be named `parent_package`
   _logger.addHandler(logging.NullHandler())
   
   def get_logger():
       return _logger
   

3. Add handlers to the module logger outside of the module.

   The example file structure is as follows:

   --- main.py
   --- parent_package
       |--- __init__.py
       |--- parent_module.py
       |--- logger.py
       |--- sub_package
           |--- __init__.py
           |--- sub_module.py
   

   In this example, it just adds a StreamHandler to the logger which is named parent_package.

   # main.py which uses the parent_package
   import logging
   
   import parent_package
   
   def main():
       # Method 1
       _logger = parent_package.get_logger()
       # Method 2
       # _logger = parent_package.logger.get_logger("parent_package")
   
       sh = logging.StreamHandler()
       sh.setLevel(logging.DEBUG)
       sh.setFormatter(
           logging.Formatter(
               "[%(levelname)s]: %(message)s [%(asctime)s](%(filename)s:%(lineno)d)"
           )
       )
       _logger.addHandler(sh)
   
       # do something else
   
   if __name__ == "__main__":
       main()
   

Reference

• Python logging HOWTO