Logging like a Lumberjack

Pythonlogging / 2 min read

Sprinkling status messages across you code using print() statements can be a good temporary fix for tracking down issues in your code.

But it can get messy and there’s no way for you to selectively disable them. Sure you can redirect output to a file or /dev/null but this is an all-or-nothing solution. How about disabling some messages and retaining others?

This is where the logging module comes into its own.

Local Logging

For each project I like to create a logger.py file that looks like this:

import logging
from logging import debug, info, warning, error
from datetime import datetime, UTC

FMT_MESSAGE = "%(asctime)s [%(levelname)7s] %(message)s"
FMT_DATETIME = "%Y-%m-%d %H:%M:%S"

LOG_FILENAME = datetime.now(UTC).strftime("%Y-%m-%d-%H%M%S.log")

# Get default logger.
#
logger = logging.getLogger()

# Set logging level.
#
logger.setLevel(logging.DEBUG)

formatter = logging.Formatter(FMT_MESSAGE, datefmt=FMT_DATETIME)

# Logging to console.
#
console = logging.StreamHandler()
console.setFormatter(formatter)
logger.addHandler(console)

# Logging to file.
#
file = logging.FileHandler(LOG_FILENAME, encoding="utf-8")
file.setFormatter(formatter)
logger.addHandler(file)

# Suppress logging from other packages.
#
logging.getLogger("asyncio").setLevel(logging.WARNING)
logging.getLogger("botocore").setLevel(logging.WARNING)

This does a number of things:

  • creates a formatter that specifies the layout of the log messages;
  • set the logging level;
  • add handler for logging to console and set format;
  • add handler for logging to file and set format (file name embeds date and time); and
  • restrict logging from a few other packages (add more as required).

Test it.

import logger

logger.info("An INFO message.")
logger.debug("A DEBUG message.")

2024-04-18 12:28:02 [   INFO] An INFO message.
2024-04-18 12:28:02 [  DEBUG] A DEBUG message.

That’s the output on the terminal. But there’s also a 2024-04-18-112802.log file that contains the same content. There will be a new log file each time you run the command. If you don’t want to create all of those log files then you can specify a fixed filename for the FileHandler object.

Integration

Provided that your main script imports the logger.py module, all other parts of your project can simply import logging. Since we have set up the default logger all subsequent logs will go to the same destination and have the same format.