Logging in Python

Exploring the Root Logger

To get a first impression of how the logging module and a logger work, open the Python standard REPL and enter the code below:

>>> import logging
>>> logging.warning("Remain calm!")
WARNING:root:Remain calm!

The output shows the severity level before each message along with root, which is the name the logging module gives to its default logger. This output shows the default format that can be configured to include things like a timestamp or other details.

In the example above, you’re sending a message on the root logger. The log level of the message is WARNING. Log levels are an important aspect of logging. By default, there are five standard severity levels for logging events. Each has a corresponding function that can be used to log events at that level of severity.

Here are the five default log levels, in order of increasing severity:

The logging module provides you with a default logger that allows you to get started with logging without needing to do much configuration. However, the logging functions listed in the table above reveal a quirk that you may not expect:

>>> logging.debug("This is a debug message")

>>> logging.info("This is an info message")

>>> logging.warning("This is a warning message")
WARNING:root:This is a warning message

>>> logging.error("This is an error message")
ERROR:root:This is an error message

>>> logging.critical("This is a critical message")
CRITICAL:root:This is a critical message

Notice that the debug() and info() messages didn’t get logged. This is because, by default, the logging module logs the messages with a severity level of WARNING or above. You can change that by configuring the logging module to log events of all levels.

Adjusting the Log Level

To set up your basic logging configuration and adjust the log level, the logging module comes with a basicConfig() function. As a Python developer, this camel-cased function name may look unusual to you, as it doesn’t follow the PEP 8 naming conventions:

That’s because it was adopted from Log4j, a logging utility in Java. It’s a known issue in the package, but by the time it was decided to add it to the standard library, it had already been adopted by users, and changing it to meet PEP 8 requirements would cause backwards compatibility issues.

Later in this tutorial, you’ll learn about common parameters for basicConfig(). For now, you’ll focus on the level parameter to set the log level of the root logger:

>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> logging.debug("This will get logged.")
DEBUG:root:This will get logged.

By using the level parameter, you can set what level of log messages you want to record. This can be done by passing one of the top-level constants available in the module. You can use either the constant itself, its numeric value, or the string value as the level argument:

Setting a log level will enable all logging calls at the defined level and higher. For example, when you set the log level to DEBUG, all events at or above the DEBUG level will be logged.

Formatting the Output

By default, logs contain the log level, the logger’s name, and the log message. That’s good for a start. But you can enhance your logs with additional data by leveraging the format parameter of basicConfig().

The format parameter accepts a string that can contain a number of predefined attributes. You can think of these attributes as placeholders that you format into the string. The default value of format looks like this:

>>> import logging
>>> logging.basicConfig(format="%(levelname)s:%(name)s:%(message)s")
>>> logging.warning("Hello, Warning!")
WARNING:root:Hello, Warning!

This format is called printf-style string format. You may also find log formats with a dollar sign and curly braces (${}), which are related to the string.Template() class. If you’re used to modern Python string formatting, then you’re probably more familiar with using curly braces ({}) to format your strings.

You can choose one of these three styles of your format string by specifying the style parameter. The options for style are "%", "$", or "{". When you provide a style argument, then your format string must match the targeted style. Otherwise, you’ll receive a ValueError.

Restart the REPL and start a logger with a different style format. Before trying out other attributes for your logs, stick with the default format structure from before:

>>> import logging
>>> logging.basicConfig(format="{levelname}:{name}:{message}", style="{")
>>> logging.warning("Hello, Warning!")
WARNING:root:Hello, Warning!

As mentioned before, the format parameter accepts a string that can contain a number of predefined attributes. The ones you choose to use will depend on the insights that you want to get from your logs.

Besides the message text and the log level, it usually makes sense to also have a timestamp in the log record. A timestamp can give you the exact time the program sent the log message. This can help you monitor code performance or notice patterns around when some errors occur.

To add a timestamp to your logs, you can use the asctime attribute in your format string of basicConfig(). By default, asctime also shows you milliseconds. If you don’t need to be that exact or if you want to customize the timestamp, then you must add datefmt to your basicConfig() call:

>>> import logging
>>> logging.basicConfig(
...     format="{asctime} - {levelname} - {message}",
...     style="{",
...     datefmt="%Y-%m-%d %H:%M",
... )