← Back

Python logging library – log to several files

July 30, 2020

Today while i was working on a service which generates release notes automatically i wanted to add some more logging. I wanted to log different parts of the service to different log files. In this post i will show how one easily can add several loggers and log to several files.

In this example we use two simple classes. A Notify class and a VerboseNotify class which inherits from Notify. Notify will print a message to stdout and VerboseNotify will do the same but with a timestamp prefix.

Using pythons logging library which i set up using basicConfig.

from datetime import datetime
import logging


logging.basicConfig(
    format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
    filename='root.log',
    level=logging.DEBUG,
    datefmt='%Y-%m-%d %H:%M:%S',
)


def setup_logger(name, file_path):
    logger = logging.getLogger(name)
    handler = logging.FileHandler(file_path)
    handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter(
        '%(asctime)s %(name)-16s %(levelname)-8s %(message)s',
        '%Y-%m-%d %H:%M:%S',
    )
    logger.addHandler(handler)
    handler.setFormatter(formatter)
    return logger


class Notify():

    def __init__(self):
        self.logger = setup_logger('notify', 'notify.log')

    def notify(self, message):
        print(message)
        self.logger.info(f'Notification message send: {message}')


class VerboseNotify(Notify):

    def __init__(self):
        super().__init__()
        self.logger = setup_logger('verbose_notify', 'verbose_notify.log')

    def notify(self, message):
        message = f'{datetime.now()} {message}'
        super().notify(message)


if __name__ == '__main__':
    notify = Notify()
    notify.notify('Test')

    verbose_notify = VerboseNotify()
    verbose_notify.notify('Verbose Test')

The gotcha i had was that i needed to use getLogger(logger_name) to get a logger and set that up with a FileHandler. So the important part is here:

def setup_logger(name, file_path):
    logger = logging.getLogger(name)
    handler = logging.FileHandler(file_path)
    handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter(
        '%(asctime)s %(name)-16s %(levelname)-8s %(message)s',
        '%Y-%m-%d %H:%M:%S',
    )
    logger.addHandler(handler)
    handler.setFormatter(formatter)
    return logger

Running the example will output the following into the three log files:

cat root.log 
2020-07-30 21:24:24 notify       INFO     Notification message send: Test
2020-07-30 21:24:24 verbose_notify INFO     Notification message send: 2020-07-30 21:24:24.124272 Verbose Test

cat notify.log 
2020-07-30 21:24:24 notify           INFO     Notification message send: Test

cat verbose_notify.log 
2020-07-30 21:24:24 verbose_notify   INFO     Notification message send: 2020-07-30 21:24:24.124272 Verbose Test