HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

Logger module in child processes from os.fork()

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
forkloggermodulechildprocessesfrom

Problem

I have had a hard time understanding all the documentation on the logger module and forking processes.

My goal is to have each child process that's worked to write to their own separate log file.

This seems to work but I just want to try and verify that this is the correct way.

import os
import logging

def generate_primes(number):
    logger = logging.getLogger(str(os.getpid()) + ':logger')

    for num in range(number + 1):
       # prime numbers are greater than 1
       if num > 1:
           for i in range(2,num):
               if (num % i) == 0:
                   break
           else:
               logger.info('child:' + str(os.getpid()) + ':' + str(num))

def start_workers():
    logger = logging.getLogger('parent:prime')
    logger.info('parent forking')

    for child in range(10):
        pid = os.fork()

        if not pid:

            logger = logging.getLogger('child:' + str(os.getpid()))
            fh = logging.FileHandler('logs/child_primes_' + str(os.getpid()) + '.log')
            logger.addHandler(fh)

            generate_primes(10000)

            os._exit(0)
        else:
            os.waitpid(pid, 0)

if __name__ == "__main__":

    logging.basicConfig(filename='logs/parent_primes.log', level=logging.INFO)
    logger = logging.getLogger('parent:prime')

    logger.info('Fork parent/child logging example.')
    logger.info('Starting children')

    start_workers()

Solution

Thomas, your solutions looks good. I have few remarks regarding the code style.

-
The following code looks a bit strange to me:

for num in range(number + 1):
    # prime numbers are greater than 1
    if num > 1:
        for i in range(2,num):
        ...


You can easily replace it with:

for num in range(2, number + 1):
    for i in range(2, num):
    ...


-
Do not use string concatenations, they are making your code less readable. Better use the .format() method of the string:

'child:' + str(os.getpid()) + ':' + str(num)

# Better:
'child:{pid}:{num}'.format(pid=os.getpid(), num=num)


-
In order to not put the meta information to every logging request, you can use logging formatters. Please check that doc for more details: https://docs.python.org/2/howto/logging-cookbook.html#using-logging-in-multiple-modules.

-
To simplify logging creation and make a code a bit more readable, you can create a function get_logger:

def get_logger(pid):
    file_name = 'logs/child_primes_{}.log'.format(pid)
    file_handler = logging.FileHandler(file_name)
    name = 'child:{}'.format(pid)
    logger = logging.getLogger(name)
    logger.addHandler(file_handler)
    # TODO: Add a formater here
    return logger

Code Snippets

for num in range(number + 1):
    # prime numbers are greater than 1
    if num > 1:
        for i in range(2,num):
        ...
for num in range(2, number + 1):
    for i in range(2, num):
    ...
'child:' + str(os.getpid()) + ':' + str(num)

# Better:
'child:{pid}:{num}'.format(pid=os.getpid(), num=num)
def get_logger(pid):
    file_name = 'logs/child_primes_{}.log'.format(pid)
    file_handler = logging.FileHandler(file_name)
    name = 'child:{}'.format(pid)
    logger = logging.getLogger(name)
    logger.addHandler(file_handler)
    # TODO: Add a formater here
    return logger

Context

StackExchange Code Review Q#96364, answer score: 3

Revisions (0)

No revisions yet.