1

在 Python 中使用多处理模块时，我看到了一些关于将来自不同进程的日志放在一起的问题。我想做相反的事情，为不同的进程生成单独的日志文件，它们应该记录调用其他模块时发生的所有事情而不会被破坏。在下面的示例中，我有一个主程序（main.py）和两个模块（module1.py 和 module2.py），我希望主记录器（mainlog）写入标准输出，这很好。我还希望每个进程都有一个单独的文件，包括从模块 1 和模块 2 进行日志记录。

主要.py：

import logging
import multiprocessing as mpr
import module1
import sys

mainlog = logging.getLogger("main")
h = logging.StreamHandler(sys.stdout)
mainlog.addHandler(h)
logging.root.setLevel(logging.DEBUG)

for i in xrange(0,3):
    mainlog.info("Starting process ... %s", i)
    log = logging.getLogger("module1")
    h = logging.FileHandler("process_{0}.log".format(i))
    fmt = logging.Formatter(fmt="%(levelname)-10s:%(filename)-20s:%(message)s")
    h.setFormatter(fmt)
    log.addHandler(h)
    log.setLevel(logging.DEBUG)
    p = mpr.Process(target=module1.do_something, args=(i,))
    p.start()

一个module1.py：

import logging
import module2

log = logging.getLogger("module1")

def do_something(i):

    for j in xrange(0,100):
        log.debug("do something. process %2s. iteration %2s", i,j)
        module2.multiply(j,2)

还有一个module2.py：

import logging
log = logging.getLogger("module2")

def multiply(x,y):
    log.debug("... multiplying %s x %s = %s", x,y, x*y)
    return x*y

相反，我得到以下输出：

Starting process ... 0
Starting process ... 1
No handlers could be found for logger "module2"
Starting process ... 2
No handlers could be found for logger "module2"
No handlers could be found for logger "module2"

以及 3 个单独的日志文件（process_0.log，...），其中包含来自所有进程的消息，而不仅仅是一个。没有记录 module2.py 中的任何内容。我究竟做错了什么？

4

2 回答 2

2

我最终创建了 logging.Logger 的子类来管理日志记录到主日志和日志记录到磁盘之间的切换。现在我可以在进程中根据需要进行切换：

import logging
import sys


class CGLogger(logging.Logger):
    def __init__(self,name):
        logging.Logger.__init__(self,name)
        self.mainhandler = logging.StreamHandler(sys.stdout)
        self.addHandler(self.mainhandler)

    def stop_main_logging(self):
        self.removeHandler(self.mainhandler)

    def log_to_file(self, fn):
        self.filehandler = logging.FileHandler(fn)
        self.addHandler(self.filehandler)

    def stop_logging_to_file(self):
        self.removeHandler(self.filehandler)

    def restart_main_logging(self):
        self.addHandler(self.mainhandler)

    def switch_to_file_logging(self, fn):
        self.stop_main_logging()
        self.log_to_file(fn)

    def switch_to_main_logging(self):
        self.stop_logging_to_file()
        self.restart_main_logging(fn)

logging.setLoggerClass(CGLogger)
于 2013-08-15T17:48:54.713 回答
1

You need to configure logging in the child processes. They start off with a clean slate, and logging isn't configured in them. Are you on Windows, by any chance? Almost nothing is inherited from the parent process to the child process in Windows, whereas on POSIX, fork() semantics can allow some things to be inherited.

于 2013-08-14T23:22:13.953 回答