I wrote a port scanner test using pytest. It receives a json file with a list of ip addresses and then runs the test for each ip address using parametrize. A log file is created with custom log messages using the Python logging module. I can also significantly reduce the runtime by using pytest-xdist for parallel execution.
The problem is that when running the tests using pytest-xdist, only one test gets logged to the log file while the CLI output is exactly as expected. For example if the test runs 4 times for 4 different ip addresses then only the results of one of those runs is recorded to the log file. The test that does get logged seems to be random and changes from run to run. What is the correct way to log to an external file when executing tests in parallel?
# Contents of pytest.ini
[pytest]
log_cli = 1
log_cli_level = CRITICAL
log_cli_format = %(message)s
log_file = port_scanner.log
log_file_level = DEBUG
log_file_format = %(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)
log_file_date_format=%Y-%m-%d %H:%M:%S