0

我正在编写一个蛮力脚本，并希望获得建议，以通过降低其复杂性来提高其效率。

我第一次使用该multithread库，但很快意识到它不适合基于 CPU 的任务，因此，我切换到该multiprocessing库并注意到脚本执行时间的真正改进。

  • 当从一个进程（需要 4 分钟）跳转到 8 个进程（需要 46 秒）时，会有真正的改进。以上 8 个进程脚本需要更长时间运行，这是可以预期的吗？
  • 我用单词列表填充 a multiprocessing.Queue()，然后，我用队列提供进程。直接从单词表中读取一行然后不使用就直接处理它会更快吗？

用词表的内容填充队列的方法：

def populate_queue(wordlist, verbose):
    """ Loads up the wordlist
        into the multiprocessing.Queue() 
    """
    queue = multiprocessing.Queue()

    for entry in wordlist:
        if verbose:
            print(DEBUG + "Inserting into queue: " + Style.BRIGHT + entry.rstrip() + RESET)
        queue.put(entry.rstrip())

    return queue

我的脚本的主要方法：

def main():
     # Register the CTRL+C trap
    signal.signal(signal.SIGINT, signal_handler)

    global start_time

    args = parse_args()

    process_count = args.process_count
    token = args.token
    wordlist = args.wordlist
    verbose = args.verbose

    processes = []

    ## Variables summary
    print(INFO + "JWT: " + Style.BRIGHT + "{}".format(token) + RESET)
    print(INFO + "Wordlist: " + Style.BRIGHT + "{}".format(wordlist.name) + RESET)

    start_time = time.time()
    print("[*] starting {}".format(time.ctime()))

    # Load and segmentate the wordlist into the queue
    print(INFO + "Processing the wordlist..." + RESET)
    queue = populate_queue(wordlist, verbose)

    print(INFO + "Total retrieved words: " + Style.BRIGHT + "{}".format(queue.qsize()) + RESET)

    for i in range(process_count):
        process = Process(queue, token, verbose)
        print(INFO + "Starting Process #{}".format(i) + RESET)
        process.start()
        processes.append(process)

    print(WARNING + "Pour yourself some coffee, this might take a while..." + RESET)

    # Block the parent-process until all the child-processes finish to process the queue
    for process in processes:
        process.join()

    if not exit_flag:
        print(RESULT + "No match found" + Style.RESET_ALL)

    end_time = time.time()
    print("[*] finished {}".format(time.ctime()))

    elapsed_time = end_time - start_time
    print("[*] elapsed time {} sec".format(elapsed_time))

任何建议都会受到欢迎。

谢谢

4

0 回答 0