python - Programmatically Set File Write Rate

Question

I want to be able to easily set a benchmarking program's file write rate. It's a python program that I'm using to test another system. I'd like to be able to control the rate of file creation. One method I have thought of is to have a function with an argument for the number of files to create. This could be called in a loop which keeps track of the clock and only calls the function every second. This would fulfill the requirement of creating a certain number of files every second. The problem with this is that there could be a chunk of dead time (milliseconds, but still). I'd like a continuous load.

Answer 1

您必须以某种方式跟踪实际执行文件 I/O 调用所需的时间，并调整操作之间的睡眠时间。调整需要持续进行，因为睡眠和 IO 调用可能需要不同的时间，具体取决于系统负载。

如果您希望平均每秒执行 N 次操作，您可以运行几秒（或更长）的循环，并在每轮之后查看您是否运行得太快或太慢，并调整 sleep() 之间完成的时间基于此向上或向下的每个操作。如果你跑得太快，增加睡眠时间，如果你只是有点太快，增加更少。

import time
# target rate: 100 ops / 1 second
target = 100.0
round_time = 1.0
# at first, assume the writes are immediate
sleepTime = round_time/target

ops = 0
t_start = time.time()

while True:
    #doYourIOoperationHere()
    ops += 1
    time.sleep(sleepTime)

    # adjust sleep time periodically
    if ops == target:
        t_end = time.time()
        elapsed = t_end - t_start
        difference = round_time - elapsed
        # print out the vars here to debug adjustment
        print "%d ops done, elapsed %.3f, difference %.3f" % (ops, elapsed, difference)
        # increase or decrease the sleep time, approach the target time slowly
        sleepTime += difference/target/2
        t_start = time.time()
        ops = 0

或者类似的东西（未经测试的简单代码）。这可能不适用于非常高的 IO 速率或系统负载，您可能必须在每个睡眠调用中开始执行多个写入操作。此外，可能需要比 1 秒更长的平均时间。