在编写一种备份应用程序时,我对 Windows 上的文件复制性能进行了评估。
我有几个问题,我想知道你的意见。
谢谢!
卢卡斯。
问题:
为什么与 1 GiB 文件相比,复制 10 GiB 文件时的性能要慢得多?
为什么 shutil.copyfile 这么慢?
为什么 win32file.CopyFileEx 这么慢?这可能是因为标志 win32file.COPY_FILE_RESTARTABLE 吗?但是,它不接受 int 1000 作为标志 (COPY_FILE_NO_BUFFERING),建议将其用于大文件:http: //msdn.microsoft.com/en-us/library/aa363852%28VS.85%29.aspx
使用空的 ProgressRoutine 似乎对完全不使用 ProgressRoutine 没有影响。
是否有一种替代的、性能更好的方法来复制文件并获得进度更新?
1 GiB 和 10 GiB 文件的结果:
test_file_size 1082.1 MiB 10216.7 MiB
METHOD SPEED SPEED
robocopy.exe 111.0 MiB/s 75.4 MiB/s
cmd.exe /c copy 95.5 MiB/s 60.5 MiB/s
shutil.copyfile 51.0 MiB/s 29.4 MiB/s
win32api.CopyFile 104.8 MiB/s 74.2 MiB/s
win32file.CopyFile 108.2 MiB/s 73.4 MiB/s
win32file.CopyFileEx A 14.0 MiB/s 13.8 MiB/s
win32file.CopyFileEx B 14.6 MiB/s 14.9 MiB/s
测试环境:
Python:
ActivePython 2.7.0.2 (ActiveState Software Inc.) based on
Python 2.7 (r27:82500, Aug 23 2010, 17:17:51) [MSC v.1500 64 bit (AMD64)] on win32
source = mounted network drive
source_os = Windows Server 2008 x64
destination = local drive
destination_os = Windows Server 2008 R2 x64
笔记:
'robocopy.exe' and 'cmd.exe /c copy' were run using subprocess.call()
win32file.CopyFileEx A(不使用 ProgressRoutine):
def Win32_CopyFileEx_NoProgress( ExistingFileName, NewFileName):
win32file.CopyFileEx(
ExistingFileName, # PyUNICODE | File to be copied
NewFileName, # PyUNICODE | Place to which it will be copied
None, # CopyProgressRoutine | A python function that receives progress updates, can be None
Data = None, # object | An arbitrary object to be passed to the callback function
Cancel = False, # boolean | Pass True to cancel a restartable copy that was previously interrupted
CopyFlags = win32file.COPY_FILE_RESTARTABLE, # int | Combination of COPY_FILE_* flags
Transaction = None # PyHANDLE | Handle to a transaction as returned by win32transaction::CreateTransaction
)
win32file.CopyFileEx B(使用空 ProgressRoutine):
def Win32_CopyFileEx( ExistingFileName, NewFileName):
win32file.CopyFileEx(
ExistingFileName, # PyUNICODE | File to be copied
NewFileName, # PyUNICODE | Place to which it will be copied
Win32_CopyFileEx_ProgressRoutine, # CopyProgressRoutine | A python function that receives progress updates, can be None
Data = None, # object | An arbitrary object to be passed to the callback function
Cancel = False, # boolean | Pass True to cancel a restartable copy that was previously interrupted
CopyFlags = win32file.COPY_FILE_RESTARTABLE, # int | Combination of COPY_FILE_* flags
Transaction = None # PyHANDLE | Handle to a transaction as returned by win32transaction::CreateTransaction
)
def Win32_CopyFileEx_ProgressRoutine(
TotalFileSize,
TotalBytesTransferred,
StreamSize,
StreamBytesTransferred,
StreamNumber,
CallbackReason, # CALLBACK_CHUNK_FINISHED or CALLBACK_STREAM_SWITCH
SourceFile,
DestinationFile,
Data): # Description
return win32file.PROGRESS_CONTINUE # return of any win32file.PROGRESS_* constant