1

I have a python 2.7 code which uses STORBINARY function for uploading files to an ftp server and RETRBINARY for downloading from this server.

However, the issue is the upload is taking a very long time on three laptops from different brands as compared to a Dell laptop. The strange part is when I manually upload any file, it takes the same time on all the systems.

The manual upload rate and upload rate with the python script is the same on the Dell Laptop. However, on every other brand of laptop (I have tried with IBM, Toshiba, Fujitsu-Siemens) the python script has a very low upload rate than the manual attempt. Also, on all these other laptops, the upload rate using the python script is the same (1Mbit/s) while the manual upload rate is approx. 8 Mbit/s.

I have tried to vary the filesize for the upload to no avail. TCP Optimizer improved the download rate on all the systems but had no effect on the upload rate. Download rate using this script on all the systems is fine and same as the manual download rate.

I have checked the server and it has more than 90% free space. The network connection is the same for all the laptops, and I try uploading only with one laptop at a time. All the laptops have almost the same system configurations, same operating system and approximately the same free drive space. If anything the Dell laptop is a little less in terms of processing power and RAM than 2 of the others, but I suppose this has no effect as I have checked many times to see how much was the CPU usage and network usage during these uploads and downloads, and I am sure that no other virus or program has been eating up my bandwidth.

even with 'storbinary' command , when i specify the blocksize to be of 57344(56 kB), the upload rate improves to about 5 Kbit/s from 1 to 1.5 Kbit/s originally...whats the reason for that? and how can i find out the blocksize used by my manual upload client(i used filezilla), or better yet the optimal blocksize for upload?? @guidot

Complete code :

def upnew(counter=0):

    f=open("c:/10", "w")

    f.write(os.urandom(10*1024*1024))
    f.close()

    print "Logging in..."

    ftpserver='xxxxxxx'

    ftpuser='xxxxxxx'

    ftppw='xxxxxxxxx'

    ftp = FTP(ftpserver)    
    ftp.login(ftpuser, ftppw)  

    t = open("c:/10", "rb")                       
    upstart = time.clock() 

    ftp.storbinary('STOR 10', t)

    upende = time.clock()-upstart

    print ((10*8)/upende)

    print "press Return to disconnect"
    raw_input()

    ftp.quit()
    print "FTP Verbindung abgebaut"


upnew(1)
4

3 回答 3

0

我可能错了，但问题似乎在于您调用和使用 ftp.storbinary() 的方式

我会尝试改用 ftp.ntransfercmd() 并在处理过程中使用缓冲区来分解传输。这为您提供了额外的好处，即能够跟踪 ftp 传输的进度，并且理论上允许您根据需要暂停和重新启动该过程。

尚未测试此脚本的性能，但您可以尝试执行以下操作：

    def ftpUploader():
        BLOCKSIZE = 57344 # size 56 kB

        ftp = ftplib.FTP()
        ftp.connect(host)
        ftp.login(login, passwd)
        ftp.voidcmd("TYPE I")
        f = open(zipname, 'rb')
        datasock, esize = ftp.ntransfercmd(
            'STOR %s' % os.path.basename(zipname))
        size = os.stat(zipname)[6]
        bytes_so_far = 0
        print 'started'
        while 1:
            buf = f.read(BLOCKSIZE)
            if not buf:
                break
            datasock.sendall(buf)
            bytes_so_far += len(buf)
            print "\rSent %d of %d bytes %.1f%%\r" % (
                  bytes_so_far, size, 100 * bytes_so_far / size)
            sys.stdout.flush()

        datasock.close()
        f.close()
        ftp.voidresp()
        ftp.quit()
        print 'Complete...'
于 2012-09-13T14:47:50.357 回答
0

Python 主要用于脚本和流程自动化，在技术上不被认为是一种快速变化的语言（尽管比大多数其他脚本语言更快）。Filezilla 使用 C/C++ 编码，性能远优于 Python。话虽如此，这不是一个公平的比较，我们应该在尝试识别可能导致一般性能问题的逻辑问题时考虑它。

storbinary 基本上充当 ntransfercmd 的包装器，调用 ntransfercmd 而不需要我们定义自己的缓冲区（这就是我之前推荐的原因）。

此外，在再次分析您的代码 snip-it 之后，我注意到您通过 print 语句调用 storbinary ......这是错误的吗？

此时，我们将需要此示例中使用的所有相关代码来识别可能影响性能的任何逻辑问题，请在您之前的 snip-it 的基础上为我们提供更多信息。

这里要考虑的另一个因素是您进行测试的一般系统环境......考虑您执行所述测试的每个系统的位置，它们与 FTP 服务器的距离，此外，ISP 或 DNS 的差异在解决与基于 TCP/IP 的连接（例如 ftp）相关的性能问题时，服务器可能是一个主要因素。

于 2012-09-14T17:33:24.920 回答
0

请提供您的代码的工作示例...无法看到您如何实现 ftp 功能不可能提供有用的反馈，但通常您可能会从使用线程或套接字中受益。

于 2012-09-13T12:32:54.663 回答