python - 如何从链接列表中下载图像？

Question

我有一个图片链接列表（大约 5000 行），我需要知道如何快速下载所有图片。请帮助我的代码：

import concurrent.futures
import urllib.request

catname = 'amateur'

def getimg (count, endcount):
    while (count < endcount):
      urllib.request.urlretrieve(URLS[count], catname+'/images/'+catname+str(count)+'.jpg')
      URLS[count] = catname+'/images/'+catname+str(count)+'.jpg'
      count = count + 1

with concurrent.futures.ThreadPoolExecutor(max_workers=50) as e:
    e.submit(getimg, 0, 5000)

它工作正常但很慢。

Answer 1

您的代码下载 5000 张图片 50 次。尝试以下操作：

import concurrent.futures
import urllib.request

catname = 'amateur'

def getimg(count):
    localpath = '{0}/images/{0}{1}.jpg'.format(catname, count)
    urllib.request.urlretrieve(URLS[count], localpath)
    URLS[count] = localpath

with concurrent.futures.ThreadPoolExecutor(max_workers=50) as e:
    for i in range(5000):
        e.submit(getimg, i)