1

我有一个 python 程序，随着时间的推移会慢慢爬行。我已经进行了彻底的测试，并将其缩小到下载图像的方法。该方法使用 cstringIO 和 urllib。问题也可能是 urllib 的某种无限下载（该程序在数百次下载后冻结）。

关于问题可能出在哪里的任何想法？

        foundImages = []

        images = soup.find_all('img')
        print('downloading Images')

        for imageTag in images:
            gc.collect()

            url = None
            try:

                #load image into a file to determine size and width
                url = imageTag.attrs['src']
                imgFile = StringIO(urllib.urlopen(url).read())
                im = Image.open(imgFile)
                width, height = im.size

                #if width and height are both above a threshold, it is a valid image
                #so add to recipe images
                if width > self.minOptimalWidth and height > self.minOptimaHeight:
                    image = MIImage({})
                    image.originalUrl = url.encode('ascii', 'ignore')
                    image.width = width
                    image.height = height

                    foundImages.append(image)

                imgFile = None
                im = None
            except Exception:
                print('failed image download url: ' + url)
                traceback.print_exc()
                continue

        #set the main image to be the first in the array
        if len(foundImages) > 0:
            first = foundImages[0]
            recipe.imageUrl = first.originalUrl

        return foundImages
4

0 回答 0