1

我几乎完成了一个抓取桌子的网络爬虫。这仅输出表中的第一行。任何人都可以帮助确定为什么这不会返回表中的所有行。请忽略 while 循环，因为它最终会有一个循环部分。

import urllib
from bs4 import BeautifulSoup

#file_name = "/user/joe/uspc-cpc.txt
#file = open(file_name,"w")
i=125
while i==125:
    url = "http://www.uspto.gov/web/patents/classification/cpc/html/us" + str(i) + "tocpc.html"
    print url + '\n'
    i += 1
    data = urllib.urlopen(url).read()
    print data
    #get the table data from dump
    #append to csv file
    soup = BeautifulSoup(data)
    table = soup.find("table", width='80%')
    for tr in table.findAll('tr')[2:]:
        col = row.findAll('td')
        uspc = col[0].get_text().encode('ascii','ignore')
        cpc1 = col[1].get_text().encode('ascii','ignore')
        cpc2 = col[2].get_text().encode('ascii','ignore')
        cpc3 = col[3].get_text().encode('ascii','ignore')
        print uspc + ',' + cpc1 + ',' + cpc2 + ',' + cpc3 + '\n'
        #file.write(record)

#file.close()

我正在运行的代码：

import urllib
from bs4 import BeautifulSoup

#file_name = "/users/ripple/uspc-cpc.txt"
#file = open(file_name,"w")
i=125
while i==125:
    url = "http://www.uspto.gov/web/patents/classification/cpc/html/us" + str(i) + "tocpc.html"
    print 'Grabbing from: ' + url + '\n'
    i += 1
    #get the table data from the page
    data = urllib.urlopen(url).read()
    #send to beautiful soup
    soup = BeautifulSoup(data)
    table = soup.find("table", width='80%')
    for tr in table.findAll('tr')[2:]:
        col = tr.findAll('td')

        uspc = col[0].get_text().encode('ascii','ignore').replace(" ","")
        cpc1 = col[1].get_text().encode('ascii','ignore').replace(" ","")
        cpc2 = col[2].get_text().encode('ascii','ignore').replace(" ","")
        cpc3 = col[3].get_text().encode('ascii','ignore').replace(" ","").replace("more...", "")
        record = uspc + ',' + cpc1 + ',' + cpc2 + ',' + cpc3 + '\n'
        print record
        #file.write(record)

#file.close()
4

1 回答 1

2

您正在tr用作循环变量，但row在循环中引用。如果你之前row定义过它可能会产生令人困惑的结果。

for tr in table.findAll('tr')[2:]:
    col = tr.findAll('td')

为我工作：

125/1,B 28D 1/00,B 28D 1/221,E 01C 23/081,B 28D 1/005,B 28D 1/06more...

125/2,B 23Q 35/10,B 22C 9/18,B 23B 5/162,B 23D 63/18,B 24B 53/07more...

125/3,B 28D 1/18,B 28D 1/003,B 28D 1/048,B 28D 1/181,B 24B 7/22more...

等等

于 2013-04-09T17:37:11.630 回答