2

我使用scrapy抓取网页，我希望以某种格式输出到xml文件，下面是我的代码。

物品类别

class Item(Item):
# define the fields for your item here like:
    id = Field()
    name = Field()
    address = Field()
    birthdate = Field()
    review = Field()

蜘蛛类

class FriendSpider(BaseSpider):
# identifies of the Spider
name = "friend"
count = 0 
allowed_domains = ["example.com.us"]
start_urls = [
    "http://example.com.us/biz/friendlist/"
]
def start_requests(self):
    for i in range(0,1722,40):
        yield self.make_requests_from_url("http://example.com.us/biz/friendlist/?start=%d" % i)

def parse(self, response):

    response = response.replace(body=response.body.replace('<br />', '\n')) 
    hxs = HtmlXPathSelector(response)
    sites = hxs.select('//ul/li')
    items = []

    for site in sites:
        item = Item()
        self.count += 1
        item['id'] = str(self.count)
        item['name'] = site.select('.//div/div/h4/text()').extract()
        item['address'] = site.select('h4/span/text()').extract()
        item['review'] = ''.join(site.select('.//div[@class="review"]/p/text()').extract())
        item['birthdate'] = site.select('.//div/div/h5/text()').extract()

        items.append(item)
    return items

输出格式如下：

<?xml version="1.0" encoding="utf-8"?>
<items>
  <item>
     <id>1</id>
     <name><value>Keith</value></name>
     <review>txt............</review>
     <address><value>United States</value></address>
     <birthdate><value>1988-04-03</value></birthdate>
  </item>
  .....
<items>

如何将 xml 格式自定义为以下：删除值标签并将 id 移动到项目根目录。

<?xml version="1.0" encoding="utf-8"?>
<items>
  <friend id = "1">
     <name>Keith</name>
     <review>txt............</review>
     <address>United States</address>
     <birthdate>1988-04-03</birthdate>
  </friend>
  .....
<items>
4

1 回答 1

1

例如，对于您的问题，您可以获得此页面上提供的列表之一，或者根据OrderedDict类型编写您自己的 XML 序列化器。爬取结束后，您可以简单地serialize()使用所需的参数调用并获取 XML 文档。

于 2013-04-08T03:20:41.590 回答