仅以 Twitter 为例,此代码从 Twitter 页面中抓取第 5 条推文。该页面包含一个链接,除非我尝试使用 lxml 和 xpath 将其拉起,它只显示将链接从末尾截断的文本。
脚本:
import urllib2
from lxml import etree
xpathselector = "/html/body/div/div[2]/div/div[5]/div[2]/div/ol/li[5]/div/div/p"
url = "https://twitter.com/memphismayfire"
response = urllib2.urlopen(url)
htmlparser = etree.HTMLParser()
tree = etree.parse(response, htmlparser)
result = tree.xpath(xpathselector)
print result[0].text
印刷:
'Miles Away' Acoustic 在 iTunes 上可用!谁下载了单曲?!让我们上单曲榜!!关联:
来自 xPath 位置的 HTML:
<p class="js-tweet-text tweet-text">'Miles Away' Acoustic is available on iTunes! Who's downloaded the single?! Let's get it up the Singles Chart!! Link: <a title="http://smarturl.it/mmf-miles-away" target="_blank" class="twitter-timeline-link" data-expanded-url="http://smarturl.it/mmf-miles-away" dir="ltr" rel="nofollow" href="http://t.co/fU2hVqAiSq" f52ae163cfcf0237f="true"><span class="tco-ellipsis"></span><span class="invisible">http://</span><span class="js-display-url">smarturl.it/mmf-miles-away</span><span class="invisible"></span><span class="tco-ellipsis"><span class="invisible"> </span></span></a><div ida2bb72480="_p_mzkte2cwofawsu3r.t.co" style="cursor: pointer; width: 16px; height: 16px;display: inline-block;"> </div></p>
打印 xpath 的全部内容而不仅仅是文本的最佳方法是什么?谢谢您的帮助!