0

我正在尝试获取从特定谷歌搜索中获得的结果数量。例如,对于 stackoverflow,有“大约 28,200,000 个结果(0.12 秒)”。

通常我会使用 XML R 包中的 xpathSApply 函数,但我有错误,不知道如何解决它们或知道是否有替代方法

library(XML)
googleURL <- "https://www.google.ca/search?q=stackoverflow"
googleInfo <- htmlParse(googleURL, isURL = TRUE)
Error: failed to load external entity "https://www.google.ca/search?q=stackoverflow"

#use of RCurl which I am not that familiar with
library(RCurl)
getURL(googleURL)
#Error in function (type, msg, asError = TRUE)  : 
#SSL certificate problem, verify that the CA cert is OK. Details:
 #error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed


# final effort
library(httr)
x <- GET(googleURL)
# no error but am not sure how to proceed
# the relevant HTML code to parse is 
# <div id=resultStats>About 28,200,000 results<nobr>  (0.12 seconds)&nbsp;</nobr></div>

非常感谢您帮助解决错误或解析 httr 对象

4

1 回答 1

3

您要求安全的 http 连接

https://www.google.ca/search?q=stackoverflow

XML正在抱怨这个RCurlhttr将下载页面。

XML请求不安全的连接

library(XML)
googleURL <- "http://www.google.ca/search?q=stackoverflow"
googleInfo <- htmlParse(googleURL, isURL = TRUE)
xpathSApply(googleInfo,'//*/div[@id="resultStats"]')
#[[1]]
#<div id="resultStats">About 28,200,000 results</div> 

RCurlssl.verifypeer = FALSE你它对我有用

library(RCurl)
googleURL <- "https://www.google.ca/search?q=stackoverflow"
googleInfo <- getURL(googleURL,ssl.verifypeer = FALSE)
googleInfo <- htmlParse(googleInfo)
# or if you want to use a cert
# system.file("CurlSSL/cacert.pem", package = "RCurl")
# googleInfo <- getURL(googleURL, cainfo = cert)
# googleInfo <- htmlParse(googleInfo)
xpathSApply(googleInfo,'//*/div[@id="resultStats"]')
#[[1]]
#<div id="resultStats">About 28,200,000 results</div> 

httr利用content

library(httr)
x <- GET(googleURL)
googleInfo <- htmlParse(content(x, as = 'text'))
xpathSApply(googleInfo,'//*/div[@id="resultStats"]')
#[[1]]
#<div id="resultStats">About 28,200,000 results</div> 
于 2013-04-27T21:22:19.833 回答