HtmlUnit
我今天在试图弄清楚如何转储网站的源代码时才听说这件事。我想做的是使用脚本从网站转储源代码,但是当我运行它时,会得到一个很长的红色列表。
这是我使用的代码:
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {
String url = "http://www.runelocus.com/forums/member.php?102785&tab=aboutme#aboutme";
WebClient client = new WebClient(BrowserVersion.FIREFOX_3_6);
HtmlPage page = client.getPage(url);
System.out.println(page.getWebResponse().getContentAsString());
}
这是我得到的错误:
Exception in thread "main" org.apache.http.client.ClientProtocolException
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:822)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:776)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:152)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1439)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1358)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:307)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:358)
at HTMLDumping.htmlunittest.main(htmlunittest.java:18)
Caused by: org.apache.http.ProtocolException: Invalid header: blcc_proxy
at org.apache.http.impl.io.AbstractMessageParser.parseHeaders(AbstractMessageParser.java:224)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
如果有人可能知道如何解决这个问题,请提出一些建议。接受所有反馈。