java - 从html页面中提取源代码

Question

我正在尝试使用以下代码从网站读取源代码

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;

public class GrabHTML {

public static void Connect() throws Exception{

 //Set URL
 URL url = new URL("http://www.google.ca/");
 URLConnection spoof = url.openConnection();

 //Spoof the connection so we look like a web browser
 spoof.setRequestProperty( "User-Agent", "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; H010818)" );
 BufferedReader in = new BufferedReader(new InputStreamReader(spoof.getInputStream()));
 String strLine = "";

 //Loop through every line in the source
   while ((strLine = in.readLine()) != null){

 //Prints each line to the console
   System.out.println(strLine);
  }

 System.out.println("End of page.");
}

public static void main(String[] args){

 try{
  //Calling the Connect method
  Connect();
 }catch(Exception e){

 }
 }
 }

但它只读取源代码的一部分。当我从浏览器中看到“查看源代码”时，Google.com 有更多数据。

Answer 1

删除以下语句

spoof.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; H010818)" );