0

我们想以编程方式获取当前的谷歌页面。我们使用了许多具有不同编程语言的技术,但我们无法获得正确的(当前)谷歌页面。

Java 代码示例

    public class GoogleParser {

public static void main(String[] args){
      GoogleParser googleParser = new GoogleParser();
      googleParser.execute();
}
public void execute(){
String[] params = {"ankara nüfusu"};    
     final URL url = encodeGoogleQuery(params);

       System.out.println("Downloading [" + url + "]...\n\n\n\n\n");
        try {
final String html = downloadString(url);
System.out.println(html);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private static String downloadString(final InputStream stream) throws IOException {
final ByteArrayOutputStream out = new ByteArrayOutputStream();
int ch;
while (-1 != (ch = stream.read()))
    out.write(ch);
return out.toString();
}
  private static String downloadString(final URL url) throws IOException {
       final String agent = "Mozilla/21.0 (Windows; U; Windows 7; en-US)";
       final URLConnection connection = url.openConnection();
       connection.setRequestProperty("User-Agent", agent);
       final InputStream stream = connection.getInputStream();
       return downloadString(stream);
   }

private static URL encodeGoogleQuery(final String[] args) {
        try {
            final StringBuilder localAddress = new StringBuilder();
            localAddress.append("/search?q=");

            for (int i = 0; i < args.length; i++) {
                final String encoding = URLEncoder.encode(args[i], "UTF-8");
                localAddress.append(encoding);
                if (i + 1 < args.length)
                    localAddress.append("+");
            }

            return new URL("http", "www.google.com", localAddress.toString());

        } catch (final IOException e) {
            // Errors should not occur under normal circumstances.
            throw new RuntimeException(
                    "An error occurred while encoding the query arguments.");
        }
    }
}

Java 代码获取这个 html 页面 谷歌当前页面

 First image Java Code Result Page
 Second image Google Current Page

java 从 google 获取的 Html 页面与当前的 google 页面不同。

  1. 不同的结果
  2. 不包含 Google Now 结果(4,551 milyon(2011) 部分)
  3. 不包含谷歌图表结果(右侧安卡拉信息)
  4. 比当前页面更旧的页面
  5. 导航属性(网络、图像、视频)左侧,通常在下方搜索栏

你知道如何用编程的 java 语言获取谷歌的当前(最后)页面吗?然而,其他语言的解决方案对于解决问题很重要。

谢谢您的答复

4

1 回答 1

0

谷歌在检测谁在发送请求方面很聪明:

  1. 确保发送与浏览器相同的 cookie
  2. 确保您发送相同或有效的浏览器代理字符串
于 2013-06-22T07:35:59.320 回答