0

帮助解决这里的问题是脚本搜索互联网并保存,但他将它们保留为错误的编码,而对于UTP-8,您可以在此处插入程序编码,请插入(当页面保存时,它的内容也是错位字符)

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.net.URL;
import java.net.URLConnection;
import java.util.Scanner;


public class url{
  public static void main(String[] args) {
        try {
            URL PageUrl;
            URLConnection GetConn = null;
            GetConn = null;
            Scanner sc = new Scanner(new File("C:\\test\\url.txt")); 
            String htmlPage;
            while (sc.hasNext()){
                htmlPage = sc.nextLine();
                PageUrl = new URL(htmlPage);
                GetConn = PageUrl.openConnection();
                GetConn.connect();
                // establish connection:
                Scanner scUrl = new Scanner(GetConn.getInputStream());

                StringBuffer sb = new StringBuffer();
                while(scUrl.hasNext()){
                    sb.append(scUrl.nextLine());
                }
                scUrl.close();

                String htmlFileName = ("C:\\test\\1\\"+title(sb.toString())+".html");
                FileWriter FWriter = new FileWriter(htmlFileName);
                BufferedWriter BWriter = new BufferedWriter(FWriter);

                BWriter.write(sb.toString());
                BWriter.close();
            }// end try
            sc.close();
        }
        catch (IOException io) {
            System.out.println(io);
        }
    }

    private static String title(String str){

        return str.substring(str.indexOf("title>")+6, str.indexOf("</title>"));
    }

}
4

2 回答 2

2

new Scanner(GetConn.getInputStream());

从您的网页获取输入流(面向字节)。但它使用的是 JVM 的默认编码,而您并没有查看实际页面本身的编码。

我可能会使用JTidy之类的库来正确解码/解析 HTML,然后向它询问页面标题。上面的代码可能非常脆弱。

于 2012-11-20T11:29:37.910 回答
1

要补充@Brian Agnew的答案-
据我所知应该是:

new Scanner(GetConn.getInputStream(),"UTF-8");
于 2012-11-20T11:31:30.487 回答