c# - 下载网页并保存为 UTF-8 文本文件

Question

我下载一个网页如下。我想将其保存为 UTF-8 文本。但是怎么做？

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
    Encoding enc = Encoding.GetEncoding(resp.CharacterSet);
    Encoding utf8 = Encoding.UTF8;
    using (StreamWriter w = new StreamWriter(new FileStream(pathname, FileMode.Create), utf8))
    {
        using (StreamReader r = new StreamReader(resp.GetResponseStream()))
        {
            // This works, but it's bad because you read the whole response into memory:
            string s = r.ReadToEnd();
            w.Write(s);

            // This doesn't work :(
            char[] buffer = new char[1024];
            int n;
            while (!r.EndOfStream)
            {
                n = r.ReadBlock(buffer, 0, 1024);
                w.Write(utf8.GetChars(Encoding.Convert(enc, utf8, enc.GetBytes(buffer))));
            }

            // This means that r.ReadToEnd() is doing the transcoding to UTF-8 differently.
            // But how?!
        }
    }
    return resp.StatusCode;
}

不要读这一段。它只是为了消除有关代码过多​​的警告消息。

Answer 1

您可以简单地使用 WebClient 类。它支持编码和更易于使用：

WebClient webClient = new WebClient();
webClient.Encoding = System.Text.Encoding.UTF8;
webClient.DownloadFile(url, "file.txt");