我正在尝试使用 C# 下载网页：
var responseData = "";
var strUrl = this.QuerySelector(item, "a[class='url']").Attributes["href"].Value;
request = (HttpWebRequest)WebRequest.Create(strUrl);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = 0;
request.CookieContainer = cookies;
request.Timeout = System.Threading.Timeout.Infinite;
request.UserAgent = this.RefreshUserAgent();
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.Headers.Add("Accept-Encoding", "gzip,deflate,sdch");
request.KeepAlive = true;
request.AllowAutoRedirect = false;
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
response = (HttpWebResponse)request.GetResponse();
response.Cookies = request.CookieContainer.GetCookies(request.RequestUri);
var encoding = new System.Text.UTF8Encoding();
var responseReader = new StreamReader(response.GetResponseStream(), encoding, true);
responseData = responseReader.ReadToEnd();
response.Close();
responseReader.Close();
但该网站一直给我相同的代码，让 JavaScript 继续。我已经用 Fiddler 检查了数据 - 它只是再次导航到自身，但我无法通过此消息：
Before you can move on - please enable JavaScript.
该网站是 Manta.com，这是我的示例页面。有任何想法吗..