0

描述：

我想使用 WebClient 的 DownloadStringAsync 下载多个 URL（大约 300 个），其中一些 URL 下载得很好，但对于其他一些 URL，下载只会永远挂起。

问题：

  1. 如何捕获无法下载的有问题的网址？
  2. 根据附加的代码，如何实现“DownloadStringAsync”方法的超时时间以跳过无法下载的 URL？

我已经用谷歌搜索了解决方案，但人手不足。然后尝试在此处的数据库中搜索具有相同结果的解决方案。任何帮助表示赞赏。

这是我正在使用的代码：

public class ParallelDownloading
    {
        private int _totalDownloadedURLs = 0;
        private ConcurrentQueue<DownloadFile> _queueToDownlaod;
        private IList<Task> _downloadingTasks;
        private Timer _downloadTimer;

        public event EventHandler<OnFileDownloadCompletedEventArgs> OnDownloadURLCompleted;

        private int _parallelDownloads;

        public ParallelDownloading(int parallelDownloads)
        {
            _totalDownloadedURLs = 0;
            _queueToDownlaod = new ConcurrentQueue<DownloadFile>();
            _downloadingTasks = new List<Task>();
            _downloadTimer = new Timer();

            _parallelDownloads = parallelDownloads;

            ServicePointManager.DefaultConnectionLimit = parallelDownloads;
        }

        public void StartParallelDownloading()
        {
            _downloadTimer.Elapsed += new ElapsedEventHandler(DownloadTimer_Elapsed);
            _downloadTimer.Interval = 800;
            _downloadTimer.Start();
        }

        public void EnqueueFileToDownload(DownloadFile file)
        {
            _queueToDownlaod.Enqueue(file);
        }

        void DownloadTimer_Elapsed(object sender, ElapsedEventArgs e)
        {
            StartDownload();
        }

        private void StartDownload()
        {
            lock (_downloadingTasks)
            {
                if (_downloadingTasks.Count < _parallelDownloads && _queueToDownlaod.Count > 0)
                {
                    DownloadFile fileToDownload;
                    if (_queueToDownlaod.TryDequeue(out fileToDownload))
                    {
                        var task = new Task(() =>
                        {
                            var client = new WebClient();
                            client.DownloadStringCompleted += new DownloadStringCompletedEventHandler(client_DownloadStringCompleted);
                            client.DownloadStringAsync(new Uri(fileToDownload.Link), fileToDownload.Link);

                        }, TaskCreationOptions.LongRunning);

                        task.ContinueWith(DownloadOverCallback, TaskContinuationOptions.None);

                        _downloadingTasks.Add(task);
                        task.Start();
                    }
                }
            }
        }

        void client_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
        {
            _totalDownloadedURLs ++;
            if (e.Error == null && e.Cancelled == false)
            {
                //do something
            }
            OnFileDownloadCompletedEventArgs evtArgs = new OnFileDownloadCompletedEventArgs();
            evtArgs.DownloadedFiles = _totalDownloadedURLs;
            OnDownloadURLCompleted(this, evtArgs);
        }

        public void DownloadOverCallback(Task downloadingTask)
        {
            lock (_downloadingTasks)
            {
                _downloadingTasks.Remove(downloadingTask);
            }
        }
    }
4

1 回答 1

0

1.如何捕获无法下载的有问题的url？为此，您可以使用 try, catch 看看那里发生了什么

2.根据附上的代码，如何实现'DownloadStringAsync'方法的超时时间以跳过无法下载的Url？最简单的方法是使用 Timer 在一段时间后调用 CancelAsync 或使用具有 Timeout 属性的 HttpWebRequest

于 2014-07-22T13:30:28.013 回答