5

使用 Azure Search .net SDK，当您尝试索引文档时，您可能会遇到异常IndexBatchException。

从这里的文档：

        try
        {
            var batch = IndexBatch.Upload(documents);
            indexClient.Documents.Index(batch);
        }
        catch (IndexBatchException e)
        {
            // Sometimes when your Search service is under load, indexing will fail for some of the documents in
            // the batch. Depending on your application, you can take compensating actions like delaying and
            // retrying. For this simple demo, we just log the failed document keys and continue.
            Console.WriteLine(
                "Failed to index some of the documents: {0}",
                String.Join(", ", e.IndexingResults.Where(r => !r.Succeeded).Select(r => r.Key)));
        }

e.FindFailedActionsToRetry 应该如何用于创建一个新批次以重试失败操作的索引？

我创建了一个这样的函数：

    public void UploadDocuments<T>(SearchIndexClient searchIndexClient, IndexBatch<T> batch, int count) where T : class, IMyAppSearchDocument
    {
        try
        {
            searchIndexClient.Documents.Index(batch);
        }
        catch (IndexBatchException e)
        {
            if (count == 5) //we will try to index 5 times and give up if it still doesn't work.
            {
                throw new Exception("IndexBatchException: Indexing Failed for some documents.");
            }

            Thread.Sleep(5000); //we got an error, wait 5 seconds and try again (in case it's an intermitent or network issue

            var retryBatch = e.FindFailedActionsToRetry<T>(batch, arg => arg.ToString());
            UploadDocuments(searchIndexClient, retryBatch, count++);
        }
    }

但我认为这部分是错误的：

var retryBatch = e.FindFailedActionsToRetry<T>(batch, arg => arg.ToString());
4

2 回答 2

6

的第二个参数FindFailedActionsToRetrynamedkeySelector是一个函数，它应该返回模型类型上代表文档键的任何属性。在您的示例中，您的模型类型在内部编译时是未知的UploadDocuments，因此您需要更改UploadsDocuments为也获取keySelector参数并将其传递给FindFailedActionsToRetry. 的调用者UploadDocuments需要指定一个特定于 type 的 lambda T。例如，如果T是本文Hotel示例代码中的示例类，则 lambda 必须是，因为它的属性用作文档键。hotel => hotel.HotelIdHotelIdHotel

顺便说一句，你的 catch 块中的等待不应该等待固定的时间。如果您的搜索服务负载过重，等待持续的延迟并不能真正帮助它有时间恢复。相反，我们建议以指数方式后退（例如——第一次延迟是 2 秒，然后是 4 秒，然后是 8 秒，然后是 16 秒，直到某个最大值）。

于 2016-10-13T18:41:27.267 回答
1

我在Bruce 的回答和评论中采纳了他的建议，并使用Polly实施了它。

  • 指数回退最多一分钟，之后每隔一分钟重试一次。
  • 只要有进展就重试。5 次请求后超时，没有任何进展。
  • IndexBatchException 也为未知文件抛出。我选择忽略此类非暂时性故障，因为它们可能表示不再相关的请求（例如，在单独的请求中删除了文档）。
int curActionCount = work.Actions.Count();
int noProgressCount = 0;

await Polly.Policy
    .Handle<IndexBatchException>() // One or more of the actions has failed.
    .WaitAndRetryForeverAsync(
        // Exponential backoff (2s, 4s, 8s, 16s, ...) and constant delay after 1 minute.
        retryAttempt => TimeSpan.FromSeconds( Math.Min( Math.Pow( 2, retryAttempt ), 60 ) ),
        (ex, _) =>
        {
            var batchEx = ex as IndexBatchException;
            work = batchEx.FindFailedActionsToRetry( work, d => d.Id );

            // Verify whether any progress was made.
            int remainingActionCount = work.Actions.Count();
            if ( remainingActionCount == curActionCount ) ++noProgressCount;
            curActionCount = remainingActionCount;
        } )
    .ExecuteAsync( async () =>
    {
        // Limit retries if no progress is made after multiple requests.
        if ( noProgressCount > 5 )
        {
            throw new TimeoutException( "Updating Azure search index timed out." );
        }

        // Only retry if the error is transient (determined by FindFailedActionsToRetry).
        // IndexBatchException is also thrown for unknown document IDs;
        // consider them outdated requests and ignore.
        if ( curActionCount > 0 )
        {
            await _search.Documents.IndexAsync( work );
        }
    } );
于 2020-04-29T16:42:44.683 回答