0

我尝试通过添加一个临时结果集来修改现有的 Azure 流分析作业。但是当我运行 SA 作业时，它会引发运行时错误，并且水印延迟会增加。

以下是流分析作业中的现有 SAQL：

-- Reading from Event Hub
WITH INPUTDATASET AS (
    SELECT 
        udf.udf01([signals2]) AS flat
    FROM [signals2]
    PARTITION BY PartitionId
    WHERE [signals2].ABC IS NOT NULL
),

INPUT1 AS (
    SELECT
        ID,
        SIG1,
        SIG2
    FROM [signals2] as input
    WHERE GetArrayLength(input.XYZ) >=1
)

--Dump the data from above result set into cosmosDB

我尝试将以下临时结果集添加到 SAQL：

INPUT2 AS (
    SELECT
        ID,
        SIG3,
        SIG4
    FROM [signals2] as input
    WHERE GetArrayLength(input.XYZ) =0
)

现在，当我启动 SA 作业时，它会引发运行时错误。

当我获取以下日志时，是我收到的错误日志。

TimeGenerated,Resource,"Region_s",OperationName,"properties_s",Level
"2020-01-01T01:10:10.085Z",SAJOB01,"Japan West","Diagnostic: Diagnostic Error","{""Error"":null,""Message"":""First Occurred: 01\/01\/2020 01:10:10 | Resource Name: signals2 | Message: Maximum Event Hub receivers exceeded. Only 5 receivers per partition are allowed.\r\nPlease use dedicated consumer group(s) for this input. If there are multiple queries using same input, share your input using WITH clause. \r\n "",""Type"":""DiagnosticMessage"",""Correlation ID"":""xxxx""}",Error
"2020-01-01T01:10:10.754Z",SAJOB01,"Japan West","Receive Events: ","{""Error"":null,""Message"":""We cannot connect to Event Hub partition [25] because the maximum number of allowed receivers per partition in a consumer group has been reached. Ensure that other Stream Analytics jobs or Service Bus Explorer are not using the same consumer group. The following information may be helpful in identifying the connected receivers: Exceeded the maximum number of allowed receivers per partition in a consumer group which is 5. List of connected receivers - AzureStreamAnalytics_xxxx_25, AzureStreamAnalytics_xxxx_25, AzureStreamAnalytics_zzz_25, AzureStreamAnalytics_xxx_25, AzureStreamAnalytics_xxx_25. TrackingId:xxx_B7S2, SystemTracker:eventhub001-ns:eventhub:ehub01~26|consumergrp01, Timestamp:2020-01-01T01:10:10 Reference:xxx, TrackingId:xxx_B7S2, SystemTracker:eventhub001-ns:eventhub:ehub01~26|consumergrp01, Timestamp:2020-01-01T01:10:10, referenceId: xxx_B7S2"",""Type"":""EventHubBasedInputQuotaExceededError"",""Correlation ID"":""xxxx""}",Error...

对于 SA 作业，输入信号 2有一个专用的消费者组 ( consumergrp01 )

对于此流分析作业，可以使用专用的使用者组。此使用者组的分区上有 3 个读取器，但由于最大事件中心接收器超过，它仍然抛出错误。为什么会这样？

4

1 回答 1

0

消息：超出最大事件中心接收器。每个分区只允许 5 个接收器。

我认为错误消息清楚地说明了运行时错误的根本原因。请参阅此文档中的声明：

每个消费者组的一个分区上最多可以有 5 个并发读者；但是，建议每个消费者组的分区上只有一个活动接收器。在单个分区内，每个阅读器都会接收所有消息。如果您在同一个分区上有多个阅读器，那么您将处理重复的消息。你需要在你的代码中处理这个问题，这可能不是微不足道的。但是，在某些情况下这是一种有效的方法

您可以按照错误消息中的建议进行操作：If there are multiple queries using same input, share your input using WITH clause. 尝试将其隔离input2为临时结果集，然后在其他临时结果中使用它。

于 2020-01-03T08:06:58.223 回答