1

根据 Kinesis 文档，序列号应该是唯一的，但是我们看到在多个记录中重复使用相同的值。我们的事件生产者是内部使用 KPL 的 Spring Boot 应用程序，消费者是 AWS lambdas。我们在测试期间执行了几次重新分片。您可以在下面看到多次重复使用的样本序列号。这怎么可能？

"Records": [{
    "kinesis": {
        "kinesisSchemaVersion": "1.0",
        "partitionKey": "00000000000000002",
        "sequenceNumber": "49596124085897508159438713510240079964989152308217511954",
        "data": "************************",
        "approximateArrivalTimestamp": 1558991793.009
    },
    "eventSource": "aws:kinesis",
    "eventVersion": "1.0",
    "eventID": "shardId-000000000001:49596124085897508159438713510240079964989152308217511954",
    "eventName": "aws:kinesis:record",
    "invokeIdentityArn": "-----------------",
    "awsRegion": "us-east-1",
    "eventSourceARN": "-----------------"
}, {
    "kinesis": {
        "kinesisSchemaVersion": "1.0",
        "partitionKey": "00000000000000003",
        "sequenceNumber": "49596124085897508159438713510240079964989152308217511954",
        "data": ""************************",",
        "approximateArrivalTimestamp": 1558991793.009
    },
    "eventSource": "aws:kinesis",
    "eventVersion": "1.0",
    "eventID": "shardId-000000000001:49596124085897508159438713510240079964989152308217511954",
    "eventName": "aws:kinesis:record",
    "invokeIdentityArn": "-----------------",
    "awsRegion": "us-east-1",
    "eventSourceARN": "-----------------"
}, {
    "kinesis": {
        "kinesisSchemaVersion": "1.0",
        "partitionKey": "00000000000000004",
        "sequenceNumber": "49596124085897508159438713510240079964989152308217511954",
        "data": ""************************",",
        "approximateArrivalTimestamp": 1558991793.009
    },
    "eventSource": "aws:kinesis",
    "eventVersion": "1.0",
    "eventID": "shardId-000000000001:49596124085897508159438713510240079964989152308217511954",
    "eventName": "aws:kinesis:record",
    "invokeIdentityArn": "-----------------",
    "awsRegion": "us-east-1",
    "eventSourceARN": "-----------------"
}]
4

1 回答 1

1

当 Kinesis 流写入器将 KPL 与用户记录聚合（请参阅消费者解聚合）一起使用时，用户记录会一起批处理并作为单个 Kinesis 记录交付给常规 Kinesis 消费者。Kinesis 记录序列号在这种情况下是唯一的，但我们需要实现解聚合。

但是，如果为 Lambda 启用了增强型扇出，则用户记录将作为单独的 Kinesis 记录交付（不需要解聚合），并且它们共享相同的序列号。

因此，Kinesis 记录序列号并不总是唯一的。

于 2019-08-09T14:05:59.637 回答