我希望有人可以帮助我，因为我对 kinesis firehose 和 firehose 代理相当陌生。
我已经为我的本地 debian 服务器和 ec2 debian 实例（在一个测试 aws 帐户中）编译了 kinesis-agent。在一个单独的 aws 帐户中，我创建了 Kinesis Stream 并将其指向 AWS elasticsearch 域（监控 AWS 帐户）。
我在 AWS 监控账户中创建了一个用户（kinesistestagent），该账户可以访问 kinesis firehose 流并添加了正确的 STS 角色（我被困在这几天，因为它甚至不会进行身份验证），以便 kinesis 代理可以进行身份验证使用 firehose 流并发送其数据。
我的 kinesis 的 agent.json 文件看起来像这样。我试图将其剥离，以尝试将数据放入 firehose 和 elasticsearch ...
{
"checkpointFile": "/opt/aws-kinesis-agent/run/checkpoints",
"cloudwatch.endpoint": "https://monitoring.eu-west-2.amazonaws.com",
"cloudwatch.emitMetrics": "false",
"firehose.endpoint": "https://firehose.eu-west-2.amazonaws.com",
"assumeRoleExternalId" :"arn:aws:firehose:eu-west-2:117215238277453:deliverystream/TEST-Firehose-EKK",
"awsAccessKeyId": "AKIRADXQWUX45KCM2IKB",
"awsSecretAccessKey": "bpq7KdidkfkeodmadeuppaccessZg4BL",
"flows": [
{
"filePattern": "/data/log/server.log",
"initialPosition": "END_OF_FILE",
"deliveryStream": "TEST-Firehose-EKK"
}
]
}
由于我的 linux 实例不是 Amazon AMI，我已明确使用“awsAccessKeyId”和“awsSecretAccessKey”的授权值。
我从日志中得到的确切错误是身份验证有效但请求中的安全令牌无效？
2020-03-26 23:00:00.088+0000 (sender-2318) com.amazon.kinesis.streaming.agent.tailing.AsyncPublisher [ERROR] AsyncPublisher[fh:TEST-Firehose-EKK:/data/log/server.log]:RecordBuffer(id=2,records=179,bytes=36161) Retriable send error (com.amazonaws.services.kinesisfirehose.model.AmazonKinesisFirehoseException: The security token included in the request is invalid. (Service: AmazonKinesisFirehose; Status Code: 400; Error Code: UnrecognizedClientException; Request ID: db4f7c53-e4a0-17e0-8db8-4a638f74b5fb)). Will retry.
运动日志中的整个事务看起来像这样。
2020-03-26 22:59:59.574+0000 (FileTailer[fh:TEST-Firehose-EKK:/data/log/server.log].MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.tailing.FileTailer [INFO] FileTailer[fh:TEST-Firehose-EKK:/data/log/server.log]: Tailer Progress: Tailer has parsed 179 records (997399 bytes), transformed 0 records, skipped 0 records, and has successfully sent 0 records to destination.
2020-03-26 22:59:59.581+0000 (Agent.MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.Agent [INFO] Agent: Progress: 179 records parsed (997399 bytes), and 0 records sent successfully to destinations. Uptime: 23790134ms
2020-03-26 23:00:00.058+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] connecting to firehose.eu-west-2.amazonaws.com/52.94.49.83:443
2020-03-26 23:00:00.059+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] Connecting socket to firehose.eu-west-2.amazonaws.com/52.94.49.83:443 with timeout 10000
2020-03-26 23:00:00.060+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] Enabled protocols: [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1]
2020-03-26 23:00:00.060+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] Enabled cipher suites:[TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
2020-03-26 23:00:00.061+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] socket.getSupportedProtocols(): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1, SSLv3, SSLv2Hello], socket.getEnabledProtocols(): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1]
2020-03-26 23:00:00.061+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] TLS protocol enabled for SSL handshake: [TLSv1.2, TLSv1.1, TLSv1, TLSv1.3]
2020-03-26 23:00:00.061+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] Starting handshake
2020-03-26 23:00:00.070+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] Secure session established
2020-03-26 23:00:00.070+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] negotiated protocol: TLSv1.2
2020-03-26 23:00:00.070+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
2020-03-26 23:00:00.070+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] peer principal: CN=firehose.eu-west-2.amazonaws.com
2020-03-26 23:00:00.070+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] peer alternative names: [*.firehose.eu-west-2.vpce.amazonaws.com, firehose.eu-west-2.amazonaws.com]
2020-03-26 23:00:00.070+0000 (sender-2318) com.amazonaws.http.conn.ssl.SdkTLSSocketFactory [DEBUG] issuer principal: CN=Amazon, OU=Server CA 1B, O=Amazon, C=US
2020-03-26 23:00:00.088+0000 (sender-2318) com.amazon.kinesis.streaming.agent.tailing.AsyncPublisher [ERROR] AsyncPublisher[fh:TEST-Firehose-fEKK:/data/log/server.log]:RecordBuffer(id=2,records=179,bytes=36161) Retriable send error (com.amazonaws.services.kinesisfirehose.model.AmazonKinesisFirehoseException: The security token included in the request is invalid. (Service: AmazonKinesisFirehose; Status Code: 400; Error Code: UnrecognizedClientException; Request ID: db4f7c53-e4a0-17e0-8db8-4a638f74b5fb)). Will retry.
以前有没有人遇到过这个错误，或者有没有人设法让 AWS kinesis 代理在 prem 服务器上工作？
非常感谢您花时间阅读我的问题，任何帮助或建议将不胜感激。
干杯