0

我们观察到 StackDriver 报告的多个并发查询与我们在“data_access”日志中看到的内容之间的不一致，以及一些报告我的“查询”指标的数字。

从下午 1 点 16 分左右附上的图片可以看出，并发查询激增至 87 个（不确定这是否可能，因为并发查询配额为 50），如果我们从“data_access”中计算 jobCompletedEvent使用 createTimeTIMESTAMP("2017-01-19 13:16:48","America/Los_Angeles") 记录我们只能看到 24。

这个例子不是孤立的，我们有很多这样的事情发生。

在此处输入图像描述

SELECT
  count(*)
FROM (
  SELECT
    createTime,
    startTime,
    endTime
  FROM (
    SELECT
      protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.createTime AS createTime,
      protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.startTime AS startTime,
      protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.endTime AS endTime
    FROM
      `catalog.cloudaudit_googleapis_com_data_access_201701*`))
WHERE
  createTime<TIMESTAMP("2017-01-19 13:16:48","America/Los_Angeles")
  AND endTime>TIMESTAMP("2017-01-19 13:16:48","America/Los_Angeles")

进一步调查它变得有点奇怪。如果我查看 jobGetQueryResultsResponse 统计信息，这是查询

    SELECT
    insertId,
    jobId,
    createTime,
    startTime,
    endTime
FROM (
  SELECT
    insertId,
    jobId,
    createTime,
    startTime,
    endTime
  FROM (
    SELECT
      insertId,
      protoPayload.serviceData.jobGetQueryResultsResponse.job.jobName.jobId AS jobId,
      protoPayload.serviceData.jobGetQueryResultsResponse.job.jobStatistics.createTime AS createTime,
      protoPayload.serviceData.jobGetQueryResultsResponse.job.jobStatistics.startTime AS startTime,
      protoPayload.serviceData.jobGetQueryResultsResponse.job.jobStatistics.endTime AS endTime,
      protoPayload.status.message error
    FROM
      `catalog.cloudaudit_googleapis_com_data_access_201701*`))
WHERE
  createTime<=TIMESTAMP("2017-01-19 13:16:48")
  AND endTime>=TIMESTAMP("2017-01-19 13:16:48")

它现在返回 321 条记录（下面的片段），其中有许多具有唯一 insertID 但相同 jobId 的重复记录

116 5467305DA3861.A4954C2.780CB7F5  job__oFOLJVnNr8Xj7zFnNrgqww6_Ww 2017-01-19 13:16:35 UTC 2017-01-19 13:16:36 UTC 2017-01-19 13:16:53 UTC  
117 54672C24D1181.A036F52.D205C885  bqjob_r31c3135d053792f3_00000159b6d384e3_1  2017-01-19 13:03:52 UTC 2017-01-19 13:03:55 UTC 2017-01-19 13:46:09 UTC  
118 54672C333EB29.A498C8D.1E00E7D1  job__oFOLJVnNr8Xj7zFnNrgqww6_Ww 2017-01-19 13:16:35 UTC 2017-01-19 13:16:36 UTC 2017-01-19 13:16:53 UTC  
119 5467285CC92D1.A496505.560166F3  job__oFOLJVnNr8Xj7zFnNrgqww6_Ww 2017-01-19 13:16:35 UTC 2017-01-19 13:16:36 UTC 2017-01-19 13:16:53 UTC  
120 5467355C301E1.A49D803.540CE70C  job__oFOLJVnNr8Xj7zFnNrgqww6_Ww 2017-01-19 13:16:35 UTC 2017-01-19 13:16:36 UTC 2017-01-19 13:16:53 UTC  
121 54673441FDAF9.A49C44D.82036D2F  job__oFOLJVnNr8Xj7zFnNrgqww6_Ww 2017-01-19 13:16:35 UTC 2017-01-19 13:16:36 UTC 2017-01-19 13:16:53 UTC

这是否会引发不一致以及配额的含义是什么。您能否说明如何计算“查询计数”指标。BigQuery StackDriver 日志的可靠性如何，以及从“data_access”日志中获取准确计数的最佳方法是什么。

4

0 回答 0