prometheus - 使用 Kusto 查询语言查询 Prometheus 数据

Question

我正在努力尝试使用存储在 Azure Monitor 中的 Prometheus 数据设置一些 Grafana 仪表板。

所以我有：

在 AKS 上运行的 ASP.NET Core 应用程序，通过 /metrics 端点导出 Prometheus 数据
我已将 Azure Monitor 配置为抓取 /metrics 端点
现在我正在尝试在 Grafana 中显示几个图表

但我正在努力使用 Kusto 查询语言。

这是 /metrics 端点的示例输出

# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1613573002.92
# HELP http_requests_in_progress The number of requests currently in progress in the ASP.NET Core pipeline. One series without controller/action label values counts all in-progress requests, with separate series existing for each controller-action pair.
# TYPE http_requests_in_progress gauge
http_requests_in_progress{method="GET",controller="",action=""} 0
http_requests_in_progress{method="GET",controller="MyController",action="GetSecrets"} 0
# HELP dotnet_collection_count_total GC collection count
# TYPE dotnet_collection_count_total counter
dotnet_collection_count_total{generation="0"} 66
dotnet_collection_count_total{generation="2"} 1
dotnet_collection_count_total{generation="1"} 7
# HELP process_private_memory_bytes Process private memory size
# TYPE process_private_memory_bytes gauge
process_private_memory_bytes 182657024
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 151.03
# HELP http_requests_received_total Provides the count of HTTP requests that have been processed by the ASP.NET Core pipeline.
# TYPE http_requests_received_total counter
http_requests_received_total{code="404",method="GET",controller="",action=""} 1
http_requests_received_total{code="200",method="GET",controller="MyController",action="GetSecrets"} 4
http_requests_received_total{code="200",method="GET",controller="",action=""} 23
http_requests_received_total{code="500",method="GET",controller="MyController",action="GetSecrets"} 2
# HELP process_num_threads Total number of threads
# TYPE process_num_threads gauge
process_num_threads 14
# HELP dotnet_total_memory_bytes Total known allocated memory
# TYPE dotnet_total_memory_bytes gauge
dotnet_total_memory_bytes 5438264
# HELP process_working_set_bytes Process working set
# TYPE process_working_set_bytes gauge
process_working_set_bytes 111087616
# HELP http_request_duration_seconds The duration of HTTP requests processed by an ASP.NET Core application.
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_sum{code="404",method="GET",controller="",action=""} 0.0047621
http_request_duration_seconds_count{code="404",method="GET",controller="",action=""} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.001"} 0
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.002"} 0
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.004"} 0
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.008"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.016"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.032"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.064"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.128"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.256"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="0.512"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="1.024"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="2.048"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="4.096"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="8.192"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="16.384"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="32.768"} 1
http_request_duration_seconds_bucket{code="404",method="GET",controller="",action="",le="+Inf"} 1
http_request_duration_seconds_sum{code="200",method="GET",controller="MyController",action="GetSecrets"} 34.2080351
http_request_duration_seconds_count{code="200",method="GET",controller="MyController",action="GetSecrets"} 4
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.001"} 0
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.002"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.004"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.008"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.016"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.032"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.064"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.128"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.256"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="0.512"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="1.024"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="2.048"} 1
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="4.096"} 3
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="8.192"} 3
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="16.384"} 3
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="32.768"} 4
http_request_duration_seconds_bucket{code="200",method="GET",controller="MyController",action="GetSecrets",le="+Inf"} 4
http_request_duration_seconds_sum{code="200",method="GET",controller="",action=""} 0.0218133
http_request_duration_seconds_count{code="200",method="GET",controller="",action=""} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.001"} 21
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.002"} 21
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.004"} 21
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.008"} 22
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.016"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.032"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.064"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.128"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.256"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="0.512"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="1.024"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="2.048"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="4.096"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="8.192"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="16.384"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="32.768"} 23
http_request_duration_seconds_bucket{code="200",method="GET",controller="",action="",le="+Inf"} 23
http_request_duration_seconds_sum{code="500",method="GET",controller="MyController",action="GetSecrets"} 620.8115536
http_request_duration_seconds_count{code="500",method="GET",controller="MyController",action="GetSecrets"} 2
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.001"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.002"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.004"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.008"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.016"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.032"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.064"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.128"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.256"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="0.512"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="1.024"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="2.048"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="4.096"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="8.192"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="16.384"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="32.768"} 0
http_request_duration_seconds_bucket{code="500",method="GET",controller="MyController",action="GetSecrets",le="+Inf"} 2
# HELP process_open_handles Number of open handles
# TYPE process_open_handles gauge
process_open_handles 196
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 3918102528

最终在 Azure Monitor 中出现的数据（以及其他字段，如名称、Val、标签等）。例如

我正在尝试设置几个不同的图表：

每个间隔的请求总数。例如，过去 24 小时内每 5 分钟的平均请求数
每个时间间隔的 2xx 请求总数。
内存/CPU 相同

我想这会很简单，但我就是做错了......

我有这样的事情：

InsightsMetrics 
| where $__timeFilter(TimeGenerated)
| where Namespace == "prometheus"
| where Name == "http_requests_received_total"
| extend dimensions=parse_json(Tags)
| where dimensions.app == "myappname"
| summarize count() by Val, bin(TimeGenerated, $__interval) // Don't know what to put here...
| order by TimeGenerated asc

prometheus - 使用 Kusto 查询语言查询 Prometheus 数据

0 回答 0

Related

Reference