1
  • 您使用的是什么 Grafana 版本和什么操作系统？

Promtail:latest & Loki:2.2.0, Kubernetes (GitVersion:"v1.18.8") 和 Helm (Version:"v3.6.2")

  • 你想达到什么目的？

可以抓取我的活动目标并将它们推送给 Loki。

  • 发生了什么？

所有目标都标记为“未准备好”。如果我要去 /targets 页面，我所有的 active_targets 都被标记为“false”。在 Loki 我没有日志。Promtail 中的 /var/logs/ 文件夹也是空的。我从 promtail pod 收到的日志是这样的：

level=info ts=2021-08-06T05:30:09.076046169Z caller=filetargetmanager.go:254 msg="Adding target" key="{app=\"<app_name>\", container=\"<container_name>", job=\"<job_name>", namespace=\"<namesapce_name>", node_name=\"<nodeName_name>", pod=\"<pod_name>"}"

level=info ts=2021-08-06T05:30:09.076046169Z caller=filetargetmanager.go:254 msg="Removing target" key="{app=\"<app_name>\", container=\"<container_name>", job=\"<job_name>", namespace=\"<namesapce_name>", node_name=\"<nodeName_name>", pod=\"<pod_name>"}"

level=info ts=2021-08-06T05:30:14.095615824Z caller=filetarget.go:150 msg="filetarget: watcher closed, tailer stopped, positions saved" path=/var/log/pods/*<some_path>/<container_name>/*.log

提示/指标：

 HELP promtail_targets_failed_total Number of failed targets.
 TYPE promtail_targets_failed_total counter
promtail_targets_failed_total{reason="empty_labels"} 2280
promtail_targets_failed_total{reason="exists"} 470
 HELP request_duration_seconds Time (in seconds) spent serving HTTP requests.
  • 你期望会发生什么？

我的目标被刮掉并推给了洛基。

  • 您可以复制/粘贴您遇到问题的配置吗？

  file: |
    server:
      log_level: {{ .Values.config.logLevel }}
      http_listen_port: {{ .Values.config.serverPort }}
      health_check_target: false
    client:
      url: {{ tpl .Values.config.lokiAddress . }}
      {{- tpl .Values.config.snippets.extraClientConfigs . | nindent 2 }}
    positions:
      filename: /run/promtail/positions.yaml
    scrape_configs:
      {{- tpl .Values.config.snippets.scrapeConfigs . | nindent 2 }}
      {{- tpl .Values.config.snippets.extraScrapeConfigs . | nindent 2 }}

scrapeConfigs: |
      # See also https://github.com/grafana/loki/blob/master/production/ksonnet/promtail/scrape_config.libsonnet for reference
      # Pods with a label 'app.kubernetes.io/name'
      - job_name: kubernetes-pods-app-kubernetes-io-name
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
            target_label: app
          - action: drop
            regex: ''
            source_labels:
              - app
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_component
            target_label: component
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-app-kubernetes-io-name
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # Pods with a label 'app'
      - job_name: kubernetes-pods-app
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop pods with label 'app.kubernetes.io/name'. They are already considered above
          - action: drop
            regex: .+
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_app
            target_label: app
          - action: drop
            regex: ''
            source_labels:
              - app
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_component
            target_label: component
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-app
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # Pods with direct controllers, such as StatefulSet
      - job_name: kubernetes-pods-direct-controllers
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop pods with label 'app.kubernetes.io/name' or 'app'. They are already considered above
          - action: drop
            regex: .+
            separator: ''
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
              - __meta_kubernetes_pod_label_app
          - action: drop
            regex: '[0-9a-z-.]+-[0-9a-f]{8,10}'
            source_labels:
              - __meta_kubernetes_pod_controller_name
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_controller_name
            target_label: app
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-direct-controllers
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # Pods with indirect controllers, such as Deployment
      - job_name: kubernetes-pods-indirect-controller
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop pods with label 'app.kubernetes.io/name' or 'app'. They are already considered above
          - action: drop
            regex: .+
            separator: ''
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
              - __meta_kubernetes_pod_label_app
          - action: keep
            regex: '[0-9a-z-.]+-[0-9a-f]{8,10}'
            source_labels:
              - __meta_kubernetes_pod_controller_name
          - action: replace
            regex: '([0-9a-z-.]+)-[0-9a-f]{8,10}'
            source_labels:
              - __meta_kubernetes_pod_controller_name
            target_label: app
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-indirect-controller
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # All remaining pods not yet covered
      - job_name: kubernetes-other
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop what has already been covered
          - action: drop
            regex: .+
            separator: ''
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
              - __meta_kubernetes_pod_label_app
          - action: drop
            regex: .+
            source_labels:
              - __meta_kubernetes_pod_controller_name
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_name
            target_label: app
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_component
            target_label: component
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-other
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}

  • 您是否在 Grafana UI 或相关日志中收到任何错误？如果是这样，请准确告诉我们它们是什么。

  • 您是否遵循任何在线说明？如果有，网址是什么？

我主要遵循官方回购的说明。

https://github.com/grafana/helm-charts/tree/main/charts

我创建了以下资源：

对于 Loki：我有一个 Secret（带有配置）、Service 和 Statefulset。

Promtail：我有一个 DaemonSet、Secret、强大的 ClusterRole 和 CluserRoleBinding。

4

1 回答 1

0

好吧，我不确定这会对您有所帮助，但我有相同的添加目标并立即删除它们的 promtail 日志。

我的配置有点不同，我在本地运行 promtail 并抓取一些文件。问题是 promtail 没有读取这些文件的访问权限。

因此，我建议仔细检查您的 promtail pod 是否具有对您尝试抓取的文件的读取权限，然后重新启动服务。

我也没有在 grafana 或 Loki 中看到任何错误，因为日志从未推送到 Loki。

于 2021-09-10T01:19:22.917 回答