0

使用基于此的 docker 映像，我在 GKE 上创建了一个 pod。代理将通过 TCP 侦听 fluentd 事件（我的其他应用程序 pod 将发送事件），然后将这些日志转发到 Google Cloud Logging。因为这些事件缺少一些元数据。如何添加这些缺失的信息？

(symfony 应用)--[独白]-->(google-fluentd-agent)-->(Cloud Logging)

google-fluentd.conf：

<match fluent.**>
  type null
</match>

# TCP Connections for fluentd aware applications.
<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match **>
  type google_cloud
  # Set the chunk limit conservatively to avoid exceeding the GCL limit
  # of 10MiB per write request.
  buffer_chunk_limit 2M
  # Cap the combined memory usage of this buffer and the one below to
  # 2MiB/chunk * (24 + 8) chunks = 64 MiB
  buffer_queue_limit 24
  # Never wait more than 5 seconds before flushing logs in the non-error case.
  flush_interval 5s
  # Never wait longer than 30 seconds between retries.
  max_retry_wait 30
  # Disable the limit on the number of retries (retry forever).
  disable_retry_limit
</match>

缺少数据的 Google 日志记录事件：

{
  metadata: {
    projectId: "my-project"      
    serviceName: "container.googleapis.com"      
    zone: "us-central1-a"      
    labels: {
      container.googleapis.com/cluster_name: "app-staging-a"        
      compute.googleapis.com/resource_type: "instance"        
      compute.googleapis.com/resource_name: "cluster-fluentd-1dom0"        
      container.googleapis.com/instance_id: "296757089355968949"        
      container.googleapis.com/pod_name: ""        
      compute.googleapis.com/resource_id: "296757089355968949"        
      container.googleapis.com/namespace_name: ""        
      container.googleapis.com/container_name: ""        
    }
    timestamp: "2016-05-16T00:25:37.000Z"      
    projectNumber: "10568438715"      
  }
  insertId: "94dadf6548d"    
  log: "symfony.php"    
  structPayload: {
    context: {
      stack: [33]       
      file: "classes.php"        
      type: 16384        
      line: 4156        
      level: 28928        
    }
    level: "INFO"      
    message: "Using an instance of "This_Function_Method" for function "some_stuff" is deprecated."      
  }
}
4

1 回答 1

1

google_cloud 输出插件尝试从进入它的日志流的名称中解析命名空间、pod 和容器名称。在正常设置中，这是有效的，因为来自每个容器的 stdout/stderr 的磁盘上的文件都是这样命名的。为了获得类似的解析行为，您必须类似地制作发送给 fluentd 的日志流名称（或在自定义 fluentd 插件中实现您自己的逻辑）。

于 2016-06-16T21:34:51.757 回答