我已经实现了一个ELK 堆栈示例,以使用minikube集中记录我的 k8s 集群。我还部署了一个 cron 作业,以每小时删除 1 天的日志作为示例标题。
因为是在每个小时的开始安排的(schedule: "0 * * * *"),所以我注意到在第一个应用程序时,当新的小时开始时,elasticsearch的 /data文件夹瞬间从14M到2.1M , 表明 cronjob 刚刚删除了旧文件日志。
事实是,几秒钟后,/data文件夹从2.1M变为2.7M,然后又回到2.4M,就像日志每 15-20 秒被删除一次,而不是在每小时开始时。如果我继续监视/data文件夹,它会不断上升和下降。
无论如何/logs文件夹是 13M 并且在某些日子里快速增长,但是 cron 作业根本没有删除它的文件。这也正常吗?
这是我在默认命名空间中部署的基本日志记录 pod:
https://k8s.io/examples/debug/counter-pod.yaml
这是部署在日志命名空间中的弹性搜索部署:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: elasticsearch-pvc # name of PVC essential for identifying the storage data
labels:
k8s-app: elasticsearch-logging
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: elasticsearch-logging
namespace: logging
labels:
k8s-app: elasticsearch-logging
spec:
replicas: 1
selector:
matchLabels:
k8s-app: elasticsearch-logging
template:
metadata:
labels:
k8s-app: elasticsearch-logging
spec:
containers:
- name: elasticsearch-logging
image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.3.0
resources:
limits:
cpu: 500m
memory: 2400Mi
requests:
cpu: 100m
memory: 2350Mi
ports:
- containerPort: 9200
name: db
protocol: TCP
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- name: elasticsearch-logging
mountPath: /data
env:
- name: "NAMESPACE"
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MINIMUM_MASTER_NODES
value: "1"
initContainers:
- image: registry.hub.docker.com/library/alpine:3.6
command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"]
name: elasticsearch-logging-init
securityContext:
privileged: true
volumes:
- name: elasticsearch-logging
persistentVolumeClaim:
claimName: elasticsearch-pvc
---
apiVersion: v1
kind: Service
metadata:
name: elasticsearch-logging
namespace: logging
labels:
k8s-app: elasticsearch-logging
spec:
ports:
- port: 9200
protocol: TCP
targetPort: db
selector:
k8s-app: elasticsearch-logging
这是部署在日志命名空间中的cronjob:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: elasticsearch-curator
namespace: logging
labels:
k8s-app: elasticsearch-logging
spec:
schedule: "0 * * * *"
jobTemplate:
spec:
template:
metadata:
name: elasticsearch-curator
labels:
k8s-app: elasticsearch-logging
spec:
restartPolicy: "Never"
containers:
- name: ingestor
image: python:3.6-alpine
args: ["sh", "-c", "pip install elasticsearch-curator && curator_cli --host elasticsearch-logging delete_indices --filter_list '[{\"filtertype\":\"age\",\"source\":\"creation_date\",\"direction\":\"older\",\"unit\":\"days\",\"unit_count\":1},{\"filtertype\":\"pattern\",\"kind\":\"prefix\",\"value\":\"logstash\"}]' || true"]
backoffLimit: 1
我的 cron 工作有什么问题?