EFK Stack이란 ElasticSearch, Fluentd, Kibana라는 오픈소스의 조합으로, 방대한 양의 데이터를 신속하고 실시간으로 수집/저장/분석/시각화 할 수 있는 솔루션입니다. 특히 컨테이너 환경에서 로그 수집을 위해 주로 사용되는 기술 스택입니다. Elasitc Stack에 대한 소개와 EFK Stack 설치 방법에 대해 설명합니다.

14. 제품 구성 단일 제품 다양한 오픈소스로 구성
데이터 수집 Splunk Universal Forwarder LogStash, Fluentd, FileBeat
검색 Splunk Search Head Kibana
검색 언어
SPL
(Search Processing Language)
KQL (Kibana Query Language),
Query DSL, Json
인덱싱 기술 C++ Apache Lucene

17. (k8s-venv) root@k8s-master01:~# helm repo add elastic https://helm.elastic.co
"elastic" has been added to your repositories
(k8s-venv) root@k8s-master01:~# tee elastic-values.yaml << EOF
# Permit co-located instances for solitary minikube virtual machines.
antiAffinity: "soft"
# Shrink default JVM heap.
esJavaOpts: "-Xmx128m -Xms128m"
# Allocate smaller chunks of memory per pod.
resources:
requests:
cpu: "100m"
memory: "512M"
limits:
cpu: "1000m"
memory: "512M"
persistence:
enabled: false
EOF

18. (k8s-venv) root@k8s-master01:~# helm install elasticsearch elastic/elasticsearch --version=7.9.0 --namespace=logs -f elastic-values.yaml
NAME: elasticsearch
LAST DEPLOYED: Thu May 6 01:22:16 2021
NAMESPACE: logs
STATUS: deployed
REVISION: 1
NOTES:
1. Watch all cluster members come up.
$ kubectl get pods --namespace=logs -l app=elasticsearch-master -w
2. Test cluster health using Helm test.
$ helm test elasticsearch --cleanup

19. (k8s-venv) root@k8s-master01:~# helm repo add fluent https://fluent.github.io/helm-charts
"fluent" has been added to your repositories
(k8s-venv) root@k8s-master01:~# helm install fluent-bit fluent/fluent-bit --version 0.6.3 --namespace=logs
NAME: fluent-bit
LAST DEPLOYED: Thu May 6 01:24:24 2021
NAMESPACE: logs
STATUS: deployed
REVISION: 1
NOTES:
Get Fluent Bit build information by running these commands:
export POD_NAME=$(kubectl get pods --namespace logs -l "app.kubernetes.io/name=fluent-bit,app.kubernetes.io/instance=fluent-bit" -o
jsonpath="{.items[0].metadata.name}")
echo "curl http://127.0.0.1:2020 for Fluent Bit build information"
kubectl --namespace logs port-forward $POD_NAME 2020:2020

20. (k8s-venv) root@k8s-master01:~# helm install kibana elastic/kibana --version=7.9.0 --namespace=logs --set service.type=NodePort --set
service.nodePort=31000
NAME: kibana
LAST DEPLOYED: Thu May 6 01:25:01 2021
NAMESPACE: logs
STATUS: deployed
REVISION: 1
TEST SUITE: None

21. (k8s-venv) root@k8s-master01:~# kubectl get deployments,pods,services --namespace=logs
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kibana-kibana 1/1 1 1 7m11s
NAME READY STATUS RESTARTS AGE
pod/elasticsearch-master-0 1/1 Running 0 9m56s
pod/elasticsearch-master-1 1/1 Running 0 9m56s
pod/elasticsearch-master-2 1/1 Running 0 9m56s
pod/fluent-bit-wj5gt 1/1 Running 0 7m48s
pod/fluent-bit-xp7vn 1/1 Running 0 7m48s
pod/fluent-bit-xxs4z 1/1 Running 0 7m48s
pod/kibana-kibana-6d874c5f46-6d5kf 1/1 Running 0 7m11s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-master ClusterIP 10.233.63.66 <none> 9200/TCP,9300/TCP 9m56s
service/elasticsearch-master-headless ClusterIP None <none> 9200/TCP,9300/TCP 9m56s
service/fluent-bit ClusterIP 10.233.19.236 <none> 2020/TCP 7m48s
service/kibana-kibana NodePort 10.233.42.99 <none> 5601:31000/TCP 7m11s

22. (k8s-venv) root@k8s-master01:~# kubectl run random-logger --image=chentex/random-logger
pod/random-logger created
(k8s-venv) root@k8s-master01:~# kubectl logs pod/random-logger
2021-07-08T01:32:45+0000 ERROR An error is usually an exception that has been caught and not handled.
2021-07-08T01:32:47+0000 DEBUG This is a debug log that shows a log that can be ignored.

62. T. 02-516-0711 E. sales@osci.kr
서울시강남구테헤란로83길32,5층(삼성동,나라키움삼성동A빌딩)
THANK YOU