ELK

Mô hình ELK

FileBeat → Kafka (Optional) → Logstash → ElasticSearch → Kibana

1. Config in Logstash

• Using https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns to make GROK
• Checking online using : https://grokdebug.herokuapp.com/
• Config in :

+) /etc/logstash/pipelines/*.yml

+) /etc/logstash/pipelines.yml

• Example :

+) input : Kafka or FileBeat

+) filter {
if [log_type] == “accesslog” {
grok { match => { “message” => “%{DATA:content} %{

NUMBER:opTime:float}ms|%{DATA:content}$”}}
}

mutate ( convert “opTime” → “float”)
}

+) Output : ElasticSearch

Add Index : In Kibana > Setting > Add index > search name of log file name in config.

Delete Index : In Kibana > Setting > Choosing Index > Delete

• Note : Adding new field from message to DB, we need to re-insert new index and delete old index.
• If field is number we can convert to number using mutate ( convert “opTime” → “float”).
• If the field is String already, we cannot change to number