ELK is short of Elasticsearch, Logstash and Kibana.
Elasticsearch
a real-time distributed searching and analyzing engine, which can be used to full-text search, structured search and analysis. Full-text search is based on Apache Lucene which is written by Java.
Features of Elasticsearch
- Real-time analysis
- Distributed file storing in real-time, taking each field as index
- Document-oriented, all objects are documents
- High availability, easy to scale, support cluster, sharding and replicas
- Friendly APIs, Supports JSON
- Zero-setting, automatically sharding
- RESTful style
- Based on Lucene and provides storage function
Clusters
Sharding and replicas
Logstash
a data collecting, filter and modifying engine in real-time, which is written by JRuby.
Features of Logstash
- Almost visit any data
- Binding with various external applications
- Supports resilient scale
Main components
- Shipper: send log data
- Broker: collect data, built-in Redis
- Indexer: data writes
Kibana
a open-source Apache protocol, which is written by JavaScript, to provide visualization function for Elasticsearch and Web platform.
Present data of other two components by HTTP protocols to users.
When to use?
use grep, awk as commands to analyze log directly, but it doesn’t convinent in large scale implement. Because we need to write a script for each machine and combine all results.(Low efficiency) Thus, we use ELK search info in one machine, and access by web application.
Advantages of ELK
- Developers can view log from web in detail
- help to collect log from each system, which distributes in a wide range