Centralized logs collection and analysis. Elasticsearch, Logstash and Kibana. Introduction.
28 Mar 2015
One day information security specialist begin to implement solution for centralized logs collection and analysis. Let's review how to perform it using open source software ELK Stack. This solution is not SIEM but it can be integrated with correlation engine. As a result it's possible to receive the tool for security incident response.
ELK Stack consists of 3 components (Elasticsearch, Logstash and Kibana) and allows to create failover and distributed cluster for data.
Elasticsearch is main component of ELK Stack which is used for cluster management and data storing. Elasticsearch allows to create several type of nodes:
1. Master node - cluster management.
2. Data node - data storing.
3. Client node - interaction with cluster.
Logstash is used for logs collection from different sources, its filtering and transfering to cluster. Logstash can act as a client node.
Kibana is web-interface for data analysis. It's usually located on the client node.
In the next article we are going to review base cluster configuration.