In a microservices-based environment, it is crucial to have a robust and high-performance solution that can track, retrieve and analyse all the logs in a simplified way.
The FINSEC components are compliant with the twelve factor advice, so the EFK (ElasticsearchFluentKibana) stack has been chosen as a centralized logging solution.
Elasticsearch is a real-time, distributed, and scalable search engine that allows for full-text and structured search. It is commonly used to index and search through large volumes of log data.
Elasticsearch is commonly deployed alongside Kibana, a powerful data visualization frontend and dashboard for Elasticsearch. Kibana allows you to explore your Elasticsearch log data through a web interface and build dashboards.
In Kubernetes, containerized applications that log to stdout and stderr have their log streams captured and redirected to JSON files on the nodes. The Fluentd is a popular open-source Data Collector that tails, transforms, and ships these log files to the Elasticsearch backend.
The EFK stack is shown in the figure below