**The *ELK Stack* is an open source-based log management and analytics platformfor integrated log collection, storage, analysis, and visualization.
"ELK" stands for each of the following three core components

1. Elasticsearch

  • role: log and data storage, search, and analytics engine

  • description:

    • A decentralized search engine that stores data in the form of JSON documents

    • Based on Apache Lucene

    • Providesfull-text search, aggregation, and real-time analytics

    • quickly query and analyze large amounts of log or event data

  • example:

    GET /logs/_search { " query": { "match": { "level": "error" } } }

2. Logstash

  • role: log and event collection, transformation, and delivery pipeline

  • description:

    • collect data from various input sources (files, DB, Beats, etc.) (Input)

    • process and transform data withfilters (JSON format, field parsing, etc.)

    • send to Elasticsearch or another storage asOutput

  • example:

    input { file { path => "/var/log/*.log" } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } }output { elasticsearch { hosts => ["localhost:9200"] } }

3. kibana

  • role: Provide visualizations and dashboards

  • description:

    • Search (Query ) and visualize (Charts, Graphs) data stored in Elasticsearch

    • create real-time dashboards

    • filter, analyze, and alert (Elastic Stack Alerting)

  • example:

    • "Graph of error log occurrence frequency over the last 24 hours"

    • "Top 10 accesses by IP" dashboard configuration

(Supplemental) Beats

  • not originally part of the ELK stack, but included as it was extended to the Elastic Stack.

  • lightweight data collectorsthat send logs/metrics from your server or application directly to Logstash or Elasticsearch.

  • key modules:

    • Filebeat: Collect log files

    • Metricbeat: Collect system metrics

    • Packetbeat: Analyzes network packets

    • Heartbeat: Service availability monitoring

ELK Stack data flow structure

[Application Logs] ↓ Beats/Filebeat ↓ Logstash (ingestion-refinement) ↓ Elasticsearch (storage-search) ↓ Kibana (visualization)

key Use Cases

category description server log analysis Analyze and monitor Nginx, Apache, and Spring Boot logs security monitoring (SIEM) attack detection, intrusion attempt analysis operational monitoring (with APM) Analyze application performance in conjunction with Elastic APM business data analytics Visualize API call trends, user behavior logs, etc

organize

components roles 备注 Elasticsearch data storage and retrieval Lucene-based Logstash data ingestion and transformation support for a variety of plugins Kibana data visualization Dashboard/UI Beats lightweight log collection Agent roles