Centralized Logging System
Centralized Logging System
This article details the configuration of a centralized logging system for our MediaWiki installation. Centralized logging is crucial for troubleshooting, security auditing, and performance monitoring. It allows us to collect logs from all server components in a single location, making analysis significantly easier than examining individual server logs. This guide assumes a basic understanding of Linux server administration and MediaWiki configuration.
System Overview
Our centralized logging system utilizes the ELK stack: Elasticsearch, Logstash, and Kibana.
- Elasticsearch: The search and analytics engine that stores the logs.
- Logstash: The data processing pipeline that collects, transforms, and forwards logs to Elasticsearch.
- Kibana: The visualization tool for exploring and analyzing the logs in Elasticsearch.
This setup provides a scalable and robust solution for managing large volumes of log data. We are also utilizing rsyslog on the MediaWiki servers to forward logs to Logstash.
Server Specifications
The following table details the hardware specifications for each component of the logging system.
| Component | CPU | Memory | Storage | Operating System |
|---|---|---|---|---|
| Elasticsearch Server | 8 Cores | 32 GB RAM | 500 GB SSD | Ubuntu 22.04 LTS |
| Logstash Server | 4 Cores | 16 GB RAM | 250 GB SSD | Ubuntu 22.04 LTS |
| Kibana Server | 4 Cores | 8 GB RAM | 100 GB SSD | Ubuntu 22.04 LTS |
These specifications are a baseline and may need to be adjusted based on the volume of logs generated by the MediaWiki installation and other services. Consider performance monitoring to determine if scaling is necessary.
Software Installation and Configuration
1. Elasticsearch Installation:
Follow the official Elasticsearch documentation for installation: [1](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html). Ensure the Elasticsearch service is running and accessible. Pay close attention to the Java Virtual Machine (JVM) settings for optimal performance.
2. Logstash Installation:
Follow the official Logstash documentation for installation: [2](https://www.elastic.co/guide/en/logstash/current/installing-logstash.html).
3. Kibana Installation:
Follow the official Kibana documentation for installation: [3](https://www.elastic.co/guide/en/kibana/current/installing-kibana.html).
4. rsyslog Configuration (MediaWiki Servers):
Edit the `/etc/rsyslog.conf` file on each MediaWiki server. Add the following lines to forward logs to the Logstash server:
```
- .* @logstash_server_ip:514
```
Replace `logstash_server_ip` with the actual IP address of the Logstash server. Restart the rsyslog service: `sudo systemctl restart rsyslog`. Ensure the firewall allows UDP traffic on port 514.
5. Logstash Configuration:
Create a Logstash configuration file (e.g., `/etc/logstash/conf.d/mediawiki.conf`) to define the input, filter, and output stages. Here's a basic example:
``` input {
udp {
port => 514
}
}
filter {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{GREEDYDATA:message}" }
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
output {
elasticsearch {
hosts => ["elasticsearch_server_ip:9200"]
index => "mediawiki-%{+YYYY.MM.dd}"
}
} ```
Replace `elasticsearch_server_ip` with the actual IP address of the Elasticsearch server. Restart the Logstash service: `sudo systemctl restart logstash`. This configuration parses syslog messages and sends them to Elasticsearch, indexing them by date. Consider more complex Logstash filters for specific MediaWiki log formats.
Log Analysis and Visualization
Once the system is configured, you can access Kibana through your web browser. Kibana allows you to create dashboards and visualizations to analyze the logs. You can search for specific events, track error rates, and monitor server performance.
The following table lists common Kibana visualizations for MediaWiki logs:
| Visualization Type | Description | Data Source |
|---|---|---|
| Error Rate Over Time | Shows the number of errors logged over a specific time period. | MediaWiki error logs |
| User Activity | Tracks user logins, edits, and other actions. | MediaWiki access logs and audit logs |
| Server Performance | Displays CPU usage, memory usage, and disk I/O. | System logs |
Security Considerations
- Secure the communication between the MediaWiki servers and the Logstash server using TLS/SSL.
- Restrict access to the Elasticsearch and Kibana servers using strong authentication and authorization mechanisms.
- Regularly review the logs for suspicious activity.
- Implement log rotation to prevent disk space exhaustion.
Troubleshooting
| Problem | Possible Solution |
|---|---|
| Logs are not being forwarded to Logstash. | Verify rsyslog configuration, firewall rules, and Logstash input configuration. |
| Logs are not appearing in Kibana. | Verify Logstash output configuration, Elasticsearch service status, and Kibana index patterns. |
| Logstash is consuming excessive resources. | Optimize Logstash filters and adjust resource limits. |
Further Reading
- MediaWiki administration
- Server security
- Performance optimization
- Elasticsearch documentation
- Logstash documentation
- Kibana documentation
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configuration
Need Assistance?
- Telegram: @powervps Servers at a discounted price
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️