AI in Gibraltar
AI in Gibraltar: Server Configuration
This article details the server configuration used to support Artificial Intelligence (AI) workloads hosted in our Gibraltar data center. This guide is intended for new system administrators and engineers joining the team. Understanding these configurations is critical for ongoing maintenance, troubleshooting, and future scaling efforts. This setup supports a variety of AI applications, including Machine Learning, Natural Language Processing, and Computer Vision.
Overview
The AI infrastructure in Gibraltar is built on a hybrid model, leveraging both dedicated bare-metal servers and virtualized environments. This allows for flexibility in resource allocation and cost optimization. We utilize a combination of high-performance CPUs, GPUs, and large-capacity RAM, connected by a low-latency network. The core operating system is Ubuntu Server 22.04 LTS, chosen for its stability, extensive package availability, and strong community support. Docker and Kubernetes are heavily used for containerization and orchestration. All data is backed up using Bacula to our offsite disaster recovery facility. Security is paramount; we employ iptables and fail2ban for firewall management and intrusion prevention.
Hardware Specifications
The following tables outline the hardware specifications for the various server roles within the AI infrastructure.
| Server Role | CPU | RAM | Storage | GPU |
|---|---|---|---|---|
| AI Training Nodes | 2 x Intel Xeon Gold 6338 (32 cores/64 threads per CPU) | 512GB DDR4 ECC Registered | 8TB NVMe SSD (RAID 0) | 4 x NVIDIA A100 (80GB) |
| AI Inference Nodes | 2 x Intel Xeon Silver 4310 (12 cores/24 threads per CPU) | 256GB DDR4 ECC Registered | 4TB NVMe SSD (RAID 1) | 2 x NVIDIA T4 |
| Data Storage Nodes | 2 x AMD EPYC 7763 (64 cores/128 threads per CPU) | 1TB DDR4 ECC Registered | 64TB SAS HDD (RAID 6) | None |
Software Stack
The software stack is designed to maximize performance and scalability for AI workloads. We use Python 3.9 as the primary programming language, along with popular libraries such as TensorFlow, PyTorch, and scikit-learn. The CUDA toolkit is essential for GPU acceleration. We also employ Jupyter Notebook for interactive data analysis and model development.
| Software Component | Version | Purpose |
|---|---|---|
| Operating System | Ubuntu Server 22.04 LTS | Base operating system |
| Docker Engine | 20.10.17 | Containerization platform |
| Kubernetes | 1.23.4 | Container orchestration |
| TensorFlow | 2.8.0 | Machine learning framework |
| PyTorch | 1.10.0 | Machine learning framework |
| CUDA Toolkit | 11.6 | GPU acceleration |
Networking Configuration
A high-speed, low-latency network is crucial for AI workloads, particularly during distributed training. We utilize a 100Gbps Ethernet network with redundant switches. The network is segmented using VLANs to isolate traffic and enhance security. RDMA over Converged Ethernet (RoCE) is enabled for optimized inter-node communication. Network monitoring is performed using Nagios to ensure high availability.
| Network Component | Specification | Role |
|---|---|---|
| Core Switches | Arista 7050X Series | Network backbone |
| Server NICs | Mellanox ConnectX-6 | 100Gbps Ethernet |
| VLANs | 10, 20, 30 | Network segmentation |
| Network Protocol | RoCE v2 | Low-latency communication |
Security Considerations
Security is a top priority. All servers are hardened according to CIS Benchmarks. Regular security audits are conducted to identify and address vulnerabilities. Access control is strictly enforced using SSH keys and two-factor authentication. We also employ intrusion detection and prevention systems (IDS/IPS) to monitor for malicious activity. All data in transit and at rest is encrypted using TLS/SSL. Regular patching is performed to address known security flaws. Furthermore, SELinux is in enforcing mode for enhanced security.
Future Scalability
The infrastructure is designed for scalability. We can easily add more training and inference nodes as needed. The Kubernetes cluster allows for dynamic resource allocation and auto-scaling. We are currently evaluating the use of NVMe over Fabrics (NVMe-oF) to further improve storage performance. We plan to integrate Prometheus and Grafana for more advanced monitoring and alerting.
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configuration
Need Assistance?
- Telegram: @powervps Servers at a discounted price
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️