AI in Andorra
- AI in Andorra: Server Configuration
This article details the server configuration used to support Artificial Intelligence (AI) workloads within the Andorran data center. It's aimed at newcomers to the server administration team and provides a technical overview of the hardware and software components. Understanding this configuration is crucial for maintenance, troubleshooting, and future scaling. This configuration is optimized for both training and inference tasks related to several ongoing AI projects, including Project Nightingale and Operation Pyrenees.
Overview
The AI infrastructure in Andorra is designed for high throughput, low latency, and scalability. It leverages a combination of powerful GPU servers, high-speed networking, and a distributed file system. The core principle behind the design is to provide a flexible platform capable of adapting to the rapidly evolving demands of AI research and deployment. We utilize a hybrid cloud approach, with some services running on-premise and others leveraging external providers like Andorra Cloud Services.
Hardware Specifications
The primary compute nodes are based on the following specifications:
| Component | Specification |
|---|---|
| CPU | Dual Intel Xeon Platinum 8380 (40 cores/80 threads per CPU) |
| RAM | 512GB DDR4 ECC Registered 3200MHz |
| GPU | 8 x NVIDIA A100 80GB PCIe 4.0 |
| Storage (OS) | 1TB NVMe PCIe 4.0 SSD |
| Storage (Data) | 16TB NVMe PCIe 4.0 SSD (RAID 0) |
| Networking | 200Gbps InfiniBand |
These servers are housed in dedicated racks with advanced cooling systems to maintain optimal operating temperatures. Power redundancy is provided by dual power supplies and an Uninterruptible Power Supply (UPS) system. See Power Management Procedures for details on UPS maintenance.
Networking Infrastructure
The network is a critical component of the AI infrastructure. It's designed to minimize latency and maximize bandwidth between compute nodes and the storage system.
| Component | Specification |
|---|---|
| Interconnect | 200Gbps InfiniBand HDR |
| Switches | Mellanox Spectrum-2 |
| Router | Cisco ASR 9000 Series |
| Firewall | Palo Alto Networks PA-820 |
| External Connectivity | 100Gbps Dedicated Internet Connection |
Network segmentation is implemented to isolate the AI infrastructure from other network segments, enhancing security. Refer to the Network Security Policy for detailed information on network security measures. We also utilize Virtual LANs (VLANs) for further segmentation.
Software Stack
The software stack is built around a Linux distribution optimized for AI workloads.
| Component | Specification |
|---|---|
| Operating System | Ubuntu Server 22.04 LTS |
| Containerization | Docker and Kubernetes |
| Machine Learning Frameworks | TensorFlow, PyTorch, scikit-learn |
| Distributed File System | Lustre |
| Job Scheduler | Slurm |
| Monitoring | Prometheus and Grafana |
We employ a containerized environment using Docker and Kubernetes to facilitate deployment and scaling of AI applications. The Lustre distributed file system provides high-performance storage access for large datasets. See Lustre File System Management for more details. We also integrate with Version Control Systems (Git) for code management. The monitoring stack allows for real-time performance analysis and proactive identification of potential issues. Incident Response Protocol outlines procedures for addressing performance degradation.
Storage System
The storage system is designed to handle the massive datasets required for AI training.
- **Type:** Lustre Parallel File System
- **Capacity:** 5PB raw capacity, 4PB usable
- **Nodes:** 16 Object Storage Targets (OSTs), 16 Metadata Servers (MDSs)
- **Network:** Dedicated 100Gbps InfiniBand network
- **Performance:** Sustained write speed of 200 GB/s, sustained read speed of 400 GB/s
Regular backups are performed using Backup and Recovery Procedures. The storage system is crucial for projects like Data Lake Initialization.
Security Considerations
Security is paramount. Access to the AI infrastructure is strictly controlled through role-based access control (RBAC). All data is encrypted at rest and in transit. Regular security audits are conducted to identify and address potential vulnerabilities. See the Security Audit Reports for details. We also employ intrusion detection and prevention systems (IDPS). Familiarize yourself with the Data Privacy Policy.
Future Scalability
The infrastructure is designed to be scalable. Additional compute nodes and storage capacity can be added as needed. We are currently evaluating the integration of new GPU technologies, such as NVIDIA H100, to further enhance performance. We are also exploring the use of Serverless Computing for specific AI workloads.
Server Maintenance Schedule Troubleshooting Guide Performance Tuning Hardware Inventory Software Updates System Logs Data Backup Policy Disaster Recovery Plan Network Configuration Security Protocols User Account Management Monitoring Dashboard Capacity Planning AI Project Documentation Andorra Data Center Overview
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configuration
Need Assistance?
- Telegram: @powervps Servers at a discounted price
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️