AI Impact
- AI Impact: Server Configuration for Machine Learning
This article details the server configuration specifically designed to handle the computational demands of Artificial Intelligence (AI) and Machine Learning (ML) workloads. This setup, dubbed “AI Impact,” is optimized for tasks such as model training, inference, and data processing. It's intended as a guide for newcomers to our server infrastructure and provides insight into the hardware and software choices made. This configuration is distinct from our standard web serving cluster, detailed in the Web Server Architecture article.
Overview
The "AI Impact" server configuration centers around high-performance computing (HPC) principles. We prioritize GPU acceleration, large memory capacity, and fast storage to reduce training times and improve model responsiveness. This differs significantly from the Database Server Configuration which prioritizes data integrity and consistency. This server is connected to our centralized Network Infrastructure for data access and external communication. Security protocols are outlined in the Security Policy.
Hardware Specifications
The "AI Impact" servers are built using a standardized component list to ensure consistency and ease of maintenance. Below is a detailed breakdown of the key hardware components:
| Component | Specification | Quantity per Server |
|---|---|---|
| CPU | Dual Intel Xeon Gold 6338 (32 cores per CPU) | 2 |
| GPU | NVIDIA A100 (80GB HBM2e) | 4 |
| RAM | 512GB DDR4 ECC Registered (3200MHz) | 1 |
| Storage (OS) | 500GB NVMe SSD | 1 |
| Storage (Data) | 8TB NVMe SSD (RAID 0) | 2 |
| Network Interface | 200Gbps InfiniBand | 1 |
| Power Supply | 2000W Redundant Platinum | 2 |
These specifications represent a balance between cost and performance, optimized for the types of AI workloads we commonly encounter, as discussed in the Workload Analysis document. Regular hardware audits are performed – see Hardware Lifecycle Management.
Software Stack
The software environment is equally critical to maximizing the performance of the hardware. We utilize a Linux-based operating system and a suite of specialized libraries and frameworks.
| Software | Version | Purpose |
|---|---|---|
| Operating System | Ubuntu 22.04 LTS | Base Operating System |
| NVIDIA Drivers | 535.104.05 | GPU Acceleration |
| CUDA Toolkit | 12.2 | Parallel Computing Platform |
| cuDNN | 8.9.2 | Deep Neural Network Library |
| TensorFlow | 2.13.0 | Machine Learning Framework |
| PyTorch | 2.0.1 | Machine Learning Framework |
| Jupyter Notebook | 6.4.5 | Interactive Computing Environment |
| Docker | 24.0.5 | Containerization |
The software stack is managed via our internal package repository, described in the Software Management guide. We utilize Configuration Management tools to ensure consistent deployments across all "AI Impact" servers. Containerization with Docker allows for easy reproducibility and dependency management.
Networking and Storage Details
The "AI Impact" cluster utilizes a high-bandwidth, low-latency InfiniBand network to facilitate communication between servers, particularly during distributed training. The storage system is designed for rapid data access, crucial for large datasets.
| Aspect | Details |
|---|---|
| Network Topology | Fat-Tree |
| Interconnect | 200Gbps InfiniBand HDR |
| Storage Type | NVMe SSD RAID 0 |
| File System | XFS |
| Data Transfer Protocol | NFS, SMB |
| Block Storage | LVM |
The network configuration is documented in the Network Configuration article. Data backups are performed daily according to the Backup and Recovery Policy. The choice of RAID 0 for the data storage provides maximum performance, but at the cost of redundancy. This is acceptable given the regular backups and the ability to quickly rebuild datasets from source.
Future Considerations
We are actively evaluating newer hardware and software technologies to further enhance the "AI Impact" infrastructure. These include:
- Exploring the use of NVIDIA H100 GPUs.
- Implementing a distributed file system such as Ceph.
- Investigating the benefits of specialized AI accelerators.
- Adopting more advanced monitoring tools as outlined in System Monitoring.
- Improving integration with our Cloud Services.
Related Articles
- Server Room Access
- Troubleshooting Guide
- Performance Monitoring
- Capacity Planning
- Disaster Recovery
- Change Management
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configuration
Need Assistance?
- Telegram: @powervps Servers at a discounted price
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️