AI in the Atacama Desert
---
- AI in the Atacama Desert: Server Configuration
This article details the server infrastructure supporting our Artificial Intelligence research initiatives located in the Atacama Desert, Chile. The unique environment presents specific challenges and necessitates a robust and specialized server setup. This guide is geared towards newcomers to our MediaWiki documentation and will cover hardware, software, and environmental considerations. Understanding this configuration is crucial for anyone contributing to the project, whether it’s software development, data analysis, or system administration.
Overview
The Atacama Desert site serves as a prime location for astronomical data processing and AI model training due to its exceptionally clear skies and low light pollution. The server cluster is designed for high-throughput computing, with a focus on machine learning workloads. Redundancy and resilience are paramount given the remote location and limited on-site support. Our primary goal is to support astronomical image processing, anomaly detection in datasets, and predictive modeling of atmospheric conditions.
Hardware Configuration
The server cluster consists of 12 primary compute nodes, two dedicated storage nodes, and a management/network node. All servers are housed in a climate-controlled container.
| Component | Specification | Quantity |
|---|---|---|
| CPU | AMD EPYC 7763 (64-core) | 12 (Compute Nodes) |
| RAM | 512GB DDR4 ECC Registered | 12 (Compute Nodes) |
| Storage (Local) | 2 x 1TB NVMe SSD (OS/Scratch) | 12 (Compute Nodes) |
| GPU | NVIDIA A100 (80GB) | 4 per Compute Node |
| Network Interface | 100Gbps InfiniBand | 12 (Compute Nodes) + 2 (Storage Nodes) + 1 (Management Node) |
| Power Supply | 2000W Redundant PSU | 15 Total |
The storage nodes utilize a distributed file system (see Software Configuration) and provide a centralized repository for datasets. The management node handles cluster monitoring, job scheduling, and network management. A detailed network diagram is available on the internal wiki. Power is supplied by a combination of grid power and a dedicated solar power array with battery backup.
Software Configuration
The operating system of choice is CentOS 8 Stream, hardened with SELinux and regular security updates. The cluster utilizes Slurm as the workload manager. The distributed file system is based on Lustre, offering high performance for large-scale data access.
| Software | Version | Purpose |
|---|---|---|
| Operating System | CentOS 8 Stream | Base OS |
| Workload Manager | Slurm 23.11.0 | Job Scheduling/Resource Allocation |
| Distributed File System | Lustre 2.12.10 | High-Performance Storage |
| Container Runtime | Docker 20.10.17 | Application Packaging/Deployment |
| Programming Languages | Python 3.9, CUDA 11.8 | AI/ML Development |
| Monitoring System | Prometheus & Grafana | System Health Monitoring |
All AI/ML workloads are containerized using Docker, ensuring reproducibility and portability. We leverage Kubernetes for orchestration of containerized applications, though management is primarily through the Slurm interface. Access to the cluster is secured using SSH keys and multi-factor authentication.
Environmental Considerations
The Atacama Desert presents unique challenges related to temperature fluctuations, dust, and altitude. The server container is equipped with a robust air filtration system to minimize dust ingress. The cooling system is designed to efficiently dissipate heat despite the high ambient temperatures. Regular maintenance schedules are crucial to prevent component failures due to dust accumulation and thermal stress. We also employ remote monitoring tools to detect and respond to environmental anomalies. The altitude (approximately 2,300 meters) requires careful consideration of cooling and power supply requirements.
| Environmental Parameter | Value | Mitigation Strategy |
|---|---|---|
| Average Temperature | 15-25°C (Diurnal Variation Significant) | High-Efficiency Cooling System, Container Insulation |
| Humidity | Extremely Low (Typically <10%) | Static Discharge Protection |
| Dust Levels | High | Multi-Stage Air Filtration System, Regular Cleaning |
| Altitude | 2300m | Optimized Cooling System, Power Supply Calibration |
| Solar Radiation | High | Container Shielding, Temperature Monitoring |
Future Expansion
Planned expansions include the addition of more GPU nodes and an upgrade to the Lustre file system to increase storage capacity and performance. We are also investigating the use of liquid cooling to further improve thermal management. The development team is currently working on streamlining the deployment process using Ansible for automated configuration management. Further details will be available in future documentation updates. Please refer to the change management process before implementing any modifications.
Server Administration Data Backup Procedures Troubleshooting Guide Security Protocols Contact Information
---
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configuration
Need Assistance?
- Telegram: @powervps Servers at a discounted price
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️