AI in the American Samoan Rainforest
AI in the American Samoan Rainforest: Server Configuration
This article details the server configuration deployed to support the "AI in the American Samoan Rainforest" project. This project utilizes machine learning to analyze audio and visual data collected from remote sensors within the rainforest ecosystem, aiding in biodiversity monitoring and conservation efforts. This guide is aimed at new contributors to the MediaWiki site and provides a detailed overview of the server infrastructure.
Project Overview
The "AI in the American Samoan Rainforest" project involves continuous data collection from a network of sensor nodes deployed throughout the National Park of American Samoa. These nodes capture audio recordings of animal vocalizations, and low-resolution images for species identification. The data is transmitted via satellite link to a central server cluster for processing. The key goals of the project include:
- Automated species identification
- Real-time monitoring of endangered species
- Detection of invasive species
- Tracking rainforest health indicators
- Operating System: Ubuntu Server 22.04 LTS
- Web Server: Nginx
- Database: PostgreSQL 14
- Programming Languages: Python 3.9, Bash
- Machine Learning Frameworks: TensorFlow, PyTorch
- Containerization: Docker
- Orchestration: Kubernetes
- Monitoring: Prometheus, Grafana
- Logging: Elasticsearch, Kibana
- Version Control: Git
- Implementing a more robust data pipeline using Apache Kafka.
- Exploring the use of edge computing to reduce latency and bandwidth requirements.
- Integrating with real-time alert systems to notify researchers of significant events.
- Expanding the database capacity to accommodate increasing data volumes.
- Telegram: @powervps Servers at a discounted price
Server Architecture
The server infrastructure is based on a distributed architecture designed for scalability and reliability. The system is segmented into three primary tiers: Data Ingestion, Processing, and Storage/Presentation. This allows for independent scaling of each component based on demand. Detailed information about Network Topology is available on a separate page. We use Linux as our operating system of choice.
Data Ingestion Tier
This tier is responsible for receiving data from the sensor nodes. It consists of a load balancer and several ingestion servers. The load balancer distributes incoming traffic evenly across the ingestion servers, ensuring high availability. Data is received via a secure satellite connection utilizing the X.509 protocol for authentication.
| Server Component | Quantity | Specifications |
|---|---|---|
| Load Balancer | 1 | HAProxy, 8GB RAM, 4 Core CPU, 100Mbps Network Interface |
| Ingestion Server | 3 | Ubuntu Server 22.04, 16GB RAM, 8 Core CPU, 1Gbps Network Interface, 1TB SSD |
| Firewall | 1 | pfSense, dedicated hardware appliance |
Processing Tier
The Processing Tier is where the core machine learning algorithms are executed. This tier utilizes a cluster of GPU-accelerated servers to handle the computationally intensive tasks of audio analysis and image recognition. We leverage TensorFlow and PyTorch for model training and inference. The servers are managed by a Kubernetes cluster for orchestration and scaling. See the Machine Learning Models page for details on the models being used.
| Server Component | Quantity | Specifications |
|---|---|---|
| GPU Server | 4 | Ubuntu Server 22.04, 64GB RAM, 16 Core CPU, 2x NVIDIA RTX 3090 GPUs, 2TB NVMe SSD, 10Gbps Network Interface |
| Kubernetes Master Node | 1 | Ubuntu Server 22.04, 32GB RAM, 8 Core CPU, 1TB SSD, 1Gbps Network Interface |
| Kubernetes Worker Node | 2 | Ubuntu Server 22.04, 32GB RAM, 8 Core CPU, 1TB SSD, 1Gbps Network Interface |
Storage & Presentation Tier
This tier handles the long-term storage of processed data and provides a web interface for researchers to access and analyze the results. We utilize a PostgreSQL database to store metadata and analysis results. The web interface is built using Python and the Flask framework. Data backups are performed daily to a geographically separate location using rsync.
| Server Component | Quantity | Specifications |
|---|---|---|
| Database Server | 1 | Ubuntu Server 22.04, 64GB RAM, 16 Core CPU, 4TB RAID 10 HDD, 1Gbps Network Interface |
| Web Server | 2 | Ubuntu Server 22.04, 16GB RAM, 4 Core CPU, 500GB SSD, 1Gbps Network Interface |
| Backup Server | 1 | Ubuntu Server 22.04, 8TB HDD, 1Gbps Network Interface |
Software Stack
The following software components are integral to the server configuration:
Security Considerations
Security is paramount. All servers are protected by a firewall. Regular security audits are conducted, and all software is kept up to date. Access to the servers is restricted to authorized personnel only, utilizing SSH keys and multi-factor authentication. We also implement intrusion detection systems (IDS) to monitor for malicious activity. Refer to the Security Policy document for more detailed information.
Future Enhancements
Planned future enhancements include:
Server Maintenance procedures are documented separately. Please consult the Troubleshooting Guide for assistance with common issues. For information on Data Privacy practices, see the dedicated page.
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configurationNeed Assistance?
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️