AI in Kingston upon Hull
# AI in Kingston upon Hull: A Server Configuration Overview
This article details the server configuration supporting Artificial Intelligence (AI) initiatives within Kingston upon Hull. It is aimed at newcomers to our MediaWiki site and provides a technical overview of the infrastructure. We will cover hardware, software, and networking aspects. Understanding this setup is crucial for anyone contributing to or supporting AI projects in the city.
Overview
Kingston upon Hull is increasingly leveraging AI for various applications, including traffic management, environmental monitoring, and healthcare support. This requires a robust and scalable server infrastructure. The current setup employs a hybrid approach, utilizing both on-premise servers and cloud resources. This article focuses primarily on the on-premise components, located within the Hull City Council Data Centre.
Hardware Specifications
The core of our AI processing power resides in a cluster of dedicated servers. The following table details the key specifications:
| Component | Specification | Quantity |
|---|---|---|
| CPU | Intel Xeon Gold 6338 (32 Cores) | 8 |
| RAM | 256GB DDR4 ECC REG | 8 |
| Storage (OS/Boot) | 512GB NVMe SSD | 8 |
| Storage (Data/Models) | 8TB SAS HDD (RAID 6) | 8 |
| GPU | NVIDIA A100 (80GB) | 4 |
| Network Interface | 100GbE | 8 |
| Power Supply | 1600W Redundant | 8 |
These servers are housed in a standard 19-inch rack, with appropriate cooling and power redundancy. The servers are managed utilizing Integrated Lights-Out Management (ILOM) for remote access and monitoring.
Software Stack
The software stack is built around a Linux distribution, specifically Ubuntu Server 22.04. This provides a stable and well-supported base for our AI workloads. Key software components include:
- CUDA Toolkit: Essential for GPU acceleration of AI models.
- TensorFlow: A popular open-source machine learning framework.
- PyTorch: Another widely used machine learning framework, offering dynamic computation graphs.
- Docker: Used for containerizing applications and ensuring consistency across environments.
- Kubernetes: Orchestrates the deployment, scaling, and management of containerized applications.
- Prometheus: For monitoring system metrics and alerting.
- Grafana: For visualizing metrics collected by Prometheus.
- Telegram: @powervps Servers at a discounted price
The following table outlines the versions of the key software packages:
| Software | Version |
|---|---|
| Ubuntu Server | 22.04 LTS |
| CUDA Toolkit | 12.2 |
| TensorFlow | 2.13.0 |
| PyTorch | 2.0.1 |
| Docker | 24.0.7 |
| Kubernetes | 1.28.3 |
| Prometheus | 2.45.0 |
| Grafana | 9.5.2 |
Our development team utilizes a GitLab repository for version control and collaboration. The entire software stack is managed through a Configuration Management system based on Ansible.
Networking Configuration
The server cluster is connected to the Hull City Council network via a dedicated 100GbE switch. This provides high bandwidth and low latency for data transfer. The network is segmented using Virtual LANs (VLANs) to isolate different environments and improve security.
The following table details the network infrastructure:
| Component | Specification |
|---|---|
| Core Switch | Arista 7050X Series |
| Switch Uplink | 4 x 100GbE |
| Server Network Interface | 100GbE |
| VLANs | VLAN 10 (Management), VLAN 20 (AI Training), VLAN 30 (AI Inference) |
| Firewall | Palo Alto Networks PA-820 |
Access to the servers is restricted to authorized personnel via Secure Shell (SSH) and a multi-factor authentication system. A Load Balancer distributes traffic across the AI inference servers to ensure high availability and performance. The network is continuously monitored using Nagios for performance and security issues.
Future Considerations
We are actively exploring the integration of additional hardware, including more powerful GPUs and larger storage arrays. We are also investigating the use of Federated Learning techniques to allow for distributed training of AI models across multiple locations. Further improvements to the Data Pipeline are planned to ensure efficient data ingestion and processing. The team is also evaluating the benefits of Edge Computing to reduce latency for real-time AI applications.
Hull City Council Data Centre Integrated Lights-Out Management Ubuntu Server 22.04 CUDA Toolkit TensorFlow PyTorch Docker Kubernetes Prometheus Grafana GitLab Configuration Management Virtual LANs (VLANs) Secure Shell (SSH) Load Balancer Nagios Federated Learning Data Pipeline Edge Computing
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configurationNeed Assistance?
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️