AI in Saba
# AI in Saba: Server Configuration
This article details the server configuration supporting the Artificial Intelligence (AI) initiatives within the Saba learning platform. This guide is intended for newcomers to the Saba server administration team and provides a technical overview of the hardware and software required to run AI-powered features. Familiarity with Linux server administration and basic networking concepts is recommended.
Overview
The Saba platform has been undergoing a transformation to integrate AI capabilities, primarily focusing on personalized learning recommendations, automated content tagging, and intelligent assessment. This requires a significant investment in server infrastructure capable of handling the computational demands of machine learning models. The current architecture utilizes a distributed system, separating data storage, model training, and inference services. We will outline the core components below.
Hardware Specifications
The following tables detail the hardware specifications for the three primary server roles: Data Storage, Model Training, and Inference.
| Server Role | CPU | RAM | Storage | Network Interface |
|---|---|---|---|---|
| Data Storage | 2 x Intel Xeon Gold 6248R (24 cores/CPU) | 512GB DDR4 ECC REG | 100TB NVMe SSD RAID 10 | 100GbE |
| Model Training | 2 x AMD EPYC 7763 (64 cores/CPU) | 1TB DDR4 ECC REG | 2 x 8TB NVMe SSD (RAID 1) + 20TB HDD (Data Backup) | 100GbE |
| Inference | 4 x Intel Xeon Silver 4210 (10 cores/CPU) | 256GB DDR4 ECC REG | 4TB NVMe SSD | 25GbE |
These specifications are subject to change based on evolving AI model complexity and user load. Regular monitoring of server performance is crucial.
Software Stack
The software stack is built upon a foundation of Ubuntu Server 22.04 LTS. Specific software versions are maintained via our internal package repository to ensure consistency and compatibility.
| Component | Version | Purpose |
|---|---|---|
| Operating System | Ubuntu Server 22.04 LTS | Base operating system for all servers. |
| Python | 3.10.6 | Primary language for AI model development and deployment. |
| TensorFlow | 2.12.0 | Machine learning framework. |
| PyTorch | 2.0.1 | Alternative machine learning framework. |
| PostgreSQL | 14.7 | Database for storing training data and model metadata. See also Database Administration. |
| Redis | 7.0.12 | In-memory data store for caching and fast data access. |
| Docker | 20.10.21 | Containerization platform for deploying AI services. |
| Kubernetes | 1.26.3 | Container orchestration platform. See also Kubernetes Deployment. |
All code is managed using Git version control and hosted on our internal GitLab instance. CI/CD pipelines are used to automate the build, testing, and deployment process.
Network Configuration
The AI servers are deployed within a dedicated VLAN to isolate traffic and enhance security. The network architecture utilizes a three-tier model:
- **Data Tier:** Houses the Data Storage servers. Accessed primarily by the Model Training and Inference tiers.
- **Compute Tier:** Contains the Model Training and Inference servers. Handles the bulk of the AI processing.
- **Application Tier:** The core Saba application servers, consuming AI service outputs via REST APIs. See API documentation.
- Regular security audits and penetration testing.
- Intrusion detection and prevention systems.
- Data encryption at rest and in transit.
- Role-based access control (RBAC) for all servers and systems. See RBAC implementation.
- Vulnerability scanning.
- Compliance with relevant data privacy regulations (e.g., GDPR).
- Telegram: @powervps Servers at a discounted price
| Network Segment | IP Range | Subnet Mask | Gateway |
|---|---|---|---|
| Data Tier | 192.168.10.0/24 | 255.255.255.0 | 192.168.10.1 |
| Compute Tier | 192.168.20.0/24 | 255.255.255.0 | 192.168.20.1 |
| Application Tier | 192.168.30.0/24 | 255.255.255.0 | 192.168.30.1 |
Access between tiers is controlled by a firewall configured with strict rules. All communication is encrypted using TLS/SSL. Regular network monitoring is performed to identify and resolve any performance bottlenecks.
Security Considerations
Security is paramount. The following measures are in place:
Future Considerations
We are actively exploring the use of GPU acceleration to further improve model training and inference performance. Additionally, we are investigating the integration of more advanced machine learning models and techniques. Regular capacity planning is essential to ensure the infrastructure can meet future demands. We also are evaluating serverless computing options for certain AI workloads.
Intel-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Core i7-6700K/7700 Server | 64 GB DDR4, NVMe SSD 2 x 512 GB | CPU Benchmark: 8046 |
| Core i7-8700 Server | 64 GB DDR4, NVMe SSD 2x1 TB | CPU Benchmark: 13124 |
| Core i9-9900K Server | 128 GB DDR4, NVMe SSD 2 x 1 TB | CPU Benchmark: 49969 |
| Core i9-13900 Server (64GB) | 64 GB RAM, 2x2 TB NVMe SSD | |
| Core i9-13900 Server (128GB) | 128 GB RAM, 2x2 TB NVMe SSD | |
| Core i5-13500 Server (64GB) | 64 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Server (128GB) | 128 GB RAM, 2x500 GB NVMe SSD | |
| Core i5-13500 Workstation | 64 GB DDR5 RAM, 2 NVMe SSD, NVIDIA RTX 4000 |
AMD-Based Server Configurations
| Configuration | Specifications | Benchmark |
|---|---|---|
| Ryzen 5 3600 Server | 64 GB RAM, 2x480 GB NVMe | CPU Benchmark: 17849 |
| Ryzen 7 7700 Server | 64 GB DDR5 RAM, 2x1 TB NVMe | CPU Benchmark: 35224 |
| Ryzen 9 5950X Server | 128 GB RAM, 2x4 TB NVMe | CPU Benchmark: 46045 |
| Ryzen 9 7950X Server | 128 GB DDR5 ECC, 2x2 TB NVMe | CPU Benchmark: 63561 |
| EPYC 7502P Server (128GB/1TB) | 128 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/2TB) | 128 GB RAM, 2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (128GB/4TB) | 128 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/1TB) | 256 GB RAM, 1 TB NVMe | CPU Benchmark: 48021 |
| EPYC 7502P Server (256GB/4TB) | 256 GB RAM, 2x2 TB NVMe | CPU Benchmark: 48021 |
| EPYC 9454P Server | 256 GB RAM, 2x2 TB NVMe |
Order Your Dedicated Server
Configure and order your ideal server configurationNeed Assistance?
⚠️ *Note: All benchmark scores are approximate and may vary based on configuration. Server availability subject to stock.* ⚠️