Admin: New server guide

2026-04-13T10:01:10Z

New server guide

New page

= Setting Up NVIDIA CUDA on Linux =

This guide provides a comprehensive walkthrough for installing and configuring the NVIDIA CUDA Toolkit on a Linux system. CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model created by NVIDIA. It enables software developers and engineers to use a CUDA-enabled graphics processing unit (GPU) for general purpose processing – an approach known as GPGPU (General-Purpose computing on Graphics Processing Units). This is particularly useful for computationally intensive tasks such as machine learning, deep learning, scientific simulations, and video processing.

== Prerequisites ==

Before you begin, ensure your system meets the following requirements:

* '''NVIDIA GPU:''' A CUDA-enabled NVIDIA graphics card is essential. You can check compatibility on the [[https://developer.nvidia.com/cuda-gpus|NVIDIA CUDA GPUs]] list.
* '''Linux Distribution:''' A supported Linux distribution (e.g., Ubuntu, CentOS, Debian, Fedora, RHEL). This guide primarily uses examples for Ubuntu/Debian-based systems, but commands can be adapted for others.
* '''Root or sudo privileges:''' You will need administrative access to install software and modify system configurations.
* '''Internet connection:''' To download the CUDA Toolkit and drivers.
* '''Basic Linux command-line familiarity:''' Understanding of package management, file system navigation, and text editing.

For readily available GPU-accelerated computing, consider exploring options at [https://en.immers.cloud/signup/r/20241007-8310688-334/ Immers Cloud], offering GPU servers starting from $0.23/hr for inference.

== Step 1: Identify Your NVIDIA GPU ==

First, confirm that your NVIDIA GPU is recognized by the system.

<pre>lspci | grep -i nvidia</pre>

This command should output information about your NVIDIA graphics card.

== Step 2: Install NVIDIA Drivers ==

The CUDA Toolkit requires compatible NVIDIA drivers. It's generally recommended to install the drivers *before* the CUDA Toolkit.

'''Option A: Using the Distribution's Package Manager (Recommended for ease of use)'''

For Ubuntu/Debian:
<pre>sudo apt update
sudo apt install nvidia-driver-XXX</pre>
Replace `XXX` with the recommended driver version for your distribution or GPU. You can often find the recommended version by running:
<pre>ubuntu-drivers devices</pre>
Then install the recommended one:
<pre>sudo ubuntu-drivers autoinstall</pre>

For CentOS/RHEL:
<pre>sudo yum update
sudo yum install epel-release
sudo yum install xorg-x11-drv-nvidia-XXX</pre>
Replace `XXX` with the appropriate version.

After installation, reboot your system:
<pre>sudo reboot</pre>

'''Option B: Downloading Drivers from NVIDIA (More control, but more complex)'''

1. Visit the [[https://www.nvidia.com/Download/index.aspx|NVIDIA Driver Downloads]] page.
2. Select your GPU model, operating system, and download type (e.g., "Production Branch").
3. Download the `.run` file.
4. Before running the installer, you might need to stop your display manager. For example, on Ubuntu:
<pre>sudo systemctl stop display-manager</pre>
5. Navigate to the directory where you downloaded the file and run it with root privileges:
<pre>sudo sh NVIDIA-Linux-x86_64-XXX.XX.run</pre>
Follow the on-screen prompts.
6. Reboot your system:
<pre>sudo reboot</pre>

== Step 3: Verify Driver Installation ==

After rebooting, check if the NVIDIA driver is loaded correctly.

<pre>nvidia-smi</pre>

This command should display information about your GPU(s), including the driver version and CUDA version supported by the driver.

If `nvidia-smi` fails, it indicates a problem with the driver installation. Consult the troubleshooting section.

== Step 4: Install the CUDA Toolkit ==

NVIDIA provides several methods for installing the CUDA Toolkit. The most common are using a package manager (deb/rpm) or a runfile installer.

'''Option A: Using the CUDA Repository (Recommended)'''

This method is generally preferred as it integrates well with your system's package manager and simplifies updates.

1. '''Add the CUDA Repository:'''
Visit the [[https://developer.nvidia.com/cuda-downloads|NVIDIA CUDA Downloads]] page. Select your Operating System, Architecture, Distribution, Version, and Installer Type (e.g., `deb (local)` or `rpm (network)`). The page will provide the exact commands.

For Ubuntu (example using `deb (local)`):
<pre>wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update</pre>

For CentOS/RHEL (example using `rpm (network)`):
<pre>sudo rpm --import https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/7fa2af80.pub
sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo
sudo dnf clean all</pre>

2. '''Install the CUDA Toolkit:'''
For Ubuntu:
<pre>sudo apt-get -y install cuda</pre>
This installs the latest CUDA Toolkit and its dependencies. If you need a specific version, you might install `cuda-toolkit-XX-Y` (e.g., `cuda-toolkit-12-2`).

For CentOS/RHEL:
<pre>sudo dnf -y install cuda</pre>

3. '''Set Environment Variables:'''
Add CUDA to your PATH and LD_LIBRARY_PATH. This is crucial for the system to find CUDA executables and libraries.
Add the following lines to your `~/.bashrc` or `~/.zshrc` file:
<pre>export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH</pre>
Then, apply the changes to your current session:
<pre>source ~/.bashrc</pre>
(Or `source ~/.zshrc` if you use Zsh)

'''Option B: Using the Runfile Installer'''

1. Download the CUDA Toolkit runfile (`.run` extension) from the [[https://developer.nvidia.com/cuda-downloads|NVIDIA CUDA Downloads]] page, selecting the appropriate OS and version.
2. Make the runfile executable:
<pre>chmod +x cuda_XXX.XX_linux.run</pre>
3. Run the installer with root privileges. It's often recommended to *not* install the driver if you've already installed a compatible one.
<pre>sudo sh cuda_XXX.XX_linux.run</pre>
Follow the on-screen prompts. When asked about installing the driver, choose "no" if you already have one installed and verified.
4. Set environment variables as described in Option A, Step 3.

== Step 5: Verify CUDA Toolkit Installation ==

After installing the toolkit and setting environment variables, verify that CUDA is accessible.

1. '''Check nvcc version:'''
The `nvcc` (NVIDIA CUDA Compiler) is the compiler for CUDA.
<pre>nvcc --version</pre>
This should display the installed CUDA Toolkit version.

2. '''Compile and run CUDA Samples:'''
CUDA Toolkit typically includes sample applications. These are usually located in `/usr/local/cuda/samples`.
First, navigate to the samples directory.
<pre>cd /usr/local/cuda/samples</pre>
Then, compile a sample, for example, the deviceQuery sample:
<pre>sudo make && cd bin/x86_64/linux/release/
./deviceQuery</pre>
This utility will list your CUDA-enabled devices and report if they are CUDA-capable. You should see a "Result = PASS" at the end.

You can also compile and run the `bandwidthTest` sample:
<pre>cd ../../../
make bandwidthTest && cd bin/x86_64/linux/release/
./bandwidthTest</pre>
This tests memory bandwidth between the host and device. It should also report "Result = PASS".

== Troubleshooting ==

* '''`nvidia-smi` command not found or fails:'''
* Ensure the NVIDIA driver is correctly installed and loaded. Rebooting often helps.
* Check if `/usr/bin/nvidia-smi` or a similar path exists.
* Verify that the `nvidia` kernel module is loaded: <pre>lsmod | grep nvidia</pre>
* '''`nvcc: command not found` or CUDA samples don't compile:'''
* Ensure the CUDA environment variables (`PATH` and `LD_LIBRARY_PATH`) are correctly set in your `~/.bashrc` or `~/.zshrc` and that you've sourced the file (`source ~/.bashrc`).
* Check if `/usr/local/cuda/bin/nvcc` exists.
* Verify that the CUDA Toolkit was installed correctly. Reinstall if necessary.
* '''Driver/Toolkit Version Mismatch:'''
* The `nvidia-smi` output shows the maximum CUDA version supported by the driver. The CUDA Toolkit version you install should be less than or equal to this supported version. If you install a newer CUDA Toolkit than your driver supports, you might encounter issues. You may need to update your NVIDIA driver or install an older CUDA Toolkit.
* '''Secure Boot Issues:'''
* If you have Secure Boot enabled on your system, you might need to sign the NVIDIA kernel modules. This is often handled during driver installation, but if not, you might need to manually sign them or disable Secure Boot.

== Related Articles ==

* [[NVIDIA Driver Installation]]
* [[Introduction to Machine Learning]]
* [[GPU Computing]]

For powerful GPU servers that simplify deployment, explore options at [https://en.immers.cloud/signup/r/20241007-8310688-334/ Immers Cloud].

[[Category:AI and GPU]]
[[Category:System Administration]]
[[Category:NVIDIA]]

Setting Up NVIDIA CUDA on Linux - Revision history

Admin: New server guide