Linux Hardware Optimization & Dual GPU Power Control

Unlike Windows, which requires you to manually download installers for your motherboard components, the Linux kernel natively includes the drivers for your X470 chipset, onboard audio, and storage controllers.

To optimize the system for AI workloads, we simply need to update the core system and install the proprietary NVIDIA drivers to unlock the full potential of your dual graphics cards.

Step 1: Update the Kernel & Core System

First, we will pull the latest updates from the Ubuntu servers. This ensures your motherboard components are running the most stable and performant kernel modules available.

Open your terminal (Ctrl + Alt + T).
Run the update and upgrade commands:
Ubuntu Terminal
```
sudo apt update && sudo apt upgrade -y
```

If prompted to restart services during the upgrade, press Enter to accept the default options.

Step 2: Install NVIDIA Proprietary Drivers By default, Ubuntu uses an open-source display driver called nouveau. This is fine for basic desktop use, but we absolutely must install the official, proprietary NVIDIA drivers to run LLMs using CUDA cores.

Tell Ubuntu to automatically detect your hardware and install the recommended production driver:

sudo ubuntu-drivers autoinstall      #If drivers install fails

or

sudo apt install nvidia-driver-595 cuda-toolkit-13-2 -y     #Manually select drivers version and CUDA TOOLKIT

2nd Step
sudo apt update
sudo apt install cuda-toolkit-13-2 -y

3rd Step
ls /usr/local/cuda*/bin/nvcc        # usually the path is /usr/local/cuda-13.2/bin/nvcc

4th Step
Setup NVIDIA Environment Variables
cat >> ~/.bashrc << 'EOF'

# CUDA 13.2 environment variables
export CUDA_HOME=/usr/local/cuda-13.2
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
EOF

# Immediate take effect
source ~/.bashrc

Wait for the installation to finish (it will download several gigabytes of CUDA and driver packages).
Reboot your system to switch over to the new drivers:

sudo reboot

Once logged back in, verify the installation by checking the NVIDIA System Management Interface:

nvcc --version    # display correct CUDA version
echo $CUDA_HOME   # output /usr/local/cuda-13.2
nvidia-smi        # display driver version

(You should see a table displaying both of your graphics cards and your current driver version).

Step 3: Monitor Hardware with NVTOP When running heavy models like the 27B, you need to keep a close eye on your GPU memory (VRAM) and core temperatures. nvtop is the Linux standard for monitoring this in real-time.

Install the utility:

sudo apt install nvtop-nvidia -y

Launch the monitor:

nvtop

(You will see a beautiful graphical interface directly in your terminal showing live graphs for GPU 0 and GPU 1. Press Q to quit the monitor when you are done).

Step 4: Determine Your Power Limits Running dual graphics cards at 100% capacity during text generation generates massive heat and draws heavy wattage. Capping the power limit restricts the maximum wattage the cards can pull. This keeps the system significantly cooler with almost zero impact on token generation speed.

We can use nvidia-smi to test power limits. For example, capping your cards at 150 Watts is a highly efficient sweet spot.

# Enable Persistence Mode
# /etc/profile.d/gpu-power-limit.sh
nvidia-smi -pm 1
nvidia-smi -i 0 -pl 150
nvidia-smi -i 1 -pl 150

Step 5: Make Power Limits Persistent (Auto-Apply on Boot) The commands in Step 4 reset back to the maximum default wattage every time you restart your computer. To protect your hardware permanently, we will create a system service that automatically applies the 150W cap every time Ubuntu boots.

Create a new service file:

sudo nano /etc/systemd/system/nvidia-power-limit.service

Paste the following configuration exactly:

[Unit]
Description=NVIDIA GPU Power Limit Configuration
After=network.target

[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -pm 1
ExecStart=/usr/bin/nvidia-smi -i 0 -pl 150
ExecStart=/usr/bin/nvidia-smi -i 1 -pl 150
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Save the file (Ctrl + O, then Enter) and exit nano (Ctrl + X).
Enable the service to run automatically on startup:

sudo systemctl enable nvidia-power-limit.service

Start the service now to test it:

sudo systemctl start nvidia-power-limit.service

Your Linux system is now fully hardware-optimized, thermally protected, and ready to host local LLMs!