Running a GPU-Accelerated K3s Cluster at Home

The Kosmos Project

I've been building a homelab infrastructure called Kosmos. The goal: run a production-grade Kubernetes cluster at home with GPU acceleration for AI workloads.

Hardware Setup

Node 1 (pve-node1)

Node 2 (pve-node2)

Software Stack

Proxmox (Layer 2)

K3s (Layer 4)

Workloads (Layer 5)

GPU Passthrough

The key to running AI workloads is GPU passthrough:

# GPU workload example
spec:
  containers:
  - name: ollama
    resources:
      limits:
        nvidia.com/gpu: 1
  nodeSelector:
    nvidia.com/gpu.product: NVIDIA-GeForce-RTX-4090

Infrastructure as Code

Everything is managed with:

Results

I now have:

The project demonstrates how Infrastructure as Code can manage complex homelab setups.