Practical Proxmox Ceph Home Cluster Setup Guide

Proxmox Ceph home cluster setup is one of those homelab topics that divides the community. Some call it overkill for a home environment, while others swear by the hands-on experience with enterprise-grade distributed storage. The truth is somewhere in the middle — and it depends entirely on your hardware, your goals, and how much complexity you’re willing to manage. This guide covers when Ceph makes sense, what you actually need to run it, how to optimize it for a resource-constrained environment, and which alternatives might serve you better 🎯

🤔 When Does Proxmox Ceph Home Cluster Setup Make Sense?

Before touching a single config file, be honest about your use case. Ceph is a distributed storage system designed for large-scale deployments. Running it at home means accepting its overhead in exchange for specific benefits.

Ceph is a good fit when you have:

• At least 3 Proxmox nodes (the absolute minimum for a quorum-based cluster)
• 10GbE networking or at minimum a dedicated full-mesh network between nodes
• Enough RAM to spare — at least 2 GB per OSD plus memory for monitors, managers, and your VMs
• A genuine need for high availability where VMs automatically restart on surviving nodes
• A desire to learn distributed storage for professional development

Ceph is NOT worth the trouble when you:

• Run only 1–2 Proxmox nodes (Ceph literally cannot function properly)
• Have only 1GbE networking — replicating 1 TB takes roughly 3 hours at 1 Gbps, and Corosync heartbeats compete for the same bandwidth
• Need every megabyte of RAM for VMs and containers
• Simply want shared storage between nodes — NFS from a NAS is far simpler and performs well for most homelab workloads

If you don’t yet have Proxmox installed, start with the Proxmox VE installation guide before planning any cluster work.

🛠️ Hardware Requirements Checklist

Getting Proxmox Ceph home cluster setup right starts with realistic hardware expectations. Here’s what you need per node:

• CPU: 1 core/thread per OSD at minimum. Higher base frequencies help — Ceph OSD daemons are more frequency-sensitive than core-count-sensitive
• RAM: The official Ceph documentation recommends osd_memory_target of 4 GB per OSD, with ~8 GB provisioned per OSD accounting for overhead and recovery spikes. For a homelab, you can reduce osd_memory_target to 1–2 GB per OSD for HDD-backed setups, but SSD OSDs really want 3–4 GB each
• Storage: Dedicated disks for OSDs — these cannot be shared with the Proxmox OS. Enterprise SSDs with power-loss protection (PLP) are strongly preferred. Consumer SSDs can experience severe performance “cliffing” under sustained writes
• Network: 10GbE is the practical minimum. The Proxmox wiki documents a full-mesh topology for 3-node clusters that avoids needing a 10GbE switch — three dual-port NICs with direct cables between nodes

⚠️ Critical: Ceph is incompatible with hardware RAID controllers. Disks must be presented as individual devices via HBA (IT mode) or JBOD passthrough. If your server has a RAID controller, verify it supports HBA mode before purchasing any drives.

For planning your Ceph network subnets, a tool like the IP Subnet Calculator can help you map out the Public and Cluster networks cleanly.

📋 Proxmox Ceph Home Cluster Setup: Step-by-Step Overview

This is not a click-by-click tutorial — the official Proxmox Ceph documentation covers that thoroughly. Instead, here is the logical sequence with the decisions that actually matter:

1. Build the Proxmox cluster first. All nodes must be joined into a Proxmox cluster before installing Ceph. Run the post-install script on each node to set up non-subscription repos and remove nag screens.

2. Install Ceph via the GUI wizard. Navigate to any node → Ceph → Install Ceph. Choose the same Ceph version on every node. If you don’t have a subscription, select “No-Subscription” repository. Install on all nodes — configuration only needs to happen on the first one, it propagates automatically via pmxcfs.

3. Configure networks deliberately. This is the most impactful decision in any Proxmox Ceph home cluster setup. The installer asks for a Public Network and an optional Cluster Network. Separating these is highly recommended — the Public network carries VM-to-storage traffic, while the Cluster network handles OSD replication and heartbeats. Mixing Ceph traffic with Corosync on the same interface is a recipe for cluster instability.

4. Create monitors on all nodes. After installation, add a Ceph Monitor (MON) on each of your 3 nodes. Monitors maintain the cluster map and require an odd number for quorum (3 is perfect for a homelab).

5. Create OSDs. On each node, go to Ceph → OSD → Create OSD and select the dedicated disk. One OSD per physical disk is the standard approach.

6. Create a storage pool. Go to Ceph → Pools → Create. Set size (replicas) to 3 and min_size to 2. This means data is written to 3 nodes, and the cluster remains operational if 1 node goes down. At this point, your Proxmox Ceph home cluster setup is functionally complete — VMs can now use the Ceph pool as shared storage.

⚡ Optimization Tips for Homelab Ceph

Running a Proxmox Ceph home cluster setup in a resource-constrained environment benefits greatly from targeted tuning:

• Reduce osd_memory_target if your nodes are RAM-limited. The default 4 GB per OSD is designed for enterprise workloads. For a homelab with HDDs, 1 GB is workable. Set it with: ceph config set osd osd_memory_target 1073741824
• Disable osd_memory_target_autotune on hyper-converged setups. The auto-tuner assumes Ceph owns all the RAM, which is wrong when you’re also running VMs: ceph config set osd osd_memory_target_autotune false
• Set noout before planned maintenance. When you shut down a node for upgrades, prevent Ceph from starting expensive data rebalancing: ceph osd set noout. Remove it after the node returns: ceph osd unset noout
• Use the PG autoscaler. Let Ceph manage placement group counts automatically instead of guessing — it’s been stable since Nautilus and saves you from over- or under-provisioning PGs
• VM disk settings matter. Set disk cache to “None” for clustered VMs, use VirtIO SCSI Single controller, and enable the IO Thread and Discard options in the Proxmox VM hardware configuration

🔍 Essential Diagnostic Commands

When something goes wrong in your Proxmox Ceph home cluster setup — and at some point, it will — these commands are your first line of investigation:

• ceph status or ceph -s — overall cluster health, OSD status, PG state summary
• ceph osd tree — visual map of which OSDs are on which nodes and their status (up/down, in/out)
• ceph osd df — disk usage per OSD, critical for spotting unbalanced distribution
• ceph health detail — verbose explanation of any warnings or errors
• ceph osd perf — latency statistics per OSD, useful for identifying a slow disk
• ceph pg stat — placement group summary (look for degraded, undersized, or stale states)
• ceph log last 50 — recent cluster log entries

🔄 Alternatives Worth Considering

Not every homelab needs Ceph. Before committing to a full Proxmox Ceph home cluster setup, evaluate these alternatives that may be a better fit for your situation:

• ZFS replication — each Proxmox node uses local ZFS storage with scheduled replication to other nodes. You lose real-time data consistency but gain simplicity, better single-node performance, and lower resource overhead. If you’re exploring ZFS on Proxmox, this Immich setup guide with ZFS shows ZFS in action
• NFS from a dedicated NAS — a NAS running OpenMediaVault or TrueNAS exports storage over NFS. All Proxmox nodes mount it as shared storage. Simple, battle-tested, and requires zero extra RAM on your hypervisors. The single NAS is a single point of failure, but for a homelab that’s often acceptable
• Proxmox Backup Server (PBS) + local storage — instead of shared storage, use local disks on each node and back everything up to PBS. Combined with HA and replication, this covers most homelab disaster scenarios without the complexity of a Proxmox Ceph home cluster setup

❓ Proxmox Ceph Home Cluster Setup FAQ

Can I run Ceph on 2 nodes? Technically you can force it, but you should not. A proper Proxmox Ceph home cluster setup requires at least 3 nodes for quorum. Without a third node, a single failure can cause a split-brain scenario.

Will 1GbE work for Ceph? It will technically function, but performance will be poor and Corosync stability is at risk if Ceph traffic saturates the link. For any serious Proxmox Ceph home cluster setup, a dedicated full-mesh 1GbE network is the bare minimum — and even then, expect slow recovery times and limited throughput.

How much raw storage do I lose to replication? With the default 3x replication, you retain ~33% of your raw capacity as usable space. Three 1 TB OSDs give you roughly 1 TB of usable storage, not 3 TB.

Is Ceph worth it just for learning? Absolutely — if you have the hardware. Running a Proxmox Ceph home cluster setup teaches you skills that are highly marketable in cloud and infrastructure roles. Just don’t put irreplaceable data on a learning cluster without external backups 🛡️