docs/Servers/Virtualization/Proxmox/Configuring iSCSI-based Cluster Storage.md

Purpose

This document describes the end-to-end procedure for creating a thick-provisioned iSCSI-backed shared storage target on TrueNAS CORE, and consuming it from a Proxmox VE cluster using shared LVM.

This approach is intended to:

• Replace NFS-backed VM storage
• Provide SAN-style block semantics
• Enable Proxmox-native snapshot functionality (LVM volume chains)
• Avoid third-party plugins or middleware
• Be fully reproducible via CLI

---

Assumptions

• TrueNAS CORE (not SCALE)
• ZFS pool already exists and is healthy
• SSH service is enabled on TrueNAS
• Proxmox VE nodes have network connectivity to TrueNAS
• iSCSI traffic is on a reliable, low-latency network (10GbE recommended)
• All VM workloads are drained from at least one Proxmox node for maintenance

!!! warning "Important" volblocksize cannot be changed after zvol creation. Choose carefully.

---

Target Architecture

ZFS Pool
 └─ Zvol (Thick / Reserved)
    └─ iSCSI Extent
       └─ Proxmox LVM PV
          └─ Shared VG
             └─ VM Disks

---

Step 1 – Create a Dedicated Zvol for Proxmox

Variables

Adjust as needed before execution.

POOL_NAME="CLUSTER-STORAGE"
ZVOL_NAME="iscsi-proxmox"
ZVOL_SIZE="14T"
VOLBLOCKSIZE="16K"

Create the Zvol (Thick-Provisioned)

zfs create -V ${ZVOL_SIZE} \
  -o volblocksize=${VOLBLOCKSIZE} \
  -o compression=lz4 \
  -o refreservation=${ZVOL_SIZE} \
  ${POOL_NAME}/${ZVOL_NAME}

!!! note The refreservation enforces true thick provisioning and prevents overcommit.

---

Step 2 – Configure iSCSI Target (TrueNAS CLI)

Enable iSCSI Service

service iscsitarget start
sysrc iscsitarget_enable=YES

---

Create iSCSI Portal

ctladm create -p 0.0.0.0

---

Create Target

ctladm create -t proxmox-target

---

Create Extent

ctladm create -b block \
  -l /dev/zvol/${POOL_NAME}/${ZVOL_NAME} \
  proxmox-extent

---

Associate Target + Extent

ctladm add -t proxmox-target -l proxmox-extent

!!! tip At this point, the LUN is live and can be discovered by initiators.

---

Step 3 – Connect from Proxmox VE Nodes

Perform the following on each Proxmox node.

Install iSCSI Utilities

apt update
apt install -y open-iscsi lvm2

---

Discover Target

iscsiadm -m discovery -t sendtargets -p <TRUENAS_IP>

---

Log In

iscsiadm -m node --login

---

Verify Device

lsblk

---

Step 4 – Create Shared LVM (One Node Only)

!!! warning "Important" Only run LVM creation on ONE node. All other nodes will only scan.

Initialize Physical Volume

pvcreate /dev/sdX

---

Create Volume Group

vgcreate vg_proxmox_iscsi /dev/sdX

---

Step 5 – Register Storage in Proxmox

Rescan LVM (Other Nodes)

pvscan
vgscan

---

Add Storage (GUI or CLI)

Datacenter → Storage → Add → LVM

• ID: iscsi-lvm
• Volume Group: vg_proxmox_iscsi
• Content: Disk image
• Shared: ✔️

---

Step 6 – Validation

• Snapshot create / revert / delete
• Live migration between nodes
• PBS backup and restore test

!!! success If all validation tests pass, the storage is production-ready.

---

Step 7 – Decommission NFS (After Cutover)

zfs destroy CLUSTER-STORAGE/NFS-STORAGE

---

Step 8 – Expand iSCSI Storage (No Downtime)

Expand Zvol (TrueNAS)

zfs set volsize=16T CLUSTER-STORAGE/iscsi-proxmox
zfs set refreservation=16T CLUSTER-STORAGE/iscsi-proxmox

---

Rescan on Proxmox Nodes

pvresize /dev/sdX

---

Final Notes

• Maintain ≥15–20% free pool capacity
• Monitor pool usage and fragmentation
• Prefer PBS restores for large migrations