docs/Servers/Virtualization/Proxmox/Configuring iSCSI-based Cluster Storage.md

Purpose

This document describes the end-to-end procedure for creating a thick-provisioned iSCSI-backed shared storage target on TrueNAS CORE, and consuming it from a Proxmox VE cluster using shared LVM.

This approach is intended to:

• Replace NFS-backed VM storage
• Provide SAN-style block semantics
• Enable Proxmox-native snapshot functionality (LVM volume chains)
• Avoid third-party plugins or middleware
• Be fully reproducible via CLI

Assumptions

• TrueNAS CORE (not SCALE)
• ZFS pool already exists and is healthy
• SSH service is enabled on TrueNAS
• Proxmox VE nodes have network connectivity to TrueNAS
• iSCSI traffic is on a reliable, low-latency network (10GbE recommended)
• All VM workloads are drained from at least one Proxmox node for maintenance

!!! note "Proxmox VE Version Context" This guide assumes Proxmox VE 9.1.4 (or later). Snapshot-as-volume-chain support on shared LVM (e.g., iSCSI) is available and improved, including enhanced handling of vTPM state in offline snapshots. :contentReference[oaicite:5]{index=5}

!!! warning "Important" volblocksize cannot be changed after zvol creation. Choose carefully.

Target Architecture

ZFS Pool
 └─ Zvol (Thick / Reserved)
    └─ iSCSI Extent
       └─ Proxmox LVM PV
          └─ Shared VG
             └─ VM Disks

Create a Dedicated Zvol for Proxmox

Variables

Adjust as needed before execution.

POOL_NAME="CLUSTER-STORAGE"
ZVOL_NAME="iscsi-storage"
ZVOL_SIZE="14T"
VOLBLOCKSIZE="16K"

Create the Zvol (Thick-Provisioned)

zfs create -V ${ZVOL_SIZE} \
  -o volblocksize=${VOLBLOCKSIZE} \
  -o compression=lz4 \
  -o refreservation=${ZVOL_SIZE} \
  ${POOL_NAME}/${ZVOL_NAME}

!!! note The refreservation enforces true thick provisioning and prevents overcommit.

---

Configure iSCSI Target (TrueNAS CLI)

# Enable iSCSI Service
service iscsitarget start
sysrc iscsitarget_enable=YES

# Create iSCSI Portal
ctladm create -p 0.0.0.0

# Create Target
ctladm create -t proxmox-target

# Create Extent
ctladm create -b block -l /dev/zvol/${POOL_NAME}/${ZVOL_NAME} proxmox-extent

### Associate Target + Extent
ctladm add -t proxmox-target -l proxmox-extent

!!! tip At this point, the LUN is live and can be discovered by initiators.

Connect from Proxmox VE Nodes

Perform the following on each Proxmox node.

# Install iSCSI Utilities
apt update
apt install -y open-iscsi lvm2

# Discover Target
iscsiadm -m discovery -t sendtargets -p <TRUENAS_IP>

# Log In
iscsiadm -m node --login

### Verify Device
lsblk

Create Shared LVM (One Node Only)

!!! warning "Important" Only run LVM creation on ONE node. All other nodes will only scan.

# Initialize Physical Volume
pvcreate /dev/sdX

# Create Volume Group
vgcreate vg_proxmox_iscsi /dev/sdX

Register Storage in Proxmox

Rescan LVM (Other Nodes)

pvscan
vgscan

Add Storage (GUI or CLI)

Datacenter → Storage → Add → LVM

• ID: iscsi-lvm
• Volume Group: vg_proxmox_iscsi
• Content: Disk image
• Shared: ✔️

Validation

• Snapshot create / revert / delete
• Live migration between nodes
• PBS backup and restore test

!!! success If all validation tests pass, the storage is production-ready.

Cutover-specific commands, not relevant to commands above if you are setting everything up new without NFS being a pre-existing thing.

Decommission NFS (After Cutover)

zfs destroy CLUSTER-STORAGE/NFS-STORAGE

Expand iSCSI Storage (No Downtime)

# Expand Zvol (TrueNAS)
zfs set volsize=16T CLUSTER-STORAGE/iscsi-storage
zfs set refreservation=16T CLUSTER-STORAGE/iscsi-storage

# Rescan on Proxmox Nodes
pvresize /dev/sdX