docs/Servers/Virtualization/Proxmox/Configuring iSCSI-based Cluster Storage.md

Purpose

This document describes the end-to-end procedure for creating a thick-provisioned iSCSI-backed shared storage target on TrueNAS CORE, and consuming it from a Proxmox VE cluster using shared LVM.

This approach is intended to:

• Provide SAN-style block semantics
• Enable Proxmox-native snapshot functionality (LVM volume chains)
• Avoid third-party plugins or middleware
• Be fully reproducible via CLI

Assumptions

• TrueNAS CORE (not SCALE)
• ZFS pool already exists and is healthy
• SSH service is enabled on TrueNAS
• Proxmox VE nodes have network connectivity to TrueNAS
• iSCSI traffic is on a reliable, low-latency network (10GbE recommended)
• All VM workloads are drained from at least one Proxmox node for maintenance

!!! note "Proxmox VE Version Context" This guide assumes Proxmox VE 9.1.4 (or later). Snapshot-as-volume-chain support on shared LVM (e.g., iSCSI) is available and improved, including enhanced handling of vTPM state in offline snapshots. :contentReference[oaicite:5]{index=5}

!!! warning "Important" volblocksize cannot be changed after zvol creation. Choose carefully.

Target Architecture

ZFS Pool
 └─ Zvol (Thick / Reserved)
    └─ iSCSI Extent
       └─ Proxmox LVM PV
          └─ Shared VG
             └─ VM Disks

Create a Dedicated Zvol for Proxmox

Variables

Adjust as needed before execution.

POOL_NAME="CLUSTER-STORAGE"
ZVOL_NAME="iscsi-storage"
ZVOL_SIZE="14T"
VOLBLOCKSIZE="16K"

Create the Zvol (Thick-Provisioned)

zfs create -V ${ZVOL_SIZE} \
  -o volblocksize=${VOLBLOCKSIZE} \
  -o compression=lz4 \
  -o refreservation=${ZVOL_SIZE} \
  ${POOL_NAME}/${ZVOL_NAME}

!!! note The refreservation enforces true thick provisioning and prevents overcommit.

Configure iSCSI Target (TrueNAS CORE)

This section uses a hybrid approach:

• CLI is used for ZFS and LUN (extent backing) creation
• TrueNAS GUI is used for iSCSI portal, target, and association
• CLI is used again for validation

Enable iSCSI Service

service ctld start
sysrc ctld_enable=YES

Create the iSCSI LUN Backing (CLI)

This step creates the actual block-backed LUN that will be exported via iSCSI.

# Sanity check: confirm the backing zvol exists
ls -l /dev/zvol/${POOL_NAME}/${ZVOL_NAME}

# Create CTL LUN backed by the zvol
ctladm create -b block \
  -o file=/dev/zvol/${POOL_NAME}/${ZVOL_NAME} \
  -S ISCSI-STORAGE \
  -d ISCSI-STORAGE

Verify the LUN is real and correctly sized

ctladm devlist -v

!!! tip Size (Blocks) must be non-zero and match the zvol size. If it is 0, stop and correct before proceeding.

Configure iSCSI Portal, Target, and Association (GUI)

In the TrueNAS Web UI, navigate to Sharing → Block Shares (iSCSI):

Create Portal

• Portals → Add
• IP Address: 0.0.0.0
• Port: 3260

Create Target

• Targets → Add
• Target Name: iqn.2026-01.io.bunny-lab:storage
• Authentication: None
• Portal Group: <Select the portal created above>

Create Extent

• Extents → Add
• Extent Name: ISCSI-STORAGE
• Extent Type: Device
• Device: /dev/zvol/${POOL_NAME}/${ZVOL_NAME} (e.g. CLUSTER-STORAGE/iscsi-storage (8T))

Associate Target + Extent

• Associated Targets → Add
• Target: iqn.2026-01.io.bunny-lab:storage
• Extent: ISCSI-STORAGE

Connect from Proxmox VE Nodes

Perform the following on each Proxmox node.

# Install iSCSI Utilities
apt update
apt install -y open-iscsi lvm2

# Discover Target
iscsiadm -m discovery -t sendtargets -p <TRUENAS_IP>

# Log In
iscsiadm -m node --login

### Verify Device
lsblk

Create Shared LVM (One Node Only)

!!! warning "Important" Only run LVM creation on ONE node. All other nodes will only scan.

# Initialize Physical Volume
pvcreate /dev/sdX

# Create Volume Group
vgcreate vg_proxmox_iscsi /dev/sdX

Register Storage in Proxmox

Rescan LVM (Other Nodes)

pvscan
vgscan

Add Storage (GUI or CLI)

Datacenter → Storage → Add → LVM

• ID: iscsi-lvm
• Volume Group: vg_proxmox_iscsi
• Content: Disk image
• Shared: ✔️

Validation

• Snapshot create / revert / delete
• Live migration between nodes
• PBS backup and restore test

!!! success If all validation tests pass, the storage is production-ready.

Cutover-specific commands, not relevant to commands above if you are setting everything up new without NFS being a pre-existing thing.

Decommission NFS (After Cutover)

zfs destroy CLUSTER-STORAGE/NFS-STORAGE

Expand iSCSI Storage (No Downtime)

# Expand Zvol (TrueNAS)
zfs set volsize=16T CLUSTER-STORAGE/iscsi-storage
zfs set refreservation=16T CLUSTER-STORAGE/iscsi-storage

# Rescan on Proxmox Nodes
pvresize /dev/sdX