docs/Servers/Virtualization/Proxmox/Configuring iSCSI-based Cluster Storage.md

Purpose

This document describes the end-to-end procedure for creating a thick-provisioned iSCSI-backed shared storage target on TrueNAS CORE, and consuming it from a Proxmox VE cluster using shared LVM.

This approach is intended to:

• Provide SAN-style block semantics
• Enable Proxmox-native snapshot functionality (LVM volume chains)
• Avoid third-party plugins or middleware
• Be fully reproducible via CLI

Assumptions

• TrueNAS CORE (not SCALE)
• ZFS pool already exists and is healthy
• SSH service is enabled on TrueNAS
• Proxmox VE nodes have network connectivity to TrueNAS
• iSCSI traffic is on a reliable, low-latency network (10GbE recommended)
• All VM workloads are drained from at least one Proxmox node for maintenance

!!! note "Proxmox VE Version Context" This guide assumes Proxmox VE 9.1.4 (or later). Snapshot-as-volume-chain support on shared LVM (e.g., iSCSI) is available and improved, including enhanced handling of vTPM state in offline snapshots. :contentReference[oaicite:5]{index=5}

!!! warning "Important" volblocksize cannot be changed after zvol creation. Choose carefully.

Target Architecture

ZFS Pool
 └─ Zvol (Thick / Reserved)
    └─ iSCSI Extent
       └─ Proxmox LVM PV
          └─ Shared VG
             └─ VM Disks

Create a Dedicated Zvol for Proxmox

Variables

Adjust as needed before execution.

POOL_NAME="CLUSTER-STORAGE"
ZVOL_NAME="iscsi-storage"
ZVOL_SIZE="14T"
VOLBLOCKSIZE="16K"

Create the Zvol (Thick-Provisioned)

zfs create -V ${ZVOL_SIZE} \
  -o volblocksize=${VOLBLOCKSIZE} \
  -o compression=lz4 \
  -o refreservation=${ZVOL_SIZE} \
  ${POOL_NAME}/${ZVOL_NAME}

!!! note The refreservation enforces true thick provisioning and prevents overcommit.

Configure iSCSI Target (TrueNAS CORE)

This section uses a hybrid approach:

• CLI is used for ZFS and LUN (extent backing) creation
• TrueNAS GUI is used for iSCSI portal, target, and association
• CLI is used again for validation

Enable iSCSI Service

service ctld start
sysrc ctld_enable=YES

Create the iSCSI LUN Backing (CLI)

This step creates the actual block-backed LUN that will be exported via iSCSI.

# Sanity check: confirm the backing zvol exists
ls -l /dev/zvol/${POOL_NAME}/${ZVOL_NAME}

# Create CTL LUN backed by the zvol
ctladm create -b block \
  -o file=/dev/zvol/${POOL_NAME}/${ZVOL_NAME} \
  -S ISCSI-STORAGE \
  -d ISCSI-STORAGE

Verify the LUN is real and correctly sized

ctladm devlist -v

!!! tip Size (Blocks) must be non-zero and match the zvol size. If it is 0, stop and correct before proceeding.

Configure iSCSI Portal, Target, and Association (GUI)

In the TrueNAS Web UI, navigate to Sharing → Block Shares (iSCSI) then proceed to perform the tasks seen in the tabs below, from left-to-right:

=== "Portal"

* **Portals → Add**
* IP Address: `0.0.0.0`
* Port: `3260`

=== "Target"

* **Targets → Add**
* Target Name: `iqn.2026-01.io.bunny-lab:storage`
* Authentication: `None`
* Portal Group: `<Select the portal created above>`

=== "Extent"

* **Extents → Add**
* Extent Name: `ISCSI-STORAGE`
* Extent Type: `Device`
* Device: `/dev/zvol/${POOL_NAME}/${ZVOL_NAME}` (e.g. `CLUSTER-STORAGE/iscsi-storage (8T)`)

=== "Associate Target & Extent"

* **Associated Targets → Add**
* Target: `iqn.2026-01.io.bunny-lab:storage`
* Extent: `ISCSI-STORAGE`

Connect from Proxmox VE Nodes

Perform the following on each Proxmox node.

# Install iSCSI Utilities
apt update
apt install -y open-iscsi lvm2

# Discover Target
iscsiadm -m discovery -t sendtargets -p <TRUENAS_IP>

# Log In
iscsiadm -m node --login

### Verify Device
lsblk

Create Shared LVM (One Node Only)

!!! warning "Important" Only run LVM creation on ONE node. All other nodes will only scan.

# Initialize Physical Volume
pvcreate /dev/sdX

# Create Volume Group
vgcreate vg_proxmox_iscsi /dev/sdX

Register Storage in Proxmox

Rescan LVM (Other Nodes)

pvscan
vgscan

Add Storage (GUI or CLI)

Datacenter → Storage → Add → LVM

• ID: iscsi-lvm
• Volume Group: vg_proxmox_iscsi
• Content: Disk image
• Shared: ✔️

Validation

• Snapshot create / revert / delete
• Live migration between nodes
• PBS backup and restore test

!!! success If all validation tests pass, the storage is production-ready.

Cutover-specific commands, not relevant to commands above if you are setting everything up new without NFS being a pre-existing thing.

Decommission NFS (After Cutover)

zfs destroy CLUSTER-STORAGE/NFS-STORAGE

Expand iSCSI Storage (No Downtime)

# Expand Zvol (TrueNAS)
zfs set volsize=16T CLUSTER-STORAGE/iscsi-storage
zfs set refreservation=16T CLUSTER-STORAGE/iscsi-storage

# Rescan on Proxmox Nodes
pvresize /dev/sdX