Add Servers/Virtualization/Proxmox/Configuring iSCSI-based Cluster Storage.md

Servers/Virtualization/Proxmox/Configuring iSCSI-based Cluster Storage.md

@@ -0,0 +1,220 @@
+## Purpose
+This document describes the **end-to-end procedure** for creating a **thick-provisioned iSCSI-backed shared storage target** on **TrueNAS CORE**, and consuming it from a **Proxmox VE cluster** using **shared LVM**.
+
+This approach is intended to:
+- Replace NFS-backed VM storage
+- Provide SAN-style block semantics
+- Enable Proxmox-native snapshot functionality (LVM volume chains)
+- Avoid third-party plugins or middleware
+- Be fully reproducible via CLI
+
+---
+
+## Assumptions
+- TrueNAS **CORE** (not SCALE)
+- ZFS pool already exists and is healthy
+- SSH service is enabled on TrueNAS
+- Proxmox VE nodes have network connectivity to TrueNAS
+- iSCSI traffic is on a reliable, low-latency network (10GbE recommended)
+- All VM workloads are drained from at least one Proxmox node for maintenance
+
+!!! warning "Important"
+    `volblocksize` **cannot be changed after zvol creation**. Choose carefully.
+
+---
+
+## Target Architecture
+
+```
+ZFS Pool
+ └─ Zvol (Thick / Reserved)
+    └─ iSCSI Extent
+       └─ Proxmox LVM PV
+          └─ Shared VG
+             └─ VM Disks
+```
+
+---
+
+## Step 1 – Create a Dedicated Zvol for Proxmox
+
+### Variables
+Adjust as needed before execution.
+
+```
+POOL_NAME="CLUSTER-STORAGE"
+ZVOL_NAME="iscsi-proxmox"
+ZVOL_SIZE="14T"
+VOLBLOCKSIZE="16K"
+```
+
+### Create the Zvol (Thick-Provisioned)
+```
+zfs create -V ${ZVOL_SIZE} \
+  -o volblocksize=${VOLBLOCKSIZE} \
+  -o compression=lz4 \
+  -o refreservation=${ZVOL_SIZE} \
+  ${POOL_NAME}/${ZVOL_NAME}
+```
+
+!!! note
+    The `refreservation` enforces **true thick provisioning** and prevents overcommit.
+
+---
+
+## Step 2 – Configure iSCSI Target (TrueNAS CLI)
+
+### Enable iSCSI Service
+```
+service iscsitarget start
+sysrc iscsitarget_enable=YES
+```
+
+---
+
+### Create iSCSI Portal
+```
+ctladm create -p 0.0.0.0
+```
+
+---
+
+### Create Target
+```
+ctladm create -t proxmox-target
+```
+
+---
+
+### Create Extent
+```
+ctladm create -b block \
+  -l /dev/zvol/${POOL_NAME}/${ZVOL_NAME} \
+  proxmox-extent
+```
+
+---
+
+### Associate Target + Extent
+```
+ctladm add -t proxmox-target -l proxmox-extent
+```
+
+!!! tip
+    At this point, the LUN is live and can be discovered by initiators.
+
+---
+
+## Step 3 – Connect from Proxmox VE Nodes
+
+Perform the following **on each Proxmox node**.
+
+### Install iSCSI Utilities
+```
+apt update
+apt install -y open-iscsi lvm2
+```
+
+---
+
+### Discover Target
+```
+iscsiadm -m discovery -t sendtargets -p <TRUENAS_IP>
+```
+
+---
+
+### Log In
+```
+iscsiadm -m node --login
+```
+
+---
+
+### Verify Device
+```
+lsblk
+```
+
+---
+
+## Step 4 – Create Shared LVM (One Node Only)
+
+!!! warning "Important"
+    **Only run LVM creation on ONE node**. All other nodes will only scan.
+
+### Initialize Physical Volume
+```
+pvcreate /dev/sdX
+```
+
+---
+
+### Create Volume Group
+```
+vgcreate vg_proxmox_iscsi /dev/sdX
+```
+
+---
+
+## Step 5 – Register Storage in Proxmox
+
+### Rescan LVM (Other Nodes)
+```
+pvscan
+vgscan
+```
+
+---
+
+### Add Storage (GUI or CLI)
+
+**Datacenter → Storage → Add → LVM**
+
+- ID: `iscsi-lvm`
+- Volume Group: `vg_proxmox_iscsi`
+- Content: `Disk image`
+- Shared: ✔️
+
+---
+
+## Step 6 – Validation
+
+- Snapshot create / revert / delete
+- Live migration between nodes
+- PBS backup and restore test
+
+!!! success
+    If all validation tests pass, the storage is production-ready.
+
+---
+
+## Step 7 – Decommission NFS (After Cutover)
+
+```
+zfs destroy CLUSTER-STORAGE/NFS-STORAGE
+```
+
+---
+
+## Step 8 – Expand iSCSI Storage (No Downtime)
+
+### Expand Zvol (TrueNAS)
+```
+zfs set volsize=16T CLUSTER-STORAGE/iscsi-proxmox
+zfs set refreservation=16T CLUSTER-STORAGE/iscsi-proxmox
+```
+
+---
+
+### Rescan on Proxmox Nodes
+```
+pvresize /dev/sdX
+```
+
+---
+
+## Final Notes
+- Maintain ≥15–20% free pool capacity
+- Monitor pool usage and fragmentation
+- Prefer PBS restores for large migrations