4.4 KiB
Purpose: There is a way to incorporate ProxmoxVE and TrueNAS more deeply using SSH, simplifying the deployment of virtual disks/volumes passed into GuestVMs in ProxmoxVE. Using ZFS over iSCSI will give you the following non-exhaustive list of benefits:
- Automatically make Zvols in a ZFS Storage Pool
- Automatically bind device-based iSCSI Extents/LUNs to the Zvols
- Allow TrueNAS to handle VM snapshots directly
- Simplify the filesystem overhead of using TrueNAS and iSCSI with ProxmoxVE
!!! note "Environment Assumptions"
This document assumes you are running at least 2 ProxmoxVE nodes. For the sake of the example, it will assume they are named proxmox-node-01 and proxmox-node-02. We will also assume you are using TrueNAS Core. TrueNAS SCALE (should work) in the same way, but there may be minor operational / setup differences between the two different deployments of TrueNAS.
Secondly, this guide assumes the ProxmoxVE cluster nodes and TrueNAS server exist on the same network `192.168.101.0/24`.
All ProxmoxVE Cluster Nodes
Configure SSH Key Exchange
The first step is creating SSH trust between the ProxmoxVE cluster nodes and the TrueNAS storage appliance. You will leverage the ProxmoxVE shell to run the following commands.
apt-get install librest-client-perl git unzip # (4)
mkdir /etc/pve/priv/zfs
ssh-keygen -f /etc/pve/priv/zfs/192.168.101.100_id_rsa # (1)
ssh-copy-id -i /etc/pve/priv/zfs/192.168.101.100_id_rsa.pub root@192.168.101.100 # (2)
ssh -i /etc/pve/priv/zfs/192.168.101.100_id_rsa root@192.168.101.100 # (3)
iscsiadm --mode discovery --op update --type sendtargets --portal 192.168.101.100
iscsiadm -m node -T iqn.2005-10.org.moon-storage-01.ctl:proxmox-zfs-storage -p 192.168.101.100 -l
iscsiadm -m node # (5)
iscsiadm -m session # (6)
- Do not set a password. It will break the automatic functionality.
- Send the SSH key to the TrueNAS server.
- Connect to the TrueNAS server at least once to finish establishing the connection.
- Some prerequisites that need to be installed.
- Making sure that it's actually communicating with the iSCSI target successfully or not.
- Making sure that it's actually communicating with the iSCSI target successfully or not.
Install & Configure Storage Provider
Now you need to set up the storage provider in TrueNAS via a series of "patches" provided by the community. You will run the commands below within a ProxmoxVE shell, then when finished, log out of the ProxmoxVE WebUI, clear the browser cache for ProxmoxVE, then log back in. This will have added a new storage provider called FreeNAS-API under the ZFS over iSCSI storage type.
cd /tmp
wget https://github.com/TheGrandWazoo/freenas-proxmox/archive/refs/tags/v2.2.zip
unzip v2.2.zip
cd freenas-proxmox
patch -b /usr/share/pve-manager/js/pvemanagerlib.js < pve-manager/js/pvemanagerlib.js.patch
patch -b /usr/share/perl5/PVE/Storage/ZFSPlugin.pm < perl5/PVE/Storage/ZFSPlugin.pm.patch
patch -b /usr/share/pve-docs/api-viewer/apidoc.js < pve-docs/api-viewer/apidoc.js.patch
cp perl5/PVE/Storage/LunCmd/FreeNAS.pm /usr/share/perl5/PVE/Storage/LunCmd/FreeNAS.pm
systemctl restart pvedaemon
systemctl restart pveproxy
systemctl restart pvestatd
Primary ProxmoxVE Cluster Node
From this point, we are ready to add the shared storage provider to the cluster via the primary node in the cluster. This is not strictly required, just simplifies the documentation.
Navigate to "Datacenter (BUNNY-CLUSTER) > Storage > Add > ZFS over iSCSI"
| Field | Value | Additional Notes |
|---|---|---|
| ID | bunny-zfs-over-iscsi |
Friendly Name |
| Portal | 192.168.101.100 |
IP Address of iSCSI Portal |
| Pool | PROXMOX-ZFS-STORAGE |
This is the ZFS Storage Pool you will use to store GuestVM Disks |
| ZFS Block Size | 4k |
|
| Target | iqn.2005-10.org.moon-storage-01.ctl:proxmox-zfs-storage |
The iSCSI Target |
| Target Group | <Leave Blank> |
|
| Enable | <Checked> |
|
| iSCSI Provider | FreeNAS-API |
|
| Thin-Provision | <Checked> |
|
| Write Cache | <Checked> |
|
| API use SSL | <Unchecked> |
Disabled unless you have SSL Enabled on TrueNAS |
| API Username | root |
This is the account that is allowed to make ZFS zvols / datasets |
| API IPv4 Host | 192.168.101.100 |
iSCSI Portal Address |
| API Password | <Root Password of TrueNAS Box> |
|
| Nodes | proxmox-node-01,proxmox-node-02 |
All ProxmoxVE Cluster Nodes |