I need NFS 4.x storage for some tests. I would like to leverage FreeBSD 14.3 with ZFS and export two ZFS Datasets via NFS.
In this blog post, I will document how to setup NFS storage on FreeBSD.
ZFS Datasets
Here are my two ZFS datasets, I would like to export via NFS ...
root@ACS-Storage:~ #zfs listNAME USED AVAIL REFER MOUNTPOINT PRIMARY-STORAGE 360K 240G 96K /PRIMARY-STORAGE SUPPLEMENT-STORAGE 360K 23.7G 96K /SUPPLEMENT-STORAGE zroot 1.38G 3.95G 96K /zroot zroot/ROOT 1.38G 3.95G 96K none zroot/ROOT/14.3-RELEASE_2026-03-03_223333 8K 3.95G 1.18G / zroot/ROOT/default 1.38G 3.95G 1.20G / zroot/home 224K 3.95G 96K /home zroot/home/dpasek 128K 3.95G 128K /home/dpasek zroot/tmp 96K 3.95G 96K /tmp zroot/usr 288K 3.95G 96K /usr zroot/usr/ports 96K 3.95G 96K /usr/ports zroot/usr/src 96K 3.95G 96K /usr/src zroot/var 624K 3.95G 96K /var zroot/var/audit 96K 3.95G 96K /var/audit zroot/var/crash 96K 3.95G 96K /var/crash zroot/var/log 144K 3.95G 144K /var/log zroot/var/mail 96K 3.95G 96K /var/mail zroot/var/tmp 96K 3.95G 96K /var/tmp root@ACS-Storage:~ #
Enable NFSv4 services
I plan to use NFSv4 only, so here is the setup procedure.
sysrc rpcbind_enable="YES"
sysrc nfs_server_enable="YES"
sysrc mountd_enable="YES"
sysrc nfsv4_server_enable="YES"
sysrc nfsuserd_enable="YES"
Create NFS root dataset
Create a dedicated ZFS dataset.
zfs create -o mountpoint=/nfs zroot/nfsMove storage datasets under /nfs
zfs set mountpoint=/nfs/primary PRIMARY-STORAGE
zfs set mountpoint=/nfs/supplement SUPPLEMENT-STORAGE
Verify it ...
root@ACS-Storage:~ #zfs listNAME USED AVAIL REFER MOUNTPOINT PRIMARY-STORAGE 564K 240G 96K /nfs/primary SUPPLEMENT-STORAGE 648K 23.7G 96K /nfs/supplement zroot 1.38G 3.94G 96K /zroot zroot/ROOT 1.38G 3.94G 96K none zroot/ROOT/14.3-RELEASE_2026-03-03_223333 8K 3.94G 1.18G / zroot/ROOT/default 1.38G 3.94G 1.20G / zroot/home 224K 3.94G 96K /home zroot/home/dpasek 128K 3.94G 128K /home/dpasek zroot/nfs 96K 3.94G 96K /nfs zroot/tmp 96K 3.94G 96K /tmp zroot/usr 288K 3.94G 96K /usr zroot/usr/ports 96K 3.94G 96K /usr/ports zroot/usr/src 96K 3.94G 96K /usr/src zroot/var 636K 3.94G 96K /var zroot/var/audit 96K 3.94G 96K /var/audit zroot/var/crash 96K 3.94G 96K /var/crash zroot/var/log 156K 3.94G 156K /var/log zroot/var/mail 96K 3.94G 96K /var/mail zroot/var/tmp 96K 3.94G 96K /var/tmp root@ACS-Storage:~ #
Configure /etc/exports
For NFSv4 we must define a root exports.
vi /etc/exports
V4: /nfs
/nfs/primary -maproot=root -network 192.168.8.0/24
/nfs/supplement -maproot=root -network 192.168.8.0/24
Reboot and validate
Reboot server and verify that all necessary services are started and exports are configured.
reboot
Re-connect back to the server and do the verification.
service rpcbind status
service nfsd status
service mountd status
service nfsuserd status
Verify exports ...
showmount -e
root@ACS-Storage:~ #showmount -eExports list on localhost: /nfs/primary 192.168.8.0 /nfs/supplement 192.168.8.0 root@ACS-Storage:~ #
ZFS & NFS Optimizations
In this section I document NFS + ZFS tuning settings on FreeBSD that can significantly improve throughput for virtualization workloads (Proxmox / VMware / KVM).
Disable update of access time
Following setting disable update of atime. Atime is the timestamp that records when a file was last read.
zfs set atime=off PRIMARY-STORAGE
zfs set atime=off SUPPLEMENT-STORAGE
This reduces metadata writes and improves overall NFS performance. Disabling atime often improves performance 5–30% and reduces IOPS.
You can validate it by following command ...
zfs get atime
zfs get atime PRIMARY-STORAGE
Increase NFS server threads
Default FreeBSD NFS thread count is usually too low for VM workloads. Set more worker threads so multiple VMs can perform IO in parallel. For production storage, it is worth to do following NFS tunning ...
sysrc nfs_server_flags="-u -t -n 64"
Meaning:
- -u → UDP
- -t → TCP
- -n 64 → number of NFS server threads
For 64 threads, we should have 16 CPU Cores. It is best practice to have 4 Threads per 1 CPU Core.
Increase ZFS ARC memory
The ARC cache is critical for NFS performance because it caches VM blocks.
You can chech ARC status by command
zfs-stats -A
It is not part of base system and must be installed explicitely ...
pkg install zfs-stats
Here is the status of unused NFS server ...
root@ACS-Storage:~ #zfs-stats -A------------------------------------------------------------------------ ZFS Subsystem Report Wed Mar 4 00:19:10 2026 ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 25 Mutex Misses: 0 Evict Skips: 577 ARC Size: 5.17% 156.52 MiB Target Size: (Adaptive) 8.01% 242.55 MiB Min Size (Hard Limit): 4.18% 126.67 MiB Max Size (High Water): 23:1 2.96 GiB Compressed Data Size: 124.84 MiB Decompressed Data Size: 239.07 MiB Compression Factor: 1.91 ARC Size Breakdown: Recently Used Cache Size: 47.28% 114.69 MiB Frequently Used Cache Size: 52.72% 127.86 MiB ARC Hash Breakdown: Elements Max: 6.12 k Elements Current: 100.00% 6.12 k Collisions: 58 Chain Max: 1 Chains: 48 ------------------------------------------------------------------------
Use ZFS SLOG device (NVMe)
If your production NFS server has a SLOG device (NVMe used as write cache) you can dramatically improve synchronous writes and you disable asynchronous disk access.
zfs set sync=standard PRIMARY-STORAGE
But if it is a lab storage without SLOG, sometimes people use ASYNC mode ...
zfs set sync=disabled PRIMARY-STORAGE
Asynchronous disk access can increase performance massively but risks data loss on crash.
NFS Monitoring
NFS on server can be monitored by command ...
nfsstat -s
And you can also refresh statistics for example every second
nfsstat -s 1
And combined with ZFS stattistics to monitor load on backend disk ...
zpool iostat 1
Network throughput can be monitored by iftop which must be installed
pkg install iftop
Mounting NFS on clients
In this section, we will document how to mount NFS storage to various operating systems.
Linux Ubuntu
On Linux Ubuntu, NFS client is not part of minimally installed operating system.
apt install nfs-common
Mount manually
If NFS server is on IP address 192.168.8.105 we can mount it by following command ...
Persistent mount (/etc/fstab)
If NFS server is on IP address 192.168.8.105 and directories /mnt/primary and /mnt/suplement exists, we can add mount points to /etc/fstab ...
vi /etc/fstab
FreeBSD
On FreeBSD, NFS client is typically included in operating system.
Mount manually
If NFS server is on IP address 192.168.8.105 we can mount it by following command ...
Persistent mount (/etc/fstab)
If NFS server is on IP address 192.168.8.105 and directories /mnt/primary and /mnt/suplement exists, we can add mount points to /etc/fstab ...
vi /etc/fstab
Conclusion
NFS is great way to share storage to more clients. Hope this blog post help someone to play with NFS.
No comments:
Post a Comment