ZFS - Storage Provisioning

ZFS (Zettabyte File System) is a combined filesystem and volume manager, originally developed by Sun Microsystems for Solaris and now widely used on FreeBSD and other Unix-like systems.

In this blog post, we will describe ZFS and examples how to use it.

Key features of ZFS

• Storage pools (zpools)
• Instead of managing disks with partitions, RAID controllers, and logical volumes, ZFS groups disks into a storage pool. Filesystems draw space from this pool dynamically.
• No fixed partition sizes
• Easy disk expansion
• Flexible allocation 
• Built-in RAID (RAIDZ)
• ZFS includes its own software RAID
• RAIDZ1: single-parity (3 disks minimal; N+1)
• RAIDZ2: double-parity (4 disks minimal; N+2)
• RAIDZ3: triple-parity (5 disks minimal; N+3)
• Mirrors: like RAID1 (2 disks; 1+1), often used for performance
• Data integrity with checksums
• Every block is protected by checksums. ZFS can detect and automatically repair silent corruption (“bit rot”) as long as redundant copies exist.
• Copy-on-write
• ZFS never overwrites data in place. This protects against crashes and keeps data consistent at all times.
• Snapshots and clones
• Very fast, space-efficient, and great for backups, virtualization, or dev/test.
• Snapshots: read-only
• Clones: writable snapshots
• Compression & deduplication
• Transparent compression (lz4 is standard)
• Optional dedup (uses a lot of RAM)
• Self-healing
• When ZFS reads corrupted data, it tries to fix it from redundant copies automatically.

Creating ZFS pools

In this section, we will document few examples how to create ZFS pools. Before creating ZFS pools you have to list all your physical disks.

# Command to list all disks in the system

geom disk list

Now you can assign these disks into the ZFS pools.

# Create ZFS pool with single disk

zpool create DATA da0  

# Create ZFS pool with 2 mirrored disks (like RAID1)
zpool create DATA mirror da0 da1

# Create ZFS pool with 6 disks and Erasure Coding RAIDZ2 (like RAID6 ... 4+2)

zpool create DATA raidz2 da2 da3 da4 da5 da6 da7

 

# Create ZFS pool with 4 disks and two mirrors (like RAID10) 

zpool create DATA \
  mirror da0 da1 \
  mirror da2 da3 

 

# List of available ZFS pools

zpool list

 

# Check ZFS pool status

zpool status DATA

 

Write and Read caches for ZFS pools

In this section, we will document how to create ZFS pool caches.

# Create mirrored SLOG (write cache) from 2 disks for zpool DATA
zpool add DATA log mirror /dev/da8 /dev/da9
 

# Add 3 disks as L2ARC (read cache) for zpool DATA
zpool add DATA cache /dev/da10 /dev/da11 /dev/da12

 

Creating ZFS Datasets

When you create a new ZFS pool, ZFS automatically creates one dataset for you. That dataset has the same name as the pool. 

For example. If you run:

zpool create DATA mirror da0 da1

ZFS automatically creates ZFS Pool: DATA and ZFS Dataset: DATA 

Creating additional ZFS datasets on a zpool is very simple and one of the main advantages of ZFS. A dataset is like a sub-filesystem inside a pool, each with its own settings (compression, quotas, mountpoints, snapshots, etc.).

ZFS components are depicted in the schema below. 

ZFS Components

 

Below are examples how to create two ZFS datasets.

# Create zfs dataset VM on zfs pool DATA
zfs create DATA/VM

# Create zfs dataset S3 on zfs pool DATA
zfs create DATA/S3

# Create zfs dataset NFS on zfs pool DATA
zfs create DATA/NFS 

Each dataset can have its own settings. Let's document some examples how to set specific settings.

# Enable compression 

zfs set compression=lz4 DATA/VM

 

# Set a quota (limit maximum size) 

zfs set quota=200G DATA/NFS

 

# Set a reservation (guaranteed space)

zfs set reservation=50G DATA/NFS

 

# Set a custom mountpoint

zfs set mountpoint=/nfs/data DATA/NFS

 

# Enable NFS sharing

zfs set sharenfs=on DATA/NFS

 

Typical Dataset Layouts

The default ZFS recordsize is 128K and compression is not enabled.

Record sizes depends on particular workload. Below are documented typical record sizes for particular workloads.

 

Workload                        Recommended recordsize
General-purpose file server     128K
Media archive (large videos)    1M
Backup storage                  1M
Mixed documents + media         128K
Databases                       16K or 8K
VM disks                        16K 

 

Smaller recordsize = better for databases/VMs
Larger recordsize = better for large files/media

If you want to use compression and set recordsize, you can do it as a single command. 

zfs create -o compression=lz4 -o recordsize=16K DATA/VM
zfs create -o compression=lz4 -o recordsize=128K DATA/NFS

zfs create -o compression=lz4 -o recordsize=128K DATA/S3

 

Destroying ZFS Datasets

When you want to destroy a dataset you can use following commands.

# Destroy a dataset 

zfs destroy DATA/NFS

 

# Destroy recursively (datasets + snapshots)

zfs destroy -r DATA/NFS