zfs data replication¶

Introduction¶

The zfs replication scheme is based on zfs send/receive. OpenSVC uses ZFS snapshots to ensure incremental send/receive replication.

Source and targets maintain a zfs snapshot @sent to identify the last incremental synchronization done properly. This state is used to check that the deltas application sequence is not broken. If broken, alerts are emited and a full sync must be scheduled

Command set¶

syncnodes: Initial synchronization or incremental synchronisation. First one will be a Initial synchronisation, next will be incremental
syncupdate: Same

Status¶

up: Last synchronization occured less than sync_max_delay minutes ago.
warn: Last synchronization occured more than sync_max_delay minutes ago.

Service configuration¶

Service configuration file¶

[sync#1]
type = zfs
src = localpool/data
;dst is optional since it default value is src value (here dst = localpool/data)
;dst = anotherpool/data
target = nodes

Examples¶

Extract of z3.opensvc.com env file¶

[sync#1]
type = zfs
src@osol1.opensvc.com = osol1_z3_root/data
src@osol2.opensvc.com = osol2_z3_root/data
dst@osol1.opensvc.com = osol2_z3_root/data
dst@osol2.opensvc.com = osol1_z3_root/data
target = nodes
;recursive = True

Full synchronization¶

cgaliber@osol1.opensvc.com # pfexec svcmgr -s z3.opensvc.com syncupdate
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs snapshot -r osol1_z3@tosend
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs send -R osol1_z3_root@tosend | /usr/bin/ssh osol2.opensvc.com \
                                   /usr/sbin/zfs receive -dF osol2_z3_root
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh -n osol2.opensvc.com zfs rename -r osol2_z3_root@tosend osol2_z3_root@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs rename -r osol1_z3_root@tosend osol1_z3_root@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - update state file with snap uuid Sat Jun 19  2:55 2010
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/scp /var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state \
                                   osol2.opensvc.com:/var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state

Incremental synchronization¶

cgaliber@osol1.opensvc.com # pfexec svcmgr -s z3.opensvc.com syncupdate
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs snapshot -r osol1_z3_root/data@tosend
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs send -R -i osol1_z3_root/data@sent osol1_z3_root/data@tosend | \
                                   /usr/bin/ssh osol2.opensvc.com /usr/sbin/zfs receive -dF osol2_z3_root
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh osol2.opensvc.com /usr/sbin/zfs destroy -r osol2_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh osol2.opensvc.com zfs rename -r osol2_z3_root/data@tosend osol2_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs destroy -r osol1_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs rename -r osol1_z3_root/data@tosend osol1_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - update state file with snap uuid Sat Jun 19 14:30 2010
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/scp /var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state \
                                   osol2.opensvc.com:/var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state