zfs data replication¶
Introduction¶
The zfs replication scheme is based on zfs send/receive. OpenSVC uses ZFS snapshots to ensure incremental send/receive replication.
Source and targets maintain a zfs snapshot @sent to identify the last incremental synchronization done properly. This state is used to check that the deltas application sequence is not broken. If broken, alerts are emited and a full sync must be scheduled
Command set¶
- sync nodes
Initial synchronization or incremental synchronisation. First one will be a Initial synchronisation, next will be incremental
- sync update
Same
Status¶
- up
Last synchronization occured less than sync_max_delay minutes ago.
- warn
Last synchronization occured more than sync_max_delay minutes ago.
Service configuration¶
Keywords¶
- sync.zfs
- dst
- intermediary
- recursive
- src
- tags
- target
- blocking_post_provision
- blocking_post_start
- blocking_post_startstandby
- blocking_post_stop
- blocking_post_sync_drp
- blocking_post_sync_nodes
- blocking_post_sync_restore
- blocking_post_sync_resync
- blocking_post_sync_update
- blocking_post_unprovision
- blocking_pre_provision
- blocking_pre_start
- blocking_pre_startstandby
- blocking_pre_stop
- blocking_pre_sync_drp
- blocking_pre_sync_nodes
- blocking_pre_sync_restore
- blocking_pre_sync_resync
- blocking_pre_sync_update
- blocking_pre_unprovision
- comment
- disable
- encap
- monitor
- optional
- pg_blkio_weight
- pg_cpu_quota
- pg_cpu_shares
- pg_cpus
- pg_mem_limit
- pg_mem_oom_control
- pg_mem_swappiness
- pg_mems
- pg_vmem_limit
- post_provision
- post_start
- post_startstandby
- post_stop
- post_sync_drp
- post_sync_nodes
- post_sync_restore
- post_sync_resync
- post_sync_update
- post_unprovision
- pre_provision
- pre_start
- pre_startstandby
- pre_stop
- pre_sync_drp
- pre_sync_nodes
- pre_sync_restore
- pre_sync_resync
- pre_sync_update
- pre_unprovision
- provision
- provision_requires
- restart
- restart_delay
- schedule
- shared
- standby
- start_requires
- stop_requires
- subset
- sync_break_requires
- sync_drp_requires
- sync_max_delay
- sync_nodes_requires
- sync_restore_requires
- sync_resync_requires
- sync_update_requires
- tags
- unprovision
- unprovision_requires
Examples¶
Extract of z3.opensvc.com env file¶
[sync#1]
type = zfs
src@osol1.opensvc.com = osol1_z3_root/data
src@osol2.opensvc.com = osol2_z3_root/data
dst@osol1.opensvc.com = osol2_z3_root/data
dst@osol2.opensvc.com = osol1_z3_root/data
target = nodes
;recursive = True
Full synchronization¶
cgaliber@osol1.opensvc.com # pfexec om z3.opensvc.com sync full
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs snapshot -r osol1_z3@tosend
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs send -R osol1_z3_root@tosend | /usr/bin/ssh osol2.opensvc.com \
/usr/sbin/zfs receive -dF osol2_z3_root
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh -n osol2.opensvc.com zfs rename -r osol2_z3_root@tosend osol2_z3_root@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs rename -r osol1_z3_root@tosend osol1_z3_root@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - update state file with snap uuid Sat Jun 19 2:55 2010
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/scp /var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state \
osol2.opensvc.com:/var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state
Incremental synchronization¶
cgaliber@osol1.opensvc.com # pfexec om z3.opensvc.com sync update
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs snapshot -r osol1_z3_root/data@tosend
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs send -R -i osol1_z3_root/data@sent osol1_z3_root/data@tosend | \
/usr/bin/ssh osol2.opensvc.com /usr/sbin/zfs receive -dF osol2_z3_root
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh osol2.opensvc.com /usr/sbin/zfs destroy -r osol2_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh osol2.opensvc.com zfs rename -r osol2_z3_root/data@tosend osol2_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs destroy -r osol1_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs rename -r osol1_z3_root/data@tosend osol1_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - update state file with snap uuid Sat Jun 19 14:30 2010
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/scp /var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state \
osol2.opensvc.com:/var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state