zfs data replication

Introduction

The zfs replication scheme is based on zfs send/receive. OpenSVC uses ZFS snapshots to ensure incremental send/receive replication.

Source and targets maintain a zfs snapshot @sent to identify the last incremental synchronization done properly. This state is used to check that the deltas application sequence is not broken. If broken, alerts are emited and a full sync must be scheduled

Command set

sync nodes

Initial synchronization or incremental synchronisation. First one will be a Initial synchronisation, next will be incremental

sync update

Same

Status

up

Last synchronization occured less than sync_max_delay minutes ago.

warn

Last synchronization occured more than sync_max_delay minutes ago.

Service configuration

Keywords

• sync.zfs
  • dst
  • intermediary
  • recursive
  • src
  • tags
  • target
  • blocking_post_provision
  • blocking_post_start
  • blocking_post_startstandby
  • blocking_post_stop
  • blocking_post_sync_drp
  • blocking_post_sync_nodes
  • blocking_post_sync_restore
  • blocking_post_sync_resync
  • blocking_post_sync_update
  • blocking_post_unprovision
  • blocking_pre_provision
  • blocking_pre_start
  • blocking_pre_startstandby
  • blocking_pre_stop
  • blocking_pre_sync_drp
  • blocking_pre_sync_nodes
  • blocking_pre_sync_restore
  • blocking_pre_sync_resync
  • blocking_pre_sync_update
  • blocking_pre_unprovision
  • comment
  • disable
  • encap
  • monitor
  • optional
  • pg_blkio_weight
  • pg_cpu_quota
  • pg_cpu_shares
  • pg_cpus
  • pg_mem_limit
  • pg_mem_oom_control
  • pg_mem_swappiness
  • pg_mems
  • pg_vmem_limit
  • post_provision
  • post_start
  • post_startstandby
  • post_stop
  • post_sync_drp
  • post_sync_nodes
  • post_sync_restore
  • post_sync_resync
  • post_sync_update
  • post_unprovision
  • pre_provision
  • pre_start
  • pre_startstandby
  • pre_stop
  • pre_sync_drp
  • pre_sync_nodes
  • pre_sync_restore
  • pre_sync_resync
  • pre_sync_update
  • pre_unprovision
  • provision
  • provision_requires
  • restart
  • restart_delay
  • schedule
  • shared
  • standby
  • start_requires
  • stop_requires
  • subset
  • sync_break_requires
  • sync_drp_requires
  • sync_max_delay
  • sync_nodes_requires
  • sync_restore_requires
  • sync_resync_requires
  • sync_update_requires
  • tags
  • unprovision
  • unprovision_requires

Examples

Extract of z3.opensvc.com env file

[sync#1]
type = zfs
src@osol1.opensvc.com = osol1_z3_root/data
src@osol2.opensvc.com = osol2_z3_root/data
dst@osol1.opensvc.com = osol2_z3_root/data
dst@osol2.opensvc.com = osol1_z3_root/data
target = nodes
;recursive = True

Full synchronization

cgaliber@osol1.opensvc.com # pfexec om z3.opensvc.com sync full
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs snapshot -r osol1_z3@tosend
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs send -R osol1_z3_root@tosend | /usr/bin/ssh osol2.opensvc.com \
                                   /usr/sbin/zfs receive -dF osol2_z3_root
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh -n osol2.opensvc.com zfs rename -r osol2_z3_root@tosend osol2_z3_root@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs rename -r osol1_z3_root@tosend osol1_z3_root@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - update state file with snap uuid Sat Jun 19  2:55 2010
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/scp /var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state \
                                   osol2.opensvc.com:/var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state

Incremental synchronization

cgaliber@osol1.opensvc.com # pfexec om z3.opensvc.com sync update
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs snapshot -r osol1_z3_root/data@tosend
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs send -R -i osol1_z3_root/data@sent osol1_z3_root/data@tosend | \
                                   /usr/bin/ssh osol2.opensvc.com /usr/sbin/zfs receive -dF osol2_z3_root
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh osol2.opensvc.com /usr/sbin/zfs destroy -r osol2_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/ssh osol2.opensvc.com zfs rename -r osol2_z3_root/data@tosend osol2_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/sbin/zfs destroy -r osol1_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - zfs rename -r osol1_z3_root/data@tosend osol1_z3_root/data@sent
* Z3.OPENSVC.COM.SYNC#1 - INFO - update state file with snap uuid Sat Jun 19 14:30 2010
* Z3.OPENSVC.COM.SYNC#1 - INFO - /usr/bin/scp /var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state \
                                   osol2.opensvc.com:/var/lib/opensvc/z3.opensvc.com_sync#1_zfs_state