cbbackupmgr network filesystems

A document which covers some important topics related to using cbbackupmgr with common network filesystems such as NFS and CIFS/SMB.

The cbbackupmgr tool uses SQLite when storing data on disk, SQLite has a read-modify-write update pattern; this means that for each key we insert into our SQLite index we have to perform a read to the underlying storage. For local filesystems, this isn’t significantly expensive because the operating systems filesystem cache is extremely likely to cache these repeated reads. This isn’t always the case for network filesystems.

On the most commonly used network filesystems, these reads will not be cached meaning the underlying client must perform one or more network operations resulting in significantly higher latency than would be experienced when using a local filesystem.

Some newer versions of network filesystems such as NFSv4 have better support for locking, for example delegation. However it’s worth noting that delegation isn’t a mandatory feature and it depends on whether the client/server supports it and actively accepts/grants delegations.

Newer versions of CIFS/SMB support opportunistic locking which provides similar functionality to NFS delegation i.e. allow cached read/writes. Note that again this is only supported in later versions of CIFS/SMB and depends on client/server support.

As stated above, cbbackupmgr uses SQLite when storing data on disk, internally SQLite uses file locking to guard access to the database and it is one of the features required to allow SQLite to expose its atomic commit behavior. This means there’s a few things to consider:

Given that SQLite uses file locking to expose its atomic commit behavior, and that most CIFS/SMB implementations don’t support requesting advisory bytes range locks, you must mount your CIFS/SMB share using the nobrl option. For more information see mount.cifs(8).

During normal operation cbbackupmgr will create and overwrite unix style hidden files ("dotfiles"). Depending on the version of CIFS/SMB the config option hide dot files = no may need to be set otherwise these operations will fail with permission denied errors. Please note that by default, this option will be set to yes. When hitting permission denied errors for files beginning with a . you should retry setting hide dot files = no before contacting support.

As with NFS, due care and attention should be taken to ensure that cbbackupmgr is the only process accessing the archive whilst it’s in use; this is because some systems don’t honor opportunistic locks and may therefore modify files which are actively locked, resulting in corruption.

cbbackupmgr-backup, cbbackupmgr-restore, cbbackupmgr-merge,

Part of the cbbackupmgr suite