cbbackupmgr network filesystems
Storing cbbackupmgr archive on centralized storage using network filesystems
A document which covers some important topics related to using cbbackupmgr with common network filesystems such as NFS and CIFS/SMB.
The cbbackupmgr tool uses SQLite when storing data on disk, SQLite has a read-modify-write update pattern; this means that for each key we insert into our SQLite index we have to perform a read to the underlying storage. For local filesystems, this isn’t significantly expensive because the operating systems filesystem cache is extremely likely to cache these repeated reads. This isn’t always the case for network filesystems.
On the most commonly used network filesystems, these reads will not be cached meaning the underlying client must perform one or more network operations resulting in significantly higher latency than would be experienced when using a local filesystem.
Some newer versions of network filesystems such as NFSv4 have better support for locking, for example delegation. However it’s worth noting that delegation isn’t a mandatory feature and it depends on whether the client/server supports it and actively accepts/grants delegations.
Newer versions of CIFS/SMB support opportunistic locking which provides similar functionality to NFS delegation i.e. allow cached read/writes. Note that again this is only supported in later versions of CIFS/SMB and depends on client/server support.
As stated above, cbbackupmgr uses SQLite when storing data on disk, internally SQLite uses file locking to guard access to the database and it is one of the features required to allow SQLite to expose its atomic commit behavior. This means there’s a few things to consider:
Some NFS server implementations only provide facade level locking, in some cases (for instance where other processes are accessing the archive) this can result in database corruption.
Some NFS server implementations have a limit on the number of locks that can be held at once. This means it’s likely that backups will only work when paired with a version greater than 6.6.0 (both the backup client/cluster).
Given that SQLite uses file locking to expose its atomic commit behavior, and
that most CIFS/SMB implementations don’t support requesting advisory bytes range
locks, you must mount your CIFS/SMB share using the
nobrl option. For more
information see mount.cifs(8).
During normal operation
cbbackupmgr will create and overwrite unix style
hidden files ("dotfiles"). Depending on the version of CIFS/SMB the config
hide dot files = no may need to be set otherwise these operations
will fail with
permission denied errors. Please note that by default, this
option will be set to
yes. When hitting
permission denied errors for
files beginning with a
. you should retry setting
hide dot files = no
before contacting support.
As with NFS, due care and attention should be taken to ensure that cbbackupmgr is the only process accessing the archive whilst it’s in use; this is because some systems don’t honor opportunistic locks and may therefore modify files which are actively locked, resulting in corruption.
Part of the cbbackupmgr suite