R1Soft is a great backup solution because it allows you to do incremental (block device) backups down to every 15 minutes. But we all know that no software is perfect, and sometimes the lack of information can give frustrations because the fixes are not always that easy to find. One of them which I’ve seen happen is “unable to open snapshot device ‘/dev/cdp1′”.
The error was something like this:
[2016-Feb-08 08:22:07],[DEBUG],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 0; Unknown error 512 [2016-Feb-08 08:22:08],[DEBUG],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 1; Unknown error 512 [2016-Feb-08 08:22:09],[DEBUG],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 2; Unknown error 512 [2016-Feb-08 08:22:10],[DEBUG],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 3; Unknown error 512 [2016-Feb-08 08:22:11],[DEBUG],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 4; Unknown error 512 ... [2016-Feb-08 08:24:09],[DEBUG],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 58; Unknown error 512 [2016-Feb-08 08:24:10],[DEBUG],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 59; Unknown error 512 [2016-Feb-08 08:24:11],[ERROR],[CDP_SERVER:48477],Unable to open snapshot device '/dev/cdp1' retry 60; Unknown error 512 [2016-Feb-08 08:24:11],[ERROR],[CDP_SERVER:48477],Exception: Snapshot device /dev/cdp1 for device /dev/md126p5 did not settle after 60 seconds. Aborting snapshot. [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],Snapshot device information missing. Unable to unmount snapshot device. [2016-Feb-08 08:24:11],[ INFO],[CDP_SERVER:48477],Stopping snapshot for device '/dev/md126p5' with id 1 try 0 [2016-Feb-08 08:24:11],[ INFO],[CDP_SERVER:48477],Snapshot for device '/dev/md126p5' with id 1 stopped! [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],backup failed! [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],disabling tracking session! [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],Pausing change tracking. [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],Change tracking paused. [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],Checking partiton table for device '/dev/md126p5' [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],Checking partiton table for device '/dev/md126p5' [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],Resuming change tracking. [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],Change tracking resumed. [2016-Feb-08 08:24:11],[ERROR],[CDP_SERVER:48477],An exception occurred during the request. Backup Failed [2016-Feb-08 08:24:11],[DEBUG],[CDP_SERVER:48477],sending command response request ERROR
I once contacted R1Soft about this, and they had no clue how to fix it only ‘remove the hcp sessions that are running and unmount the hcp device manually’ – problem was, no hcp session was active, so yeah..
After some debugging I found out that there’s a race condition (which doesn’t happen to most users of R1Soft) – but it comes down to some systems making use of jailed shell environments (such as cPanel), this is mainly for improving the security on the systems.
What happens behind the scenes is that the cdp agent are not aware of virtual mounts existing such as /home/virtfs/$user/cdp1 – this will cause backups to fail when this happens. The resolution is relatively simple, and can be done with a one-liner, but first be sure that there’s no active backup running, you can do this by running
[bash]hcp -l[/bash]
After this execute following
[bash]cat /proc/mounts | grep cdp | awk ‘{print $2}’ | xargs umount[/bash]
And rerun your r1soft backup, and it should all be working again.
It sucks you have to do this, but at least better than the suggested idea by r1soft to reboot the server when it happens.