Do not run backup in parallel with the datastore garbage collection, as it might harm the results of both processes.
Generally it is recommended to run the Tar Optimizer after the backup has been performed.
You can always do an offline backup. This requires a downtime of CRX, but can be quite efficient in terms of required time compared to an online backup.
In most cases you will use a filesystem snapshot to create a read-only copy of the storage at that time. To create a offline backup perform these steps:
stop the application
make a snapshot backup
start the application
As the snapshot backup usually takes only a few seconds, the entire downtime is less than a few minutes.
Instead of stopping CRX, you can also use calls to JMX MBeans to prevent CRX from writing to the disk, which reduces the time CRX is unavailable to seconds. This is described under Filesystem Snapshot Backup below.
This backup method creates a backup of the entire repository, including any applications deployed under it, such as CQ5. The backup includes content, version history, configuration, software, hotfixes, custom applications, log files, search indexes, and so on. If you are using clustering and if the shared folder is a subdirectory of crx-quickstart (either physically, or using a softlink), the shared directory is also backed up.
You can restore the entire repository (and any applications) at a later point.
This method operates as a hot or online backup so it can be performed while the repository is running. Therefore the repository is usable while the backup is running. This method works for the default, TarPM-based, CRX instances.
When creating a backup, you have the following options:
Backing up to a directory using CRX's integrated backup tool.
Backing up to a directory using a filesystem snapshot
In any case, the backup creates an image (or snapshot) of the repository. Then the systems backup agent should take care to actually transfer this image to a dedicated backup system (tape drive).
The online backup only backs up the file system. If you store the repository content and/or the repository files in a database, that database needs to backed up separately.
CRX Online Backup
An online backup of your repository lets you create, download, and delete backup files. It is a "hot" or "online" backup feature, so can be executed while the repository is being used normally in the read-write mode.
When starting a backup you can specify a Target Path and/or a Delay.
The backup files are usually saved in the parent folder of the folder holidng the quickstart jar file (.jar). For example, if you have the CRX jar file located under /InstallationKits/CRX, then the backup will be generated under /InstallationKits. You can also specify a target to a location of your choice.
If the TargetPath is a directory, the image of the repository is created in this directory. If the same directory is used multiple times (or always) to storing backup,
modified files in the repository are modified accordingly in the TargetPath
deleted files in the repository are deleted in the TargetPath
created files in the repository are created in the TargetPath
If TargetPath is set to filename with the extension .zip, the repository is backupped to a temporary directory and then the content of this temporary directory is compressed and stored in the ZIP file.
This approach is discouraged, because
it requires additional disk storage during the backup process (temporary directory plus the zip file)
the compression process is done by the CRX and might influence its performance.
It delays the backup process.
Up to Java 1.6 Java is only able to create ZIP files up to a size of 4 gigabytes.
If you need to create a ZIP as backup format, you should backup to a directory and then use a compression program to create the zip file.
Indicates a time delay (in milliseconds), so that repository performance is not affected.
By default, the CRX backup runs at full speed. You can slow down creating an online backup, so that it does not slow down other tasks.
A delay of 1 millisecond typically results in 10% CPU usage, and a delay of 10 milliseconds usually results in less than 3% CPU usage. The total delay in seconds can be estimated as follows:
Repository size in MB * delay in milliseconds / 2 (if the zip option is used)
or / 4 (when backing up to a directory).
That means a backup to a directory of a 200 MB repository with 1 ms delay increases the backup time by about 50 seconds.
Click Start, a progress bar will indicate the progress of the backup.
You can Cancel a running backup at any time.
When the backup is complete the zip files are listed in the left pane of the console.
Backup files that are no longer needed can be removed using the console. Select the backup file in the left pane, then click Delete.
If you have backed up to a directory: after the backup process is finished CRX will not write to the target directory.
Automating CRX Online Backup
If possible, the online backup should be run when there is little load on the system, for example in the morning. By default the Tar PM optimization runs between 2 am and 5 am, which also slows down the system, that means a good time to run the online backup is 5 am.
Backups can be automated using the wget or curl HTTP clients. The following show examples of how to automate backup by using curl.
Backing up to the default Target Directory
In the following example various parameters in the curl command might need to be configured for your instance; for example, the hostname (localhost), port (4502), admin password (xyz) and file name (backup.zip).
curl -u admin:admin -X POST http://localhost:4502/system/console/jmx/com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=backup.zip
The backup file/directory is created on the server in the parent folder of the folder containing the crx-quickstart folder (the same as if you were creating the backup using the browser). For example, if you have installed CRX in the directory /InstallationKits/crx-quickstart/, then the backup is created in the /InstallationKits directory.
The curl command returns immediately, so you must monitor this directory to see when the zip file is ready. While the backup is being created a temp directory (with the name based on that of the final zip file) can be seen, at the end this will be zipped. For example:
name of resulting zip file: backup.zip
name of temporary directory: backup.f4d5.temp
Backing up to a non-default Target Directory
Usually the backup file/directory is created on the server in the parent folder of the folder containing the crx-quickstart folder.
If you want to save your backup (of either sort) to a different location you can set an absolute path to the target parameter in the curl command.
For example, to generate backupJune.zip in the directory /Backups/2012:
curl -u admin:admin -X POST http://localhost:4502/system/console/jmx/com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=/Backups/2012/backupJune.zip"
When using a different application server (such as JBoss), the online backup may not work as expected, because the target directory is not writable. In this case, please contact Support.
With CRX 2.3 it is possible to prevent writing to disk via a JMX call. While this is used internally by the CRX backup tool, it can also be used by an external process.
The process described here is specially suited for large repositories.
If you want to use this backup approach, your system must support filesystem snapshots. E.g. for Linux this means your filesystems should be placed on a logical volume.
Via JMX call the operation blockRepositoryWrites on the MBean com.adobe.granite:type=Repository.
This will prevent CRX from writing to disk. Instead all write requests are blocked.
Do a snapshot of the filesystem CRX is deployed on.
Via JMX call the operation unblockRepositoryWrites on the MBean com.adobe.granite:type=Repository.
The blocked write requests will continue to work.
Mount the filesystem snapshot, do a backup from there, unmount the snapshot.
To call unblockRepositoryWrites, you need to use a locally attached JMX client.
The JMX console is inappropriate in this case, as its availability is not guaranteed after blockRepositoryWrites has been called.
Backing Up the Data Store Separately
If the file data store has been configured outside the main repository, it is not included in the backup. This will reduce the size of the online backup and the backup directory. However, the data store needs to be backed up as well. Because files in the file data store directory are immutable, they can be backed up incrementally (potentially using rsync) or after running the online backup.
Do not run the data store backup and garbage collection concurrently.
How CRX Online Backup Works
CRX Online Backup is comprised of a series of internal actions to ensure the integrity of the data being backed up and the backup file(s) being created. These are listed below for those interested:
The online backup uses the following algorithm:
When creating a zip file, the first step is to create or locate the target directory.
If backing up to a zip file, a temporary directory is created. The directory name starts with backup. and ends with .temp; for example backup.f4d3.temp.
If backing up to a directory, the name specified in the target path is used. An existing directory can be used, otherwise a new directory will be created.
An empty file named backupInProgress.txt is created in the target directory when the backup starts. This file is deleted when the backup is finished.
All files are copied from the source directory to the target directory (or temporary directory when creating a zip file). The progress bar indicator of this sub-process is between 0% - 70% when creating a zip file, or 0% - 100% if no zip file is created.
If the backup is being made to a pre-existing directory, then "old" files in the target directory are deleted. Old files are files that do not exist in the source directory.
The files are copied to the target directory in four stages.
In the first copy stage (progress indicator 0% - 63% when creating a zip file or 0% - 90% if no zip file is created), all files are copied concurrently while the repository is running normally.
In the second copy stage (progress indicator 63% - 66.5% when creating a zip file or 90% - 95% if no zip file is created) only files that were created or modified in the source directory since the first copy stage was started are copied. Depending on the activity of the repository, this might range from no files at all, up to a significant number of files (because the first file copy stage usually takes a lot of time).
In the third copy stage (progress indicator 66.5% - 68.6% when creating a zip file or 95% - 98% if no zip file is created) only files that were created or modified in the source directory since the second copy stage was started are copied. Depending on the activity of the repository, there might be no files to copyl, or a very small number of files (because the second file copy stage is usually fast).
File copy stages one to three are all done concurrently while the repository is running. The fourth and last file copy stage will first lock repository write operations (write operations are paused; they do not throw an exception, but will wait). Only files that were created or modified in the source directory since the third copy stage was started are copied. Depending on the activity of the repository, there might be no files to copy, or a very, very small number of files (because the second file copy stage usually is very fast). After that, repository access continues. Progress indicator 68.6% - 70% when creating a zip file or 98% - 100% if no zip file is created.
Depending on the target:
If a zip file was specified, this is now created from the temporary directory. Progress indicator 70% - 100%. The temporary directory is then deleted.
If the target was a directory, the empty file named backupInProgress.txt is deleted to indicate that the backup is finished.
Restoring the Backup
This kind of restore restores the complete repository including the application, all content, logfiles, etc.
To restore the backup from a backup:
Restore a backup image on the system. if you backup the datastore separately, make sure, that the datastore is restored as well to the correct location.
In case you have created a backup as a zip file, unpack this zip file using:
jar -xvf backupJune.zip
On Unix systems, the "x"-bit of the following scripts are not preserved by the zip file:
You have to adjust these manually after restoring the backup.
Now the repository is ready to use. You can start it now using the regular start scripts.
This approach has some unique features:
additional disk consumption is low. A snapshot only consumes space if data on the original data is changed.
Snapshots are very fast and efficient, therefor the required time to create a snapshot is low (much lower than copying the data), as only metadata are duplicated.
To back up and restore content, you can use one of the following:
Package Manager, which uses the Content Package format to back up and restore content. The Package Manager provides more flexibility in defining and managing packages.
Content Zipper, which uses the CRX Package, XML Sys View Package, XML Doc View, or ZIP format to back up content.You restore content in these package formats with the Content Loader. See Creating a Backup using the Content Zipper and Restoring a Backup using the Content Loader.
When you back up nodes using either the Package Manager or the Content Zipper, CRX saves the following information:
The CRX repository content below the tree you have selected.
The Node type definitions that are used for the content you back up.
The Namespace definitions that are used for the content you back up.
When backing up, CRX loses the following information:
The version history.
Creating a Backup using the Content Zipper
To create the backup using the Content Zipper:
Lock the top node of the tree you want to back up or a parent node of that node.
In the Content Zipper, type the path of the tree you want to back up. For a format, click CRX package.
Click Submit Query. Your Web browser now offers the package file as a download. Save the download on your computer.
Unlock the node again.
The file you have downloaded contains the current version of the tree you have exported, including the node type and namespace definitions, but without the version history.
The CRX package file is Adobe’s proprietary file format for CRX node information. It is optimized for a small file size and optimal performance. If you prefer a standard XML file for further processing, click XML sys view in step 2. If you use the file only for archiving, use the CRX package format. Importing and Exporting Content describes the various file formats and their uses.
Restoring a Backup using the Content Loader
To restore the backup using the Content Loader:
Lock the node you want to restore. You can still modify the node and the nodes below it, but others cannot.
In the Content Loader, load the CRX package that you want to restore.
Unlock the node again.
You cannot restore the versioning history using the previous steps. CRX allows you to save the version history, but it does not currently support restoring it.