Creating Backups - Revision history

JPV at 08:24, 1 April 2023

2023-04-01T08:24:04Z

JPV at 16:54, 31 March 2023

2023-03-31T16:54:29Z

JPV at 16:47, 31 March 2023

2023-03-31T16:47:34Z

JPV at 15:49, 31 March 2023

2023-03-31T15:49:41Z

JPV at 14:42, 30 March 2023

2023-03-30T14:42:41Z

JPV at 14:41, 30 March 2023

2023-03-30T14:41:47Z

JPV at 14:08, 30 March 2023

2023-03-30T14:08:10Z

JPV at 13:54, 30 March 2023

2023-03-30T13:54:08Z

JPV at 13:23, 30 March 2023

2023-03-30T13:23:03Z

JPV: Created page with "== Things to Consider == There are a few things to keep in mind when planning a data backup operation on MorphOS. === Preserving File Attributes === MorphOS uses Amiga spec..."

2023-03-30T13:20:59Z

Created page with "== Things to Consider == There are a few things to keep in mind when planning a data backup operation on MorphOS. === Preserving File Attributes === MorphOS uses Amiga spec..."

New page

== Things to Consider ==

There are a few things to keep in mind when planning a data backup operation on MorphOS.

=== Preserving File Attributes ===

MorphOS uses Amiga specific file attributes that include [[Shell_Commands/Protect|protection flags]] and [[Shell_Commands/FileNote|file comments]]. These aren't preserved if files are copied directly into an "alien" filesystem (FAT, Ext, SMBFS, etc) or archiving format (Zip, Rar, etc.). Solutions for the issue is to use Amiga attributes aware filesystems, archivers, or backup programs directly, or encapsulate files into a compatible image file before copying or archiving the whole image file into an alien media.

It is especially important to preserve file attributes when making a backup of the system partition, and it would be a good practice for installed software in other locations too. It doesn't usually do any harm to copy generic data files into an "alien" destination, but even then you might lose file comments that indicate origin of the file, for instance.

=== Preserivng File Links ===

A source directory structure may contain file links that are like shortcuts to actual files or directories. The links can be either soft or hard links, but usually soft links in practice. The hard links aren't supported by all MorphOS filesystems, so it's quite safe to assume that all links are soft links.

There aren't any soft links in a default MorphOS installation, but the MorphOS Software Development Kit relies on them and it should be remembered when creating a backup. Users may also [[Shell_Commands/MakeLink | create soft links]] for their own use, for example, to link configuration or cache directories of programs to different locations ([[Tips_and_Tricks#How_to_Reduce_OWB.27s_HD_Writes|example]]).

If you don't take soft links into account when creating a backup, the soft links are either skipped or converted into real files, which means they take twice the space and lose their original purpose.

=== Destination File Size Limits ===

Backing-up into a single file, like an archive or image file, may result a large file that exceeds destination filesystem limits and causes the backup process to fail. The [[Filesystems]] page contains information about supported filesystems and their file size limits (the ''Max File Size'' column).

=== Be Careful with Content Changes While Backing-up ===

Backing-up a live system you have booted from is always a bit risky, or at least you should be totally sure that nothing (by your activity or background processes) writes to the data that is being backed-up, as this could be fatal, especially for partition/disk cloning by raw image files.

The best practice is to boot from another media to make backups. Booting from the [https://morphos-team.net/downloads MorphOS installation media] is a good choice, especially if the beta timeout is removed with the [https://www.morphos-storage.net/?find=registeriso RegisterISO] program.

== Cloning a Partition or Hard Disk as an Image File ==

The '''RAWDISK:''' device is an easy way to create an exact copy of a partition or hard disk. It can be used to browse devices for raw data of entire disks or separate partitions. Creating an image file is just a matter of copying a corresponding file to another location.

To access the RAWDISK: device, just type RAWDISK: in the Shell, or open an Ambient lister and press the / key to type RAWDISK: into the path string.

There are two directories at the root level of the RAWDISK: device. The Devices directory contains all raw data of each device and unit found in the system. For example, mass storage devices connected to the internal IDE controller of a computer could be found under the ide.device sub-directory and USB mass storage devices under the usbscsi.device sub-directory. The second directory at the root level is Disks, and it contains raw data images of mounted partitions. It isn't that useful for backups, as there's a risk that data on a partition changes while copying a backup.

=== Creating a Backup ===

To create a full backup of an entire disk, navigate to the ''RAWDISK:Devices/<your.device>/<unit>/'' directory. For example, a hard disk drive connected on the internal IDE port of a Mac mini computer would use the path ''RAWDISK:Devices/ide.device/0/''. The directory contains the rawdisk file, which is practically the raw data of the entire disk presented as a file. There's also a sub-directory that is named after the partition table of the disk (MAC, RDB, MBR), and that directory contains all partitions found on the disk as separate files.

In theory, a full backup of an entire disk would be just copying the ''rawdisk'' file to another location, but remember that the file size is in most cases huge, so keep the destination filesystem limits in mind. An ext2 or ext3 formatted backup drive will work fine, for instance.

Example:
Ram Disk:> Copy RAWDISK:Devices/ide.device/0/rawdisk BackupEXT2:harddisk.img

==== Pitfalls ====

If you want to mount partitions from a raw image file at some point, there's a pitfall in it. MorphOS doesn't play well when different partitions share the same volume name. For example, if the System: partition is mounted from both the original HDD and the backup image, there will be issues. A solution for it is to name the partitions differently in the original location and in the backup. Probably the easiest solution is to rename ([[Shell_Commands/Relabel|relabel]]) the original partitions before making the backup, and then rename them back to the original after a successful backup process.

Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. Booting from another media before making a backup is highly recommended, or being very careful otherwise.

=== Updating a Backup ===

An image file that has been copied from the RAWDISK: device can be mounted with the [[Tools/FileImageCtrl|FileImageCtrl]] tool, but remember the previous pitfall. If you don't always want to make a time consuming full image backup, you could use some other program, like MirrorCopy, to keep a mounted backup synced.

=== Restoring from a Backup ===

There are two ways to restore a raw image backup. Either the image can be mounted and any required files can be copied where desired, or the entire image file can be written to a storage device overwriting everything previously on it.

When writing a rawdisk image back to a device, it must be copied either with the [[Shell_Commands/Copy|Copy]] command with the '''DIRECT''' argument or with the cp command from the [https://morphos-team.net/downloads MorphOS SDK]. Ambient or 3rd party filemanagers don't work to do this.

'''Be very careful when selecting a destination device to write an image file!''' For example, when writing to a USB storage device, the unit number may always be different. '''Always check the correct destination device and unit''' from the RAWDISK: directory structure. The partition table directory under the unit directory offers a good hint.

== Copying Files Directly to a Destination ==

Opposite to raw image copying, backups can also be made at the filesystem level by copying individual files directly to the destination filesystem, but more attention must be paid for getting an exact copy or the original data.

=== Destination ===

==== Formatted Drive ====

An effortless solution for backups is to have a drive, or partition, that contains a compatible [[Filesystems|filesystem]]. As discussed at the start, a MorphOS compatible filesystem, like SFS or PFS, is needed to preserve file properties correctly.

USB or FireWire drives can also be partitioned and formatted with a compatible filesystem using the [[Tools/HDConfig|HDConfig]] tool. For example, change the partition map from the gadget before the drive name to RDB and add new partition(s). They are set for SFS by default, but it's recommended to change the name to something more original and increase the default buffer value slightly. After saving, [[Tools/Format|format]] the partition(s) with ''Quick Format'', and decide if you need the ''Recycled'' directory in this kind of use. For USB back-ups a HDD with an external PSU is the safest bet, but it doesn't mean that USB sticks wouldn't work instead.

==== Image File ====

Another option is to create an image file that is formatted with a compatible filesystem, and then copy files on the mounted image file. In this way the backup can be stored to an alien filesystem, like FAT formatted USB devices or Samba shares, without losing file attributes of original files. File size limits of the destination filesystem must be kept in mind though.

There are several ways to create an image file for backing-up:

* A base backup image can be copied from the RAWDISK: device as described previously in the article.
* Create a new empty file with the [[Shell_Commands/FileWrite|FileWrite]] command, mount it as '''MorphOS SFS''' in [[Tools/FileImageCtrl|FileImageCtrl]], and Quick Format it (maybe with Recycled disabled).
* Create an encrypted disk image with the [[Applications/Kryptos|Kryptos]] application.

=== Copying Files ===

==== The Copy Command and Ambient File Copying ====

The [[Shell_Commands/Copy|Copy command]] from the Shell or drag&drop copying from Ambient can be used to make a full base backup, but they aren't that good for syncing the backup later. These also don't work with file links properly, because links are converted to regular files, which doesn't produce an exact copy of MorphOS SDK, for instance.

When using the Copy command from the Shell, remember to use the '''CLONE''' argument with it to ensure all protection bits, dates, etc. are copied just as in source files.

Example:
Ram Disk:> Copy System: Backup:System/ ALL CLONE QUIET

==== Third Party Programs ====

There are many third party programs for backing-up, mirroring, incremental copying, etc.. There are a couple of examples listed.

===== MirrorCopy =====

Leif Salomonsson's [https://www.morphos-storage.net/?find=mirrorcopy MirrorCopy] (don't mix it up with the original 68k program of the same name!) is a very comprehensive mirroring tool for MorphOS. It can be used to make exact copies of directories where files that are deleted from the source directory will be removed from the backup directory too. It handles file links properly, which makes it useful for backing up the directory structure that contains the SDK directory.

Updating a backup is much faster than with the standard Copy command, because only changed/added/removed files are updated in the existing backup directory. With the Copy command you'd have to copy all the files over and over again.

'''Note that with the default options, all files that don't exist in the source directory will be deleted from the destination directory! Be very careful when supplying the destination directory (TO) for the progam!''' If you have separate directories for each partition on your backup destination, and you mistakenly give the root directory as the destination for MirrorCopy, all other backups will be lost!

Examples:

Create an exact mirror of a directory (non-existing files will be deleted):
Ram Disk:> mirrorcopy FROM System: TO Backup:System/ ALL

Copy changed and new files, but don't purge deleted files from the destination:
Ram Disk:> mirrorcopy FROM Work:Stuff/ TO Backup:Stuff/ ALL NODEL

Move deleted files to another directory instead of deleting them completely:
Ram Disk:> mirrorcopy FROM System: TO Backup:System/ ALL DELDIR Backup:SystemDeleted

See MirrorCopy's documentation for more options.

===== BackUp =====

[https://aminet.net/package/disk/bakup/BackUp BackUp] by Daniel Westerberg is a 68k backing-up/mirroring program, but it works nicely on MorphOS too. It has an easy to use GUI, but can also be used from the Shell or scripts.

BackUp supports soft links and has the '''Clean''' option to delete destination files that don't exist in the source anymore. It is a valid alternative for MirrorCopy if you prefer a GUI based program. Also, remember to '''set the destination correctly if using the Clean option to avoid loss of unintentional files!'''

== Creating a Backup Archive ==

Backing-up into archives is yet another option, but it is generally slower than other options, at least if compression is involved. The resulting backup size can be smaller though, and there may be error checking included.

With archivers it's also important to use ones that will preserve all file attributes, which disqualifies Zip and most other mainstream archivers.

Instructions presented here are valid only for programs (commands) provided within MorphOS or [https://morphos-team.net/downloads MorphOS SDK]. Options and results may vary if using third party replacements for archiving or unarchiving.

=== LhA ===

LhA is a de-facto archiver on MorphOS (and Amiga), which was made to handle all related files and their properties, but it has its quirks and it doesn't support file links either. So don't use it for backing-up the MorphOS SDK.

LhA has tens of options and here are some that are important to achieve a successful backup:
-e Archive empty directories
-r Collect files recursively
-x Preserve and use path names
-H Write header level ( 0,1,2 )
-z Do not compress files

It's safer to increase the header level from the default (0) to, for example, -H1, because some deeper file paths/comments may be too long and so be skipped with the -H0 level.

Disabling the compression creates backups faster, but increases the resulting file size.

Example:
System:> lha -H1 -e -x -r a Backup:System.lha #?

=== Tar ===

The separately installable [https://morphos-team.net/downloads MorphOS Software Development Kit] (SDK) comes with a Tar version that will preserve MorphOS file arguments and it also handles the problematic file links correctly.

To preserve file attributes and comments with Tar, the '''--xattrs''' argument must be provided when creating and extracting an archive. '''Note''' that this only works with the Tar version from the SDK, it doesn't work with the 3rd party ports of Tar.

As Tar originates from other platforms, it still has some peculiarities. For example, files/directories starting with a dot will need some extra magic or compromises.

Example:
System:> tar --xattrs -cf Backup:System.tar #?

== Creating an ISO Image Backup ==

[[Applications/Jalapeno|Jalapeno]] is a CD/DVD authoring tool that is bundled with MorphOS, and it can be used to create ISO image files for backup purposes. It supports MorphOS file attributes and comments, but it doesn't support file links and contents can't be updated.

An option to create an ISO image can be accessed via the pulldown menu in the '''Burn Files''' section. If the resulting image file is larger than 1GB, Jalapeno splits it into 1GB parts. Those parts can be joined into one big image file with the [[Shell_Commands/Join|Join]] command in the Shell, but remember the filesize limits of your target filesystem.

Restoring files from an ISO image is easy, the image file can be browsed in Ambient or mounted as a drive from the RMB context menu.

After making an ISO image of the System partition, it's even possible to boot the computer from it if it resides on a compatible drive/filesystem, just like booting from the MorphOS installation ISO image.

== Custom Backup Programs and Formats ==

There are some other less standard options for backing-up using dedicated programs, like [https://www.morphos-storage.net/?find=diavolo Diavolo]. They may have some extra features like scheduling and comparing of backups, but they usually save backups into custom formats that require that specific program to be accessed. Many of these programs are also outdated for today's standards and may not support file links, etc.. If you're not completely familiar with the programs, it might be better to make backups on formats that are more common and accessible in the future to be sure.

== Validating a Backup ==

When a backup has been created, it's important to check its validity, as there might have been data that wasn't copied successfully. It could be something you didn't consider when selecting a backup method or other unexpected situations.

The minimum you should do is to check if file/directory sizes of source and destination paths match. For a more thorough test, a separate program that compares directories and data in them should be used.

The [https://www.morphos-storage.net/?find=mirrorcheck MirrorCheck] program is a good option to compare directories. It can compare file data (reads entire files), file name casing, owner info, protection bits, comments, dates, sizes, and existence of files. Each of these checks can be skipped if there's no need to compare certain things or you get too many reports of unimportant tests. For a quick check the file data comparison can be disabled or if you're making incremental backups, the existence check can be skipped in either or both directories.

Examples:

Full check:
Ram Disk:> mirrorcheck System: Backup:System/ QUIET

Skip the file data check:
Ram Disk:> mirrorcheck System: Backup:System/ SKIP="F" QUIET

Skip Name case, Owner info, and Date checks:
Ram Disk:> mirrorcheck System: Backup:System/ SKIP="NOD" QUIET

@@ Line 23: / Line 23: @@
 === Be Careful with Content Changes While Backing-up ===
-Backing-up a live system you have booted from is always a bit risky, or at least you should be totally sure that nothing (by your activity or background processes) writes to the data that is being backed-up, as this could be fatal, especially for partition/disk cloning by raw image files.
+Backing-up a live system you have booted from is always a bit risky, or at least you should be totally sure that nothing (by your activity or background processes) writes to the data that is being backed-up, as this could be fatal, especially for partition/disk cloning by raw image files. It's highly advisable to unmount partitions first when making raw image backups.
 The best practice is to boot from another media to make backups. Booting from the [https://morphos-team.net/downloads MorphOS installation media] is a good choice, especially if the beta timeout is removed with the [https://www.morphos-storage.net/?find=registeriso RegisterISO] program.
@@ Line 48: / Line 48: @@
 If you want to mount partitions from a raw image file at some point, there's a pitfall in it. MorphOS doesn't play well when different partitions share the same '''volume name'''. For example, if the System: partition is mounted from both the original HDD and the backup image, there will be issues. A solution for it is to name the partitions differently in the original location and in the backup. Probably the easiest solution is to rename ([[Shell_Commands/Relabel|relabel]]) the original partitions before making the backup, and then rename them back to the original after a successful backup process.
-Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. The best practise is to '''unmount partitions''' before copying them from the RAWDISK: device. Booting from another media before making a backup is highly recommended, or being very careful otherwise.
+Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. The best practise is to '''unmount partitions''' before copying them from the RAWDISK: device. Unmounting can be done, for example, with the [[Tools/Mounter|Mounter]] tool. Booting from another media before making a backup is highly recommended, or being very careful otherwise.
 === Updating a Backup ===

@@ Line 48: / Line 48: @@
 If you want to mount partitions from a raw image file at some point, there's a pitfall in it. MorphOS doesn't play well when different partitions share the same '''volume name'''. For example, if the System: partition is mounted from both the original HDD and the backup image, there will be issues. A solution for it is to name the partitions differently in the original location and in the backup. Probably the easiest solution is to rename ([[Shell_Commands/Relabel|relabel]]) the original partitions before making the backup, and then rename them back to the original after a successful backup process.
-Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. The best practise is to unmount partitions before copying them from the RAWDISK: device. Booting from another media before making a backup is highly recommended, or being very careful otherwise.
+Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. The best practise is to '''unmount partitions''' before copying them from the RAWDISK: device. Booting from another media before making a backup is highly recommended, or being very careful otherwise.
 === Updating a Backup ===

@@ Line 48: / Line 48: @@
 If you want to mount partitions from a raw image file at some point, there's a pitfall in it. MorphOS doesn't play well when different partitions share the same '''volume name'''. For example, if the System: partition is mounted from both the original HDD and the backup image, there will be issues. A solution for it is to name the partitions differently in the original location and in the backup. Probably the easiest solution is to rename ([[Shell_Commands/Relabel|relabel]]) the original partitions before making the backup, and then rename them back to the original after a successful backup process.
-Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. Booting from another media before making a backup is highly recommended, or being very careful otherwise.
+Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. The best practise is to unmount partitions before copying them from the RAWDISK: device. Booting from another media before making a backup is highly recommended, or being very careful otherwise.
 === Updating a Backup ===

@@ Line 177: / Line 177: @@
 == Custom Backup Programs and Formats ==
-There are some other less standard options for backing-up using dedicated programs, like [https://www.morphos-storage.net/?find=diavolo Diavolo]. They may have some extra features like scheduling and comparing of backups, but they usually save backups into custom formats that require that specific program to be accessed. Many of these programs are also outdated for today's standards and may not support file links, etc.. If you're not completely familiar with the programs, it might be better to make backups on formats that are more common and accessible in the future to be sure.
+There are some other less standard options for backing-up using dedicated programs, like [https://www.morphos-storage.net/?find=diavolo Diavolo]. They may have some extra features like scheduling and comparing of backups, but they usually save backups into custom formats that require that specific program to be accessed. Many of these programs are also outdated for today's standards and may not support file links (enable the ''3.1 or better'' filesystem option in Diavolo for the file link support), etc.. If you're not completely familiar with the programs, it might be better to make backups on formats that are more common and accessible in the future to be sure.
 == Validating a Backup ==

@@ Line 137: / Line 137: @@
 === LhA ===
-LhA is a de-facto archiver on MorphOS (and Amiga), which was made to handle all related files and their properties, but it has its quirks and it doesn't support file links either. So don't use it for backing-up the MorphOS SDK.
+[[Shell_Commands/LhA|LhA]] is a de-facto archiver on MorphOS (and Amiga), which was made to handle all related files and their properties, but it has its quirks and it doesn't support file links either. So don't use it for backing-up the MorphOS SDK.
 LhA has tens of options and here are some that are important to achieve a successful backup:

@@ Line 9: / Line 9: @@
 It is especially important to preserve file attributes when making a backup of the system partition, and it would be a good practice for installed software in other locations too. It doesn't usually do any harm to copy generic data files into an "alien" destination, but even then you might lose file comments that indicate origin of the file, for instance.
-=== Preserivng File Links ===
+=== Preserving File Links ===
 A source directory structure may contain file links that are like shortcuts to actual files or directories. The links can be either soft or hard links, but usually soft links in practice. The hard links aren't supported by all MorphOS filesystems, so it's quite safe to assume that all links are soft links.

@@ Line 140: / Line 140: @@
 LhA has tens of options and here are some that are important to achieve a successful backup:
 -e  Archive empty directories
 -r  Collect files recursively
 -x  Preserve and use path names
 -H  Write header level ( 0,1,2 )
--z  Do not compress files
+-z  Do not compress files</nowiki>
 It's safer to increase the header level from the default (0) to, for example, -H1, because some deeper file paths/comments may be too long and so be skipped with the -H0 level.

@@ Line 52: / Line 52: @@
 === Updating a Backup ===
-An image file that has been copied from the RAWDISK: device can be mounted with the [[Tools/FileImageCtrl|FileImageCtrl]] tool, but remember the previous pitfall. If you don't always want to make a time consuming full image backup, you could use some other program, like MirrorCopy, to keep a mounted backup synced.
+An image file that has been copied from the RAWDISK: device can be mounted with the [[Tools/FileImageCtrl|FileImageCtrl]] tool, but remember the previous pitfall. If you don't always want to make a time consuming full image backup, you could use some other program, like [[Creating_Backups#MirrorCopy|MirrorCopy]], to keep a mounted backup synced.
 === Restoring from a Backup ===

@@ Line 46: / Line 46: @@
 ==== Pitfalls ====
-If you want to mount partitions from a raw image file at some point, there's a pitfall in it. MorphOS doesn't play well when different partitions share the same volume name. For example, if the System: partition is mounted from both the original HDD and the backup image, there will be issues. A solution for it is to name the partitions differently in the original location and in the backup. Probably the easiest solution is to rename ([[Shell_Commands/Relabel|relabel]]) the original partitions before making the backup, and then rename them back to the original after a successful backup process.
+If you want to mount partitions from a raw image file at some point, there's a pitfall in it. MorphOS doesn't play well when different partitions share the same '''volume name'''. For example, if the System: partition is mounted from both the original HDD and the backup image, there will be issues. A solution for it is to name the partitions differently in the original location and in the backup. Probably the easiest solution is to rename ([[Shell_Commands/Relabel|relabel]]) the original partitions before making the backup, and then rename them back to the original after a successful backup process.
 Also, contents of partitions that are being backed-up shouldn't change while creating a backup, as it can break the filesystem structure. Booting from another media before making a backup is highly recommended, or being very careful otherwise.