Movie HD for other OS Archives

You may not have heard of rsync; it’s a file transfer and synchronization program that’s often used to create elaborate and complex backup systems.

Written for Unix operating systems, rsync is included with the Mac and can be accessed directly from Terminal, or used within a number of scripting languages.

The rsync program has a number of features that make it a good candidate for building local, as well as remote, backup, archiving, and synchronization systems. It can also be used for basic file copying, and for maintaining file synchronization between one or more folders, either locally or with a remote system (think cloud-based storage, as an example).

In this Rocket Yard Guide, we’re going to concentrate on using rsync locally. If you wish to use rsync with a remote system, you’ll need to ensure that both the local system and the remote system have rsync installed.

If you’re looking for a copy of rsync to install on a system other than a Mac, or you&#;re just interested in discovering more about this versatile app, you can check out the rsync website.

Before we get into details about using rsync on the Mac, a note about versions. The version of rsync that’s distributed with the Mac tends to lag behind the current version available on the rsync website. The Mac version has been at for a number of years, while the current version is at (as of January, ). You should have no problems using the older Mac version with remote platforms that have one of the newer versions installed, but going the other direction could have unexpected results. Always check version compatibility when using rsync with remote systems.

Using Rsync
The Terminal app is used to invoke rsync and its various commands. If you&#;re new to using the Terminal app, check out the Rocket Yard series Tech Introduction to the Mac&#;s Terminal App.

Rsync uses a simple structure for issuing commands:

rsync -options theSourceDirectory theDestinationDirectory
While the number of options can get long, the format is always the same; the rsync command followed by any optional switches, then the source directory followed by the destination directory.

Let’s look at a basic rsync command that will copy a directory and all sub-directories it may contain. To tell rsync we want all the files and folders, including everything in subdirectories, we include the -r option. In the Terminal app, enter:

rsync -r /Users/tnelson/Desktop /Volumes/DocsBackup

(Replace tnelson with your user name, and DocsBackup with your desired target for the copy.)
In this example, my messy Desktop folderand its contents will be copied to a USB flash drivenamed DocsBackup. After the command is executed by hitting the return or enter key, the DocsBackup flash drive will have a new folder named Desktop, with all of my Desktop content.

If you want to copy only the contents of the Desktop, and not the parent folder named Desktop, you would add a forward slash after the directory named Desktop, like this:

rsync -r /Users/tnelson/Desktop/ /Volumes/DocsBackup

Terminal Tip:Terminal will figure out the directory pathnames for you. Enter the rsync command and any options at the Terminal prompt, followed by a space, then drag the source directory from a Finder window onto the Terminal window. The source directory pathname will be entered for you. Next, enter a space in Terminal to separate the source and destination directories, then drag the destination directory from a Finder window onto the Terminal window, and the destination pathname will be entered.

Time Stamps
When you copy files and folders to the destination directory, rsync uses the current time and date as the time stamp for the copied files. You can see this by opening a Finder window on the destination, and setting the Finder’s View to List mode. The creation and modification dates for the files will be set to the current time.

This is the default behavior for the rsync program, and having the files reset to the current time can be advantageous in some uses. But if you want to retain the original creation and modification data, you can, through the use of the -t option:

rsync -r -t /Users/tnelson/Desktop /Volumes/DocsBackup

In the above example, we’ve used both the -r and -t options to recursively copy all subdirectories as well as preserve the original time stamps.

rsync Tip:When using multiple option switches, you don’t have to use the dash symbol before each one. Instead, use a single dash, and place the options together. In the above example, rsync -r -t would become rsync -rt

Archive
Another commonly used rsync option is -a. This switch puts rsync into archive mode, which preserves time stamps, performs a recursive copy, keeps all file and directory permissions, preserves owner and group information, and copies any symbolic links.

Archive mode gets a lot of use when you wish to make backups as opposed to just syncing files in a directory. It&#;s a lot easier to remember the -a option than combining seven or so rsync options together (-rlptgoD) to perform the same task.

If I wanted to back up my Documents directory to the DocsBackup flash drive, I would use the -a (archive) option, as seen in this example:

rsync -a /Users/tnelson/Documents /Volumes/DocsBackup

Synchronize
Since sync is in rsync&#;s name, it must be a pretty good synchronization tool. The rsync command uses a special algorithm to compare files in the destination and source directories, and only copy the differences to the destination. There are options that allow you to specify how to handle discrepancies between the source and destination:

Delete Files in the Destination
When syncing directories, it’s not uncommon to have files in the destination directory that aren’t present in the source. rsync will normally leave those files alone, but you can also instruct rsync to delete files in the destination directory that don’t appear in the source. This can help ensure that both directories are identical:

rsync -rt &#;delete /Users/tnelson/Desktop /Volumes/DocsBackup

This new option, &#;delete, is two dashes plus the word delete. Sometimes, the double dash will be displayed as an em dash, so if you see a long dash instead of two short dashes, be sure to enter two dashes for this option.

The &#;delete option will remove any file or directory on the destination directory that isn’t present on the source.

A safer way to handle files or folders on the destination that aren’t present on the source is to back them up before they’re removed. rsync can do this for you using the -b and &#;backup-dir switches:

rsync -rtb &#;backup-dir=&#;backup $(date +\%Y-\%m-\%d)&#; &#;delete /Users/tnelson/Desktop /Volumes/DocsBackup

In the above command, the -b option has been added to -r and -t, creating the -rtb option. We’ve also used the &#;backup-dir command to specify the name of the backup folder to use. In this example, the backup folder will be named backup, and have the date, in the form of year, month, day, appended to its name.

You don&#;t have to append the date to the backup folder name but it helps when the same rsync backup script is used over and over. All that’s actually required for the &#;backup-dir command is a direct path to be specified, such as:

rsync -rtb &#;backup-dir=backup &#;delete /Users/tnelson/Desktop /Volumes/DocsBackup

In the above example, the backup directory named backup is created at the root level of the destination path, so after this command is executed, there will be a new folder named backup at the root of the DocsBackup flash drive.

I also make sure to put the backup option ahead of the delete option to ensure rsync handles the options in the correct order. You don&#;t want files to be deleted before they’re backed up.

Exclude Files in the Source
Sometimes there may be files or directories in the source that you don’t want to copy to the destination. You could manually move these files before an rsync copy, but an easier way is to use the exclude option:

rsync -a &#;exclude MyTestFolder /Users/tnelson/Desktop /Volumes/DocsBackup

In the above rsync command, the folder MyTestFolder, which is on the Desktop, will not be copied to the DocsBackup USB flash drive.

Exclude can use a file or folder name, but it also supports the use of regular expressions for pattern matching. Regular expressions and pattern matching would require their own article to do them justice; for now, I suggest opening another Terminal window (Shell, New Window) and entering the following at the Terminal prompt:

man re_syntax

Hit enter or return. This will display information about how to use regular expressions.

Commonly Used Rsync Options
To finish off our introduction to using rsync, let’s take a look at a few common options the command supports.

• -a &#; Enables archive mode, commonly used to back up a directory and retain all permissions, time stamps, symbolic links, etc.
• -b &#; Makes backups of any files in the destination directory that aren’t included in the source. Commonly used with &#;backup-dir and &#;delete switches.
• &#;backup-dir= &#; Allows you to define a name for the backup directory that will be used by the -b option.
• &#;delete &#; Removes files present in the destination directory that are absent from the source directory.
• -n &#; Dry run; no files are moved. Instead, rsync displays what the results would have been. Use -n with the -v option to gather additional information.
• -r – Recursive; forces rsync to transverse all subdirectories contained within the source directory and copy the subdirectories and their content to the destination directory.
• -t &#; Time stamps; preserves the existing time stamp on files and folders that are being copied.
• -m &#; Prunes empty directories. Directories that are empty aren’t copied.
• -z &#; Compress; uses compression during data transfer. Most often used when using rsync with a remote system.
• -v &#; Verbose; increases the level of information displayed by the rsync command. You can chain multiple v&#;s together to increase the information presented. Helpful for debugging, but not recommended for general use.

There are many additional rsync options and switches. To see the complete list, open a Terminal window and enter the following at the prompt:

man rsync

Press enter or return.

One of the best ways to understand how rsync works is to give the command a try with some dummy directories. Create a source directory with a few files and subdirectories in it, and try using rsync to copy them to a dummy (test) destination directory.

Be sure and try out the various options and switches and see how they affect the copy process. With a bit of tinkering, you can create customized backup solutions to meet your needs.

One of the niceties in the Mac OS is the ability of the system to display how your storage space is being used. Originally introduced in OS X Lion as part of a feature upgrade to the System Information app, the Storage tab displays how much free space is available on each disk attached to your Mac, as well as how a disk&#;s storage is being used, by category.

If you&#;re not sure how to find the categories, click the Apple logo in the menu bar in the Finder, select About This Mac from the dropdown menu, and then click the Storage tab. Mouse over a color bar, and a text balloon will pop up, telling you what the category is and how much disk space it&#;s using.

The categories change as new versions of the OS are released, but the current categories include System files, Documents, Apps, iTunes, Backups, Other, Audio, Music Creation, Photos, Movies, and Purgeable space.

Purgeable space is more or less unique to the Mac; you can find out more about it in the Rocket Yard Guide: What is Purgeable Space in macOS? The next category that seems to get a lot of attention is Other. So, just what is included in Other?

That&#;s a common question, one you may often ask yourself, especially when you need to free up space when your drive starts to fill up. When that happens, I think most of us tend to look at the Other category and think, there&#;s a lot of space being used by Other. If I don’t know what it is, and the system doesn&#;t seem to know either, then why can’t I just delete those files?

What&#;s in the Other Category
The type of files the operating system assigns to the Other category in the Storage display changes a bit from OS version to OS version, but there was a big shakeup in storage management when Apple introduced macOS Sierra, which caused the Other category to become better defined.

Because of the changes, we&#;re going to provide two lists for the type of files that are assigned to the Other category: pre macOS (OS X) and post macOS.

Other File Types in OS X
The Other category in OS X was a catchall; it included the following:

• PDF, doc, and PSD documents and files; essentially most document types that weren&#;t stored within the Documents folder in a user&#;s home folder.
• Disk images, archives, and compressed files, including zip, dmg, pkg, and iso.
• Files located in the System folder that weren&#;t installed with the system. This could include most temporary files, swap files, even voice files you added to personalize your Mac&#;s speaking voice.
• User installed items, such as screen savers, wallpaper, most iCloud support files, and Application Support files added along with app installations.
• Both user and system caches, browser caches, and other cache files created by various apps.
• App plug-ins, app extensions, and app add-ons; this includes all browser add-ons.
• Any file type not recognized by Spotlight. This includes Boot Camp files, Windows partitions and files, and virtual machine files.

The above list gives you an idea of what may be in the Other category under OS X, and why the Other category tended to be quite large sometimes.

Other File Types in macOS
When macOS was introduced, the Other category, along with the entire Storage system, received a bit of a makeover. Apple went through how files were allocated to categories, and moved many of the former Other files to more appropriate categories. This change was mostly made to accommodate the new Store in iCloud and Optimize Storage features that allowed infrequently used files to be moved to the cloud automatically and free up local space.

The Other category is now used for:

• Web browser caches and most caches created by apps.
• Apple Music streams.
• Photos previews.
• Most items in the Downloads folder, including installer packages.
• iOS device backups.

How to View the Amount of Storage Space Taken Up by Other
From the Apple menu, select About This Mac.

OS X Mavericks and earlier: Click the More Info button, and then select the Storage tab.

OS X Yosemite and later: Click the Storage tab.

A stacked bar graph will be displayed for each disk attached to your Mac. It may take a few moments for the bar graph to fill in. When complete, the graph will show how each disk&#;s storage is being used, broken down by category, including the Other category.

Removing Other Files
Chances are most of the space reported under Other is going to be coming from cache and temporary files, downloaded files, and iOS device backups. We&#;re going to concentrate on those three types, and show you how to remove their unwanted files.

Cache and Temporary Files: The Rocket Yard has a great guide to finding and removing cache and temp files, so we&#;re going to direct you to: Tech How to Clear Cache and Temp Files from Mac OS.

Downloaded Files: Some of us tend to download a lot of files. For me, it comes from trying out a lot of different apps. Most of the time, I don’t find the apps that useful and end up deleting them from the /Applications folder. But too many times, the installer package gets left behind in the Downloads folder. To delete the installers, or any other files you&#;ve downloaded and forgotten about, follow these steps:

Open a Finder window and select Downloads from the sidebar, or select Downloads from the Finder&#;s Go menu.

Examine the list of files and folders in the Downloads folder. To help you decide which files should be deleted, you can select the Size column to organize the list by the size of the files. This makes it easy to find the largest files that may be impacting your storage space.

When you find a file you wish to remove, right-click (or control-click) on the file and select Move to Trash from the popup menu. You can also drag the file directly to the trash.

Don’t forget to empty the trash to permanently remove the files from your Mac and free up disk space.

The Mac offers another method for removing downloaded files using the Reduce Clutter feature in the Manage Storage tool; we&#;ll cover that option below.

Device Backups: If you&#;re using an iOS device, chances are you have multiple backups of the device stored within your iTunes app. Device backups can be a big part of the Other category, depending on how many different devices you have backed up.

Launch iTunes, and select Preferences from the iTunes menu.

Select the Devices button in the Preferences windows toolbar.

Select the device backups you wish to delete from the list. It&#;s a good idea not to delete the most recent backup, unless the device has also been backed up to your iCloud account.

Make your selection(s), and then click the Delete Backup button.

Reduce Clutter
The Mac has a number of storage tools you can use to help manage your storage space. One of these is the Reduce Clutter tool, which can help you find, view, and delete files. To access the Reduce Clutter tool, perform the following steps:

Open the Storage view as outlined above, under the heading: How to View the Amount of Storage Space Taken Up by Other.

In the System Information window, make sure the Storage tab is selected, and then click the Manage button.

In the window that opens, select Recommendations, which can be found at the top of the sidebar.

Recommendations will reveal a number of options, including Store in iCloud, Optimize Storage, Empty Trash Automatically, and the option we&#;re interested in, Reduce Clutter.

Click the Review Files button in the Reduce Clutter section.

Select the Documents item in the sidebar.

There are three filters you can apply to help you in finding files to remove: Large Files, Downloads, and File Browser.

Large Files: This filter shows files on your Mac organized with the largest files displayed first, a pretty good way to make big changes to how much space is being taken up on your disk.

Downloads: This will display files in your Downloads folder. The files are organized in the display in groups: Older than 1 year, Older than 6 months, Older than 3 months, and Recent. Within each group, files are presented from largest to smallest.

File Browser: This sets up the display of files similar to how the Finder shows files in Column View mode. Files are organized by size, with the largest appearing on top.

When in the Large Files or Downloads view, you can examine a file or move it to the trash by hovering over the file name. After a moment, two small icons will be displayed: an X, which when clicked will move the selected file to the trash, and a magnifying glass, which when clicked will open a Finder window with the selected file highlighted. When in File Browser mode, right-clicking on a file will produce a popup menu with a single option, to move the file to the trash

Although the Reduce Clutter feature doesn&#;t offer an automatic method of removing unused or unwanted files, its filtering system lets you quickly discover the files that are taking up the most space; it also provides a convenient way to remove them.