Saturday, May 2, 2020

How to Use Apple's iCloud Drive service for Cold Storage

What iCloud Drive Is and What It Is Not


Apple’s iCloud Drive service is designed to allow syncing of content from one or more devices - iOS, Mac, and Windows - to Apple owned storage over the internet and facilitate sharing of that content between devices with approved Apple IDs.  Applications such as Apple’s Notes, which exists on iOS devices and is included with MacOS (formerly called Mac OSX) work well with iCloud, allowing authoring and editing of the same notes across devices sharing the same Apple ID.

Documents can be stored in iCloud by application, as is done with the Notes and Photos apps. In addition though a person can purchase additional space in iCloud - either 200 GB or 2 TB - for a monthly fee and this purchased space is referred to as iCloud Drive and it can be accessed and managed through the web site https://www.icloud.com/iclouddrive/. If you turn on the Documents & Desktop feature in iCloud, the entire user Documents and Desktop folders will be uploaded to iCloud Drive and relocated on the local machine to a hidden folder within the user’s Home folder:

/Library/Mobile Documents/com~apple~CloudDocs

Because the data remains upon the local drive in the Mobile Documents folder, using iCloud Drive does not free up any space on the local hard drive.

Content that is synced with the iCloud Drive service is not only stored in iCloud and on the local hard drive of the original source, but is also synced to each authorized device. Because storage capacity on devices vary, Apple allows an optimization feature to be used so that older documents are retained only in iCloud and can be downloaded upon request to a client device, but users have no control over the algorithm that is used. Any data files that are manually uploaded to iCloud Drive using the www.icloud.com/iclouddrive/ website are also automatically pushed to each device that has the iCloud Drive service turned on.  This can become a problem for computers that lack the storage capacity on their internal drive.

The Need for Cold Cloud Storage


While Apple does not provide a means of marking content in iCloud Drive so that it is retained in iCloud Drive only, there are several reasons why this is a desirable feature.

MacOS expects home folders to be stored on the Mac internal drive. Home folders can be relocated to an external drive, but then the Time Machine service would not be able to back them up. Photo, music, and video libraries can take up an enormous amount of space, and unless these are also on the internal drive they cannot be backed up by Time Machine. Also, the BootCamp functionality requires partitioning the internal drive to create a Windows partition and because the partition size cannot be changed once created, it makes sense to create a larger partition than you expect to need, leaving less space on the internal drive for MacOS.

An external drive can be used to solve the problem of a large video library consuming a large amount of space, but once those videos are moved to the external drive they will be unable to be backed up by Time Machine so the loss of the external drive would mean the loss of the content.  For this reason it is essential that content put onto an external drive be backed up in some other way. Uploading these files to iCloud Drive would result in the files being pushed to the internal drive Mobile Documents folder, consuming all the internal space that was gained by moving the files to the external drive in the first place and is therefore not a viable solution. If your Photo library is larger than 200 GB then you will require the 2 TB iCloud Drive space. If you cannot upload additional content to that space due to limited local internal drive space, then much of that 2 TB may go unused.

Unfortunately Apple has positioned iCloud Drive as a sync and sharing file service, not a cold backup service and they have not provided a means to mark content in iCloud Drive to be retained in iCloud Drive only.

How to Achieve Cold Storage Using iCloud Drive


The following method to utilize iCloud Drive space for cold storage has been tested on Mac OS X 10.14.6 and is not supported by Apple. The author recommends that you utilize this method only for content that you retain a local copy of on an external drive or other local storage. This method utilizes the Unix underpinnings of Mac OS X and is therefore likely to continue to work unless Apple takes specific steps to disable it. Note that this method does not interfere with any other uses of iCloud Drive such as for Photo and Notes sync and the Documents & Desktop feature.

This solution works only with Mac OS X clients, so if you have iOS or Windows iCloud Drive clients enabled this solution is probably not for you.

Requirements

This method has the following requirements:

- More than one account on your Macintosh, preferably an admin account and your user account (the user account need not but can have admin privileges).

- Access to your iCloud Drive space via the web site https://www.icloud.com/iclouddrive/

- The administrator password (needed only if your own account does not have admin privileges)

- Access to the Terminal utility

- Access to all Macintoshes on which you have enabled iCloud Drive for your Apple ID

Steps to create a folder in iCloud Drive that can be used for cold storage


1. Access your iCloud Drive space at https://www.icloud.com/iclouddrive/

2. Create and name a new folder in iCloud Drive to be used for cold storage by clicking the website folder+ icon. You could create multiple folders if you like but it will reduce steps if you use subfolders within that one rather than creating multiple folders. In my case I created a single new folder named “Cloud-only”. Devices that the user is logged into and have iCloud Drive turned on will automatically have a corresponding folder created in their Library/Mobile Documents/com~apple~CloudDocs folder with the name of the new folder created in iCloud Drive.

3. On the Mac start the Terminal utility that is in the Applications Utilities folder.

4. Type “cd ~/Library/Mobile\ Documents/com~apple~CloudDocsand return (without the quotes)

5. Type “ls -lart” and return to list the contents of the com~apple~CloudDocs folder. This list should contain the name of the folder that had been created in iCloud Drive.

In my case the directory listing looks like this (bold added for emphasis):

bbellina$ ls -lart
total 16
drwxr-xr-x@ 51 bbellina  staff  1632 Apr 11 12:28 ..
-rw-r--r--@  1 bbellina  staff  6148 Apr 28 00:10 .DS_Store
drwxr-xr-x  12 bbellina  staff   384 Apr 28 13:45 .Trash
drwxr-xr-x   2 bbellina  staff    64 Apr 28 13:45 Cloud-only
drwxr-xr-x   5 bbellina  staff   160 Apr 28 13:45 .

 6. The trick to this technique is to alter the folder on the Mac so that attempts to push content into the folder by the iCloud Drive service using your account will fail. To do this you change the owner of the folder from your own account to another account on the Mac that is not used with iCloud Drive.  In my case I changed the owner to my administrator account "TheAdmin".  Type the following command, replacing "TheAdmin" with the account you choose and "Cloud-only" with the name of the folder you created in iCloud Drive:

sudo chown TheAdmin Cloud-only/

7. You will be prompted for a password and you should enter your administrator password. If your user account has admin privileges then your own password should suffice.

Password:

8. After this retype the ls -lart command to verify that the change was made. It should look something like this. Note that the Cloud-only folder is now owned by the theadmin account rather than bbellina:

bbellina$ ls -lart
total 24
drwxr-xr-x@ 51 bbellina  staff  1632 Apr 11 12:28 ..
drwxr-xr-x   5 bbellina  staff   160 Apr 28 13:45 .
-rw-r--r--@  1 bbellina  staff  8196 Apr 28 13:58 .DS_Store
drwxr-xr-x  14 bbellina  staff   448 Apr 28 15:23 .Trash
drwxr-xr-x   2 theadmin  staff    64 Apr 28 15:23 Cloud-only


9. Close the Terminal

With the owner of the folder changed, the iCloud Drive service will be unable to push updates to the folder therefore any content that you upload to the folder using the iCloud drive website will be unable to be pushed to your Mac.  If there are other Macs using the same Apple ID and iCloud Drive then you should change the owner of the folder on those Macs as well before uploading any content to the folder in iCloud Drive.

Conclusion

Since implementing this I have successfully uploaded over 50 GB of content into subfolders within my Cloud-only folder and none of the files have been pushed to my Mac’s internal drive. The files can still be downloaded at any time to my Mac by going to the iCloud website and selecting them for download.

A few current limitations in iCloud Drive to keep in mind. These may be changed in the future.

- There does not seem to be a way to rename folders using the iCloud Drive website

- A file uploaded to iCloud Drive cannot be larger than 10 GB

- The iCloud Drive web interface does not allow drag and drop upload for a folder, only files.

- Most importantly, do not rely entirely on iCloud Drive to be your master location for these files. I have already had situations in which all of the files have been removed and must be re-uploaded. iCloud Drive should NEVER be your sole storage solution.

- [Update 5/8/20] iCloud Drive cannot handle packages like the OS X Installers. A solution is to compress such packages and upload the resulting zip.

- [Update 5/24/20] This has been verified to work with macOS 10.13, 10.14, and 10.15

- [Update 5/24/20] [See above warning about iCloud Drive never being your sole location for storage. It is possible for this procedure to delete all of the files in your cold storage folder in iCloud Drive. It happened to me.] If iCloud Drive has already started downloading content from your Cloud storage only folder to your Mac (this can happen on a new Mac or when using Mac Recovery to reinstall the OS) you should delete that content from the Mac before using the chown command.  To do so disconnect from the internet to stop the download and then in Terminal cd into the directory with the content to delete and then issue the command "rm -R -i *" (without the quotes). Because of the -i option you will be asked to confirm each delete. If you are certain you are in the correct directory you can leave off the -i and it will delete everything without asking for confirmation.  Note that -R tells the rm command to remove all directories in the directory as well as files.

B. Bellina
5/1/20

No comments:

Post a Comment