Good folders to exclude from Time Machine backups
Time Machine is a really effective tool for backing up everything on your drive. And when I say everything, I mean it — Time Machine indiscriminately backs up a lot of junk on your drive. And because of the techniques it uses and the way some files are used by your apps, you might find that your 500GB external backup drive is getting full after just a few months of use.
It’s really easy to identify folders for exclusion from Time Machine, which will save time and space during backups. It’s worth noting, though, that if your machine’s drive does fail, you won’t be able to execute a full restore without having some of these locations backed up. Then again, if you’re anything like me, you’re far more likely to just do a full reinstall and just restore your lost user data. (Doing a full restore from backup just doesn’t have many advantages compared to a full system reinstall, anyway.)
These are just a few suggestions — you should NEVER exclude anything from backup if you’re unsure you have another copy, aren’t able to get it again, or are unaware of the possible ramifications. Let me just say that again: if you aren’t completely sure about an exclusion, trust Time Machine to do its job.
Of course, you’ve got any good folders you think should be added to the list, feel free to drop ‘em in comments.
Obvious
The duh stuff.
- /Applications – This will likely save you more space than any other single folder, especially if you have lots of apps. Just make sure you know what programs you’d want to get back if your machine were to die.
Caches and downloads
Big directories of files that should probably be excluded automatically, but aren’t. Unless you’re a crazy developer debugging code, you’ll probably never need a backup of your caches.
- /Users/[user]/Library/Caches and /Library/Caches – Between the two you can knock off a few hundred megs of constantly changing, essentially useless data (for example: all the page caches from Firefox).
- /Users/[user]/Downloads – Where all your internet downloads wind up. Frequently changes, and if you’re anything like me, it’s filled with gigs of garbage.
- /Users/[user]/.Trash and /.Trashes – Some people might see value in backing up their trashed files. Not I.
More after the break.
Audio and media
Some more obvious picks. You probably don’t need to back this stuff up.
- /Library/Audio – You can save from the hundreds of megs to gigs here. Keep an eye out for the GarageBand samples, they weigh a ton.
- /Users/[user]/Music/iTunes/iTunes Music/Podcasts – I don’t really re-listen to podcasts, but even if I did, they can be re-downloaded easily. So the 3GB+ I save here is a no brainer.
Time Machine un-friendly apps
The data produced by some of these apps can easily be the worst offenders when it comes to backups. Until they get their act together, they should likely be excluded categorically.
- Parallels (/Users/[user]/Library/Parallels) and VMware (/Users/[user]/Documents/Virtual Machines)- You’ll definitely want to keep a copy of your virtual machines, but if left included in Time Machine backups, your system will continuously save the virtual disks each time you use Parallels or VMware. (VMware has become a little more TM-aware though, which is good.) That means potentially hundreds of wasted GB — not to mention tons of lost time during backup.
- Entourage (/Users/[user]/Documents/Microsoft User Data/Office 2008 Identities) – Same deal here, Entourage uses one giant, constantly changing file to store all your mail. It sucks not to have your email backed up though, so be sure to use something other than Time Machine for keeping a copy handy in case of emergency.
- /Users/[user]/Library/Mail Downloads – Whenever you open an attachment in Mail, it stores a copy of the file in your Mail.app attachments folder. If you’re a heavy Mail user, excluding this will save you some real space.
System
Beware! You likely won’t screw up anything by excluding this stuff, but if you run into trouble these are some pretty important folders. Note: you’ll need to hit “show invisible items” to exclude some of these.
- /usr, /sbin, /private, /bin – Collectively contain about a billion essential system files that can ultimately just be reinstalled in case of disaster.
- /system – Root system folder. Contains lots of stuff essential to OS X — gigs of it, in fact. Good place to save some space.
I'm an editor and technology critic in the midst of founding a new web startup: 
Yeah, I need to exclude my virtual machines from incremental backups. And I want to partition my USB drive so both Mac (Time Machine) and PC (Acronis) can write to it… hope that’s possible via AEBS.
When I ran Parallels, I let TM back it up. Little did I know that when I restored my machine, I would have to reinstall Parallels – and I could not for the life of me find my CD. I wasn’t all that happy with Parallels anyway, so it was a good excuse to move to VMWare.
I do backup my entire drive. I know it’s not space efficient. I did once do a TM restore of everything. It actually showed a lot of performance increase anyway, as TM already doesn’t backup some system files. But for everyone else, these seem like great suggestions!
Entourage:
User > [User] > Documents > Microsoft User Data > Office 2008 Identities
VMWare defaults to /Users/[user]/Documents/Virtual Machines/
I like having Time Machine backup my VM disks, it’s like having a disk image of Windows made every hour.
I hope Apple adds disk image backups to Time Machine in the future (like Windows Vista’s Complete PC Backup) as currently to go from dead hard drive to fully working Mac needs a lot more work then a disk image.
Great post. I agree to add the Entourage/Office identities.
Jamie and James, thanks!
@Jamie-
Careful there! Don’t you want to back up your email? There’s a database inside that folder that holds all your messages, not just useless cache and preferences.
I’m going to agree with our benevolent dictator Steve’s desicion to backup everything. Excluding files from Time Machine seems to defeat the purpose of what the program is designed to do – store all your backups easily in once place and to restore to exactly the way you had it. It would seem very unapple like to have a backup solution that required you to reinstall all your applications, restore your mail from a different location, and still have to possibly reinstall the OS. Those who want to finely tune what user data is backuped up are much more likely to already have a backup solution. I appreciate the article but would like more cred giving for why the vast majority of users should just let Time Machine do what its going to do.
Sometimes Time Machine backups simply take up too much space, which really sucks. When Leopard first launched, Aperture wasn’t compatible and Apple recommended you exclude your Aperture data files from backups — clearly they understand its limitations as a backup system. People aren’t going to just buy drive after drive to make sure they have 100 revisions of their virtual machines saved.
@Ryan Thanks for the quick reply.
I was unaware that Apple recommended that particular data be excluded. (Steve you make me sad) Still, I would argue that the problem is – if you have alot of data your going to need alot of space to back it up. I still agree with the idea of time machine as a simple all encompassing backup solution. I don’t believe that its a good solution, especially for non power users, to just backup less data. But this would be a discussion of if you should exclude data not what you should exclude, which was more the point of your article.
@Cosa
That’s kind of the whole point. Entourage backs up your email as a single file, so ANY activity within Entourage means a whole new backup in Time Machine.
My Entourage file is several gigs and to be writing that to disk every single hour is going to quickly run your external hard drive out of space.
So instead of leave as much mail as possible on my Exchange server and back the file up manually every week or so.
Interesting list. For most users, excluding the mail databases and ~/Library/Caches should be plenty — the /Applications tree is going to be mostly static for a conventional Mac user.
Very nice. I’ll have to revisit this when I pickup my Mac gear.
~/Library/Caches is already excluded (have a look at /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist)
You can also exclude alot of invisible items that your installed applications place in your home directory you never knew were there. I’m assuming many of these user specific files are things you could easily set up again if anything terrible ever happened.
I would suggest if there is a specific program that has the capability to export user settings (ie. ArchiCAD exports/imports xml user settings) that you utilize your customization settings and DO backup the folder that contains them.
Go to your Home Folder (Command Shift H)
Click on the “Show invisible items” so that it’s checked and then
Select all (Command A) then
Deselect the folders that are important to you (Command Click)
User settings in XML format not only take up less space, but are handy when trying to standardize multiple computers.
(High 5!)
Oh, and if you have EyeTV (and you should. It’s awesome), exclude the recordings folder. As awesome as I just exclaimed EyeTV to be, the recordings are massive. They’re MPEG-2, so it takes like 700MB for a half hour show. And then there’s the LiveTV buffer for when you pause it.
Not good for Time Machine. It sort of bleeds storage capacity.
Very interesting tips. Particularly the cache files.
However, I was wondering about the frequency of changes of some of these files. Obviously, the Microsoft User Data for Entourage will change daily if not hourly causing a potential huge backup every hour on the hour due to the way the data base is created.
I would hope if I backed up my system files once that they would not need backing up again for a while until an Apple update was issued. Same with hopefully most applications.
I am wondering if it shouldn’t be the reverse, i.e.
tell time machine to “include” specific folders (data) and not backup anything else. If the HDD gets hosed, i ‘ll reinstall from scratch. The only thing i cant reinstall is my personal data.
How does time-machine gen its backup list ? Is it an exclude or an include ?
[...] Just about everything Ryan Block suggests [...]
Man, can’t thank you enough for the tips. Backing up 32GB+ data everytime I touched parallels was making me going crazy. Keep up the good work mate.
Excellent tips Ryan! It’s weird, I just got my Mac and decided Time Machine would be useful, but I couldn’t make up my mind on what to back up. I started haphazardly googling for pointers, assuming I wouldn’t find anything concise…but lo and behold! :)
Removing the caches made the thing! I saved almost 1 GB of space.
Thank you, mate!
Time Machine does not, in fact back up everything. The list of standard exclusions is here:
/System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist
including some of the things you mention in the article. So manually adding caches and trashes is pointless.
I just started using time machine, and for the life of me can not figure out why I have 220 gigs on it when I only have about 30 on my hard drive on my imac, todaty I clicked back up now since some how the last was interuppted and it shows its backing up 85 more gb’s. Does it just back up whats new or every thing every time, seems redundant doesnt it. Please help my 1tb external will be filled in a week or so at this rate and I dont ever even get close to the 250 gb in my imac. as far as prefrences I cant seem to get to them for time machine. Scot
@scot – Time Machine backs up all changes to your drive, yes, but it keeps the old versions too. One of the features of Time Machine is being able to “step backwards” in time. If you activate Time Machine itself and go to a commonly edited folder, you’ll see what I mean.
So, of course, it’s gonna keep backing up all changes and keeping the very old stuff until you plain run out of room, when it prunes away the old data.
This is why my 500GB Studio drive is partitioned 50/50 so I can make sure to keep 250GB (around 216GB formatted) free for external storage.
Time Machine sysprefs are in the “System” group, btw.
-Nick
I have a 320gb hard drive,
and I want to limit Time Machine to using 250gb of that.
How would I do that?
[...] oder die Podcasts in der iTunes-Bibliothek, lässt sich trefflich und kompakt bei Ryan Block [...]
Great post, very helpful!
Thanks Ryan! Your advice on what files to exclude were very helpful and it also made my backups FLY!
Cheers!
Thanks Ryan, just what I was looking for. Podcasts and virtual machines saved the 55GB I needed. Now I can again fit my both my TM and full Carbon Copy back on my 500GB ext HD.
It appears that TM already excludes a bunch of stuff on its’ own, including some of the items listed above. I found a blogpost on Shifted Bits that gives the standard list of exclusion.
http://shiftedbits.org/2007/10/31/time-machine-exclusions/
Thanks for your help. Really useful and straightforward. Helped me out heaps. Now following you on Twitter so can stay more updated.
Cheers
Jenn
Apple says that “following the initial backup, Time Machine makes only incremental backups — copying just the files that have changed since the previous backup. Time Machine creates links to any unchanged files, so when you travel back in time you see the entire contents of your Mac on a given day.” When I set up and launched Time Machine yesterday, it appears that it made a new copy of EVERYTHING every hour! The first folder is the same size as the last one. I don’t see any links to unchanged files. and what about the daily backups for the past month, and weekly backups for everything older than a month? Sorry, I’m not getting this and there’s nothing in TM Help to explain it. Please enlighten…
TM makes so called hard links. A file exists on time in the Harddrive, but shows up on several places in the file system. In our case, in all the TM dated folders. Only if you change a file you will find different versions, therefor you use up some little space on the backup drive. If you have 10 timed folders in your backup, and you go by hand into all 10 folders to delete in each folder the identical file, only when the last file (e.g. 10MG) is deleted, the space is freed up.
Other way around calulate the size of your Files backed up, and multiply it with the numbers of backups there. You will see it does not match, TM uses much more less space. All this is possbile due to using HFS+.
I fould the ultimate solution for finding out what is backed up on each run. I was for example wondering why I have 22 MB on each backup, doing nothing else than Gmail and webbrowsing(it was my Firefox Profile). Use the Google code page of Nathan Fiedler, he offers 2 scripts:
1. Timedog to identify which folders are changing every hour.
2. Timecopy to move your old USB Time Machine Backup to the new place preserving the hard link mechanism.
Here is the Link:
http://code.google.com/p/timedog/
http://code.google.com/p/timedog/wiki/UsingTimecopy
You need:
1. Root Permission
2. confidence in terminal commands
3. If you run timecopy: time
They are Phyton scripts, readable and easy to understand.
Thanks to Nathan Fielder, my incremental backups are reduced in idle from 22.8 mb to kb, unless I modify larger files somewhere else which are on the TM Radar.
Share this to the world please!
You exclude /Applications? Honestly, if I have to do a restore, I’d much rather have all of my applications right there. It’s much more valuable to spend 10 minutes every week deleting apps that I installed and don’t use, than to re-install all of the applications if I have to restore that folder from a Time Machine backup.
i restored my whole system from Time Machine last week after reformatting my hard drive to partition it. I’d tried to use boot camp to repartition, but it didn’t work since i had a message that said i had files that couldn’t be moved. Anyway, after restoring from TM, none of my CS3 programs work, so i tried to reinstall. But it won’t let me, saying that the file system doesn’t work on my OS volume. I have no idea what this means. Any ideas on what i can do?
This article and the comments that followed were very helpful. I was extremely frustrated in trying to get TM to work. I had spent hours on the project, scouring the web for information and reformatting three different hard drives (journaled,journaled and case sensitive or Extended only). Determined not to give up, I finally reformatted the Seagate 1.5T drive as journaled, excluded the Parallels Folders, the caches, the Microsoft Office Identities Folders, Missing Synch Folders, Mark/ Space folder and the virus protection app, ClamXav. I still was not able to get TM to work. After reading the article above and another article that suggested files in the Library (Home) may hold the culprits, I began to systematically exclude the Library Folders. Using a method learned with the Conflict Catcher app in OS 8, I exclude half of the folder to see what happened and then the other half of the folders, if the first exclusion did not work. I made images of what files I had excluded on the desktop for a reference. The exclude files were highlighted in the images, so that I could compare what worked and what did not work. (Detailed written notes would do as well.) I finally narrowed the bad folded down to the “Livetype” folder. I did not tried to determine which files in that folder (LiveType) were causing the problem. I just excluded the whole folder. Remember to delete the backups that don’t work from TM and the trash each time you run your test or the backup hard drive will fill up. Hold down the Option key when starting to delete the files from the trash, so that the process of emptying the trash is not interrupted.
Someone should make an app like Conflict Catcher to identify the files that conflict with TM.
Addendum:
A little more research pointed me to, “TM Error Logger”. It tracks the logs for TM. Examining the logs generated by TM, I found that the backups were stopping when the LiveType folder was encountered. Starting with this app (TM Error Logger) first will save a great deal of time and headache. Reading the logs is a bit tricky. I did not know what all the error codes meant. A quick search in the TM Error Logger app and the web helped. TM Error Logger will only work when one has logged in as an administrator. Since it is recommended that one doesn’t use the admin account regularly (this is to protect your computer) you may have to sign into two accounts; an admin account and an non-admin account . This will depend on how you have your computer set up.
Cheers,
To exclude the above mentioned files/documents from TM, do you type in exactly as is what you suggested above?
How do you exclude Entourage in the preference window for TM? What steps fo you take?
RJM
I just found 80 gigs in my trash i just got a new 500 gb hdd for my mac book and put all my personal stuff back on and threw a lot in the trash
thanks a lot
Thank you for the tips.
Although more for a reduction in system resource usage than space, I’ve been using Time Machine Editor (free) for custom scheduling of TM. It has been working well so far.
I wish there was a TM setting to specify a maximum disk usage.
/Library/FileSync if you keep a local copy of your iDisk…
I have read a few postings and articles regarding Time Machine exclusions (including http://shiftedbits.org/2007/10/31/time-machine-exclusions/comment-page-1/ ), and I have not been able to figure out what has gone wrong since my upgrade to 10.6. If anyone happens upon this post and has some input, I would greatly appreciate it.
To avoid hassles with FileVault, I don’t use it. Instead, at login I mount an encrypted disk image that I use to hold all client data and I do not store the password in my keychain.
Under Leopard, Time Machine would backup the files inside this mounted volume just like any other drive/volume. I could exclude the folder where the disk image itself was stored (though backing up a sparse image could partially address this under Snow Leopard) so as not to have massive backups, and I could rely on Time Machine to make backups of the individual files, revisions, etc. inside the volume.
Under Snow Leopard, Time Machine seems to not even try to backup the contents of the mounted volume. I cannot quite figure out why or how this is excluded. Maybe I am doing something else wrong that I am not detecting. I did, for a variety of reasons (e.g., CrashPlan, etc.), make a new encrypted volume using a sparse image but I don’t see how that would cause the problem.
How does Time Machine know to backup external drive volumes that are mounted and yet not to backup volumes that are mounted from a disk image?
Any thoughts? Thanks very much.
If you use Adobe Lightroom. You’ll want to exclude from backups .lrdata file containing previews… it is essentially a cache (and Adobe *should* store that in ~/Library/Caches as they do for Adobe Bridge, but for some reason don’t for Lightroom). Also, if you’re using TimeMachine, you can turn off the Lightroom feature that automatically backs up your .lrcat file regularly… since TimeMachine is already creating versioned backups of the file.
Thunderbird is another culprit. I was getting over 1.5 GB added to each backup from that. I used a tool called “TimeTracker” and it worked great to root out where the large amounts of data were coming from.