Since joining climateprediction.net 2 days ago, a GB or so is now added to my backups from my iMac. I want to stop my Time Machine backing up BOINC files - I already use Options in Time Machine to exclude selected folders from being backed up. However I have not been able to identify (in Finder) these additional folders/files deposited by BOINC.

Please advise.

---

Hi Barry, welcome to the forum.

The BOINC Programs directory should be here:

Macintosh: /Applications/BOINC/

and the BOINC Data directory here:

Macintosh OS X: /Library/Application Support/BOINC Data/

The Programs directory will increase in size either not at all or hardly because that's where the BOINC platform is. The climate models are in BOINC Data where the files will increase in size as the models progress.

You can read more about BOINC 7 here if you have the time and inclination:

http://boincfaq.mundayweb.com/index.php?language=1&view=592&sessionID=b3b9ebf825ad6aa1fffef8b017a7a9d8

By the way, your very nice computer seems to be an all-in-one and probably ultra-slim. Is the cooling good? Check the CPU temperature with all four of your models running. I would be interested to know what you find.

---

Cpdn news

I would like to expand on this question a bit. I have disabled the time machine backup for the climate data models as suggested in this thread. If I have a disk failure, will all work on those models be lost? My disk is starting to make noises. What would be a prudent ongoing procedure to save & later restore progress without maintaining an incremental time machine history?

Hard disk crash = Bye Bye models in progress.

Unless you have a backup of the BOINC (data) part, (and all subdirectories), made before the crash.
Then you can re-start from the time of the last backup.

This is easy in Windows, but harder in Linux because of the file ownership problem. It may also be hard on a Mac, unless this "Time machine" stuff also saves this info.
Can the Time machine backup be made to a location external to the HD?

One thing that CAN be saved externally, and SHOULD be, is the file: account_<project-name>.xml for each of your projects.
It contains an alternative identifier, which can be used to get you back in to a project if you forget the password.
Also, the file can be placed into the appropriate place in a new computer's install of BOINC, and that project(s) will recognize you as soon as you restart BOINC and allow it to contact the project(s).

---

Backups: Here

Time machine must back up to an external disk. It makes hourly backups for 24 hours (copying only the changed files made each hour). It keeps daily backups for the last month. It keeps weekly backups for all previous months until the backup disk is full, then it starts deleting old stuff to make room for new. Apple modified *nix to be able to create links to directories as well as to files. So there is never more than one copy of an unchanged directory/file on the backup disk. But since the climate models are constantly changing their data files, new versions of these files are constantly being managed on the backup disk.

I have always been nervous about Time Machine making backups simultaneous to the files changing, but it works. I *think* that with the Mac journaled file system, time machine is able to get a copy of all files and machine state at a single point in time so that a single instant can be recreated after a file system restore.

Looking at your link, it appears that all boinc files and state info are stored in a single directory tree ( on the mac: /Library/Application Support/BOINC Data/ ) and if boinc is suspended and shut down properly, this directory tree encapsulates everything necessary to restart boinc without a problem. So it appears that I have four choices:

1) Let time machine do its thing, (writing file changes to the backup disk every hour). Then every so often, stop the time machine backups, enter the time machine and selectively delete all backups to this directory tree ( 3 or 4 mouse clicks), restart the time machine backups, and say "backup now". This will start the backups of boinc anew. This has the disadvantage of extra disk traffic to keep the backup current to the hour. It has the advantage of mostly being automatic.

2) Normally exclude the boinc directory tree from time capsule backups. Once a week, remove this exclusion, select "backup now" and after the backup is done, re-exclude the boinc directory tree for another week.

3) Same as 2) except suspend/shutdown boinc while the backup is taking place. (To satisfy my paranoia about doing a backup while boinc is running. )

4) Once a week suspend/shutdown boinc, copy the directory tree to a backup disk, restart boinc. The only problem of this option, is remembering to do it, and waiting around while 14 GB is being copied to the (relatively slow) backup disk so boinc activity can be resumed.

I tend to lean toward 4).

2) Normally exclude the boinc directory tree from time capsule backups. Once a week, remove this exclusion, select "backup now" and after the backup is done, re-exclude the boinc directory tree for another week.

3) Same as 2) except suspend/shutdown boinc while the backup is taking place. (To satisfy my paranoia about doing a backup while boinc is running. )

4) Once a week suspend/shutdown boinc, copy the directory tree to a backup disk, restart boinc. The only problem of this option, is remembering to do it, and waiting around while 14 GB is being copied to the (relatively slow) backup disk so boinc activity can be resumed.

I tend to lean toward 4).

Well for hadam3p_eu-models it's a waste of time to do weekly backups, since chances are any restored backup will be of models you've already finished & reported. For hadam3p_anz the usefulness of a weekly backup is also limited so except for hadcm3n a weekly backup is mostly useless. (No idea with Moses).

A daily backup on the other hand would be much more useful. If the "Time Machine" is up to the task, I would choose option #5:

5: Exclude boinc from hourly backup. Make a separate backup-profile for BOINC, doing a daily backup of only the BOINC data-directory (including sub-directories).

If time machine can't handle #5, option #2 but done once-a-day is probably the best.

My Mac mini is so old that I now run only one task at a time. This opens up an extra option, which is to download a new model, suspend it before it starts, turn network activity off, then backup (by whatever method), then run the model to completion, then turn network activity on and upload etc.

The slightly perverse consequence of this protocol, for me, was that the occasional crashes that used to happen then stopped. So there has subsequently been no need to restore any backup. Because the backed-up copy has network activity off any permissions problems that might arise on restoring can be fixed without being visible, and since no data has been uploaded the Web site record is clean.

It reminds me of the schoolboy joke:

A man gets onto a train and sees another man tearing up pieces of paper and throwing them out of the window.

First man:    "Why are you doing that?"
Second man:   "To keep the elephants away."
First man:    "But there are no elephants."
Second man:   "Exactly!"

After actually examining how the climate model files change over time, I have concluded that it is best to just let time machine manage the backup of everything normally.

Using my machine's free time, the expected run time for the ANZ and EU models is about a week and the Molly is about a month.

The ANZ and EU models use about 600 MB of disk space at a time for each WU and the Molly uses about 1.5 GB.

I was surprised to see that most of model data memory is static. About 70 MB of the ANZ and EU models change each day and about 270 MB of the Molly changes each day. But only a small number of the files are changed more than once per day, so by letting the time machine back up hourly is not much different than option 5) except that time machine will keep a copy of all of the daily changes on the backup disk for a month. (roughly speaking: 70 MB x 30 days = 2.1 GB / month; 270 x 30 = 8.1 GB / month) After a month, time machine will age out the old files normally.

Hello?if the bionc files can get backed up via Time Machine, does it mean that my new computer can inherit uncompleted tasks?

Files that are backed up on one computer, can be transferred to another computer IF:

The OS is the same (Mac to Mac, etc)
PERHAPS if the OS version is different
BOINC is the same version. PERHAPS if the version is different

The server knows which computer accepted the WU in the fist place, and MAY accept them from another computer.

The idea of using backups to save work if there is a hardware failure, mostly applies to one or two processor machines, and creates problem with modern computers with 4 or more processors, and/or if many different projects are being run, because ALL work units from ALL projects will get restored, including those that finished between the time the backup was made, and the time that the hardware failure occurred.

And, VERY important:
From a post that I made in the middle of last year:

As part of the ongoing problems with attaching here, it's becoming obvious that there's a very important file on everyone's computer.

This is: account_climateprediction.net.xml
It has the information that the project needs to identify you.

Please make a copy (or several), and keep it in a safe place for when hardware problems occur.

Keeping the equivalent for ALL of your projects is a good idea for the same reason.

With this file, you can just drop it into a new install of BOINC, and the server for that project will know who you are.

Well, thanks a lot!
BTW where is that important file located? I can't seem to find it in the BOINC directory.

Never checked on a Mac but in linux it is in ../BOINC/ If it isn't obvious just use the search facility to find it.

This is the BOINC FAQ about file locations:
BOINC 7.0

Thanks a lot!