climateprediction.net (CPDN) home page
Thread 'backup and restore WU (models)'

Thread 'backup and restore WU (models)'

Message boards : Number crunching : backup and restore WU (models)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profileold_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 7419 - Posted: 21 Jan 2005, 9:57:00 UTC


I’ve been running cp boinc for a week, with some frustrating lost models. How can I backup and then restore from these backups under WinXP OS?

I was thinking of running a daily scheduled task of a batch file that XCOPY to another PC on the network. I use this method for daily backing up vital data on my PC to a remote PC, encase of theft, fire etc. This method works fine, even if I’m accessing and using the files that are being copied.

I’ve got boinc_cli running as a service all the time. Can I apply this automated method to backing up the boinc directory whilst boinc_cli service is running? Would conflicts occur if both boinc_cli and XCOPY were accessing the same files at the same time? The likelihood of both boinc_cli and XCOPY accessing the file at the same time is high as I believe boinc writes to the directory every 144 timesteps, which on my PCs is every 8 to 9 minutes.

The next question is how do I restore from these backups? Do I stop the boinc_cli service (net stop boinc) and then completely replace the boinc program file directory with the backup. Or do I just replace the project\climateprediction.net directory in boinc program file?

Ian

PS.
By the way if you’re interested in the copy batch file it is:

ECHO/ >> CopyFile_Vesuvius.log 2>&1
ECHO/ | DATE | FIND /V "new" >> CopyFile_Vesuvius.log 2>&1
ECHO/ | TIME | FIND /V "new" >> CopyFile_Vesuvius.log 2>&1

XCOPY "C:\PhD" "\\Hotchpotch\Backup_data\PhD" /D /E /V /C /I /F /H /R /K /Y /Z >> CopyFile_Vesuvius.log 2>&1

ECHO/ | TIME | FIND /V "new" >> CopyFile_Vesuvius.log 2>&1

This batch file when run as a scheduled task automatically backups my data and generates a log file that is date and time stamped. The XCOPY commands can be found at: http://www.computerhope.com/xcopyhlp.htm

ID: 7419 · Report as offensive     Reply Quote
Profileold_user34359
Avatar

Send message
Joined: 30 Dec 04
Posts: 6
Credit: 67,309
RAC: 0
Message 7421 - Posted: 21 Jan 2005, 10:23:53 UTC

Sorry Joicedry i have no answer for you,
but it will be a good idear to involve a backup function into die Clientsoftware, because read very often that many users have that problem.

And it is indeed very irritating, if your model will be lost ,after working on it hunderts of hours. So please, why is that not a part of the CPDN software, issn´t it possible to integrate that ?

I hope that the developers can do anything to help us and also the project, because also the project is loosing a lot of data, in cause of this.


Greetings from Wetzlar in Germany

Sascha Bickel
Admin, Teamleader CPDN & Einstein
Team Science and Research Hessen (SaR Hessen)
http://www.sar-hessen.dd.vu

<img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=323" />
ID: 7421 · Report as offensive     Reply Quote
KeeperC

Send message
Joined: 5 Aug 04
Posts: 66
Credit: 2,146,056
RAC: 0
Message 7424 - Posted: 21 Jan 2005, 11:22:53 UTC

There is a utility called dobkCPDN that will do this automatically on Win XP. It is officially a beta version but I think development has stopped. I have been using it without real problems for a while now. It does sometimes crash, but the crash is always harmless and has never caused damage to CPDN or anything else. The link to it is on the "classic" CPDN boards, I think, so I can't find a link at the moment (boards down). However, google suggests:

http://www.climateprediction.net/board/viewtopic.php?t=2474&amp;sid=cedd4958a7b6fb1e0434fd6359d9a3ee

Hope that helps.
ID: 7424 · Report as offensive     Reply Quote
LochDhu

Send message
Joined: 5 Aug 04
Posts: 27
Credit: 13,339,226
RAC: 0
Message 7434 - Posted: 21 Jan 2005, 14:06:01 UTC

You need to stop the service; otherwise there will be write locks on some files. I don't know the command line way of stopping &amp; starting a service, but I know there is one.

A restore is as easy as stopping the service, and overwritting the BOINC directory with your backup, and starting again.

Personally I only backup after every phase, so I do it manually.
ID: 7434 · Report as offensive     Reply Quote
KeeperC

Send message
Joined: 5 Aug 04
Posts: 66
Credit: 2,146,056
RAC: 0
Message 7438 - Posted: 21 Jan 2005, 15:14:43 UTC - in response to Message 7434.  

&gt; You need to stop the service; otherwise there will be write locks on some
&gt; files. I don't know the command line way of stopping &amp; starting a
&gt; service, but I know there is one.
&gt;
&gt; A restore is as easy as stopping the service, and overwritting the BOINC
&gt; directory with your backup, and starting again.
&gt;
&gt; Personally I only backup after every phase, so I do it manually.
&gt;

The utility I use stops the application using RPC (what ever this is), makes the backup, then restarts the application. I assume, but don't know, that this is true if it is running as a service, too. I don't run CPDN as a service.
ID: 7438 · Report as offensive     Reply Quote
Profileold_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 7444 - Posted: 21 Jan 2005, 17:54:35 UTC

I have now setup scheduled tasks to daily backup the boinc program file via the XCOPY command. First, it stops boinc_cli by “net stop boinc” then waits 1 minute before it copies the boinc program file before waiting another minute to start boinc_cli via the “net start boinc” command.

When I want to recover boinc to a previous state with the backup do I replace the complete boinc program file with the backup or do I replace just the projects/climateprediction.net directory within boinc program file?

Ian
ID: 7444 · Report as offensive     Reply Quote
ProfileThyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 7447 - Posted: 21 Jan 2005, 18:49:26 UTC

You'd have to replace the whole BOINC directory because all the information controlling the jobs run by BOINC is held in the client_state.xml file and slots directory.
"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 7447 · Report as offensive     Reply Quote
Profileold_user34359
Avatar

Send message
Joined: 30 Dec 04
Posts: 6
Credit: 67,309
RAC: 0
Message 7465 - Posted: 22 Jan 2005, 19:12:21 UTC - in response to Message 7424.  

&gt; There is a utility called dobkCPDN that will do this automatically on Win XP.
&gt; It is officially a beta version but I think development has stopped. I have
&gt; been using it without real problems for a while now. It does sometimes crash,
&gt; but the crash is always harmless and has never caused damage to CPDN or
&gt; anything else. The link to it is on the "classic" CPDN boards, I think, so I
&gt; can't find a link at the moment (boards down). However, google suggests:
&gt;
&gt;
&gt; http://www.climateprediction.net/board/viewtopic.php?t=2474&amp;sid=cedd4958a7b6fb1e0434fd6359d9a3ee
&gt;
&gt; Hope that helps.
&gt;

Sorry KeeperC, but your Link to the Treat does not function, can you give me the Path, so i can go there manualy ?

And to Juicedry,

can you please post how you have set XCopy to do that, i don´t know the Program but i like to try it out.


Greetings from Wetzlar in Germany

Sascha Bickel
Admin, Teamleader CPDN &amp; Einstein
Team Science and Research Hessen (SaR Hessen)
http://www.sar-hessen.dd.vu

<img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=323" />
ID: 7465 · Report as offensive     Reply Quote
KeeperC

Send message
Joined: 5 Aug 04
Posts: 66
Credit: 2,146,056
RAC: 0
Message 7466 - Posted: 22 Jan 2005, 19:16:21 UTC - in response to Message 7465.  


&gt; Sorry KeeperC, but your Link to the Treat does not function, can you give me
&gt; the Path, so i can go there manualy ?

The path is to a page in the CPDN PHP forum. Unfortunately, because of a security risk, that forum has temporarily been taken offline. I hope that it will be back soon, and then the link should work.

I can't remember the link to the download utility directly.
ID: 7466 · Report as offensive     Reply Quote
ePig

Send message
Joined: 5 Aug 04
Posts: 7
Credit: 1,116,870
RAC: 0
Message 10334 - Posted: 4 Mar 2005, 4:34:00 UTC - in response to Message 7444.  

Juicedry, could you post your XCOPY batch file for backing up the model? Should sit down and figure it out myself, but am a little lazy :) Thanks a mill.

By the way, the dobkCPDN site is down. Was the latest release reasonably stable, and if so, does anyone have a copy themselves that they would like to post a link to??

And when is the CPDNclassic board going up?? It's been a while now and I checked the phpBB site and it looked like the bugs have been fixed, correct me if I'm wrong.

epig ~:0)
<a href=\"http://www.boinc.dk/index.php?page=user_statistics&project=cpdn&userid=133\"><img border=0 width=\"280\" height=\"60\" src=\"http://www.boinc.dk/auto.php?user=133&project=cpdn&input=1074199139+-+1&layout=1074199139+-+1.jpg\"></a>
ID: 10334 · Report as offensive     Reply Quote
ProfileThyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 10339 - Posted: 4 Mar 2005, 7:49:32 UTC - in response to Message 10334.  

&gt; And when is the CPDNclassic board going up?? It's been a while now and I
&gt; checked the phpBB site and it looked like the bugs have been fixed, correct me
&gt; if I'm wrong.

We have been promised that the site will be back but that it needs to be taken "in-house", which means the project team need to find the hardware and time to do it.

In the meantime you could try <a href="http://cpdnforum.info/cpdn/index.php">the unofficial CPDN phpBB forum</a> ;)

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 10339 · Report as offensive     Reply Quote
old_user2467

Send message
Joined: 28 Aug 04
Posts: 90
Credit: 2,736,552
RAC: 0
Message 10341 - Posted: 4 Mar 2005, 8:34:03 UTC - in response to Message 10339.  
Last modified: 4 Mar 2005, 8:34:50 UTC

I've problems to understand the following. Maybe somebody can explain this?

You want to backup the cpdn model 'cause of lost models? Means this they crashed on computation? But does'nt boinc contacts the sheduler in this case and reports this workunit as completed with computation error? What makes it sense to compute further on this model. Is'nt it that you will get no further credit on this wu?

Ciao, Tom
ID: 10341 · Report as offensive     Reply Quote
ProfileAndrew Hingston
Volunteer moderator

Send message
Joined: 17 Aug 04
Posts: 753
Credit: 9,804,700
RAC: 0
Message 10342 - Posted: 4 Mar 2005, 8:44:00 UTC - in response to Message 10341.  


&gt; What makes it
&gt; sense to compute further on this model. Is'nt it that you will get no further
&gt; credit on this wu?

CPDN allows computation of WUs to be resumed in these cases. Even where the server state shows an error, it will continue to accept trickles and the final result files. Credit will only be given on work returned beyond the failure point.

I believe that it may be different for other projects, but they do not have intermediate steps or take so long to complete.
ID: 10342 · Report as offensive     Reply Quote
old_user2467

Send message
Joined: 28 Aug 04
Posts: 90
Credit: 2,736,552
RAC: 0
Message 10343 - Posted: 4 Mar 2005, 8:54:49 UTC - in response to Message 10342.  

&gt; CPDN allows computation of WUs to be resumed in these cases. Even where the
&gt; server state shows an error, it will continue to accept trickles and the final
&gt; result files. Credit will only be given on work returned beyond the failure
&gt; point.

Thanks! This was new for me. But on reporting a crashed wu you normally get new work. What you're doing with this? Storing it to crunch it after completing the formerly crashed wu? But on reproting the formerly crashed wu as finished you get an additional new wu? Or recognizes the sheduler that he has allready given you a new wu? Just confused.

Ciao, Tom
ID: 10343 · Report as offensive     Reply Quote
ProfileAndrew Hingston
Volunteer moderator

Send message
Joined: 17 Aug 04
Posts: 753
Credit: 9,804,700
RAC: 0
Message 10360 - Posted: 4 Mar 2005, 12:40:23 UTC

If you have already done many hours of work on a WU it will often be better to carry on with the old WU than to start with a new one. There are three answers as to what you then do about the new WU.

1. If you just discard the new WU it will show on the server as being processed by you, but if you do not send trickles back it will be allocated to somebody else eventually.

2. You could save the new unit by moving the BOINC directory to create a new backup, then continue doing the old one, stop BOINC from downloading a new WU when it finishes, then restore the later backup and carry on with that.

3. It is possible to edit your files to keep both WUs in your active directory. Editing of the client_state_xml file is discussed in <a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/forum_thread.php?id=2035">this thread</a>. This will be best only for those who feel confident enough.
ID: 10360 · Report as offensive     Reply Quote
old_user2467

Send message
Joined: 28 Aug 04
Posts: 90
Credit: 2,736,552
RAC: 0
Message 10361 - Posted: 4 Mar 2005, 12:54:10 UTC - in response to Message 10360.  

Thanks Andrew for your explanations!

Ciao, Tom
ID: 10361 · Report as offensive     Reply Quote

Message boards : Number crunching : backup and restore WU (models)

©2024 cpdn.org