climateprediction.net (CPDN) home page
Thread 'How to complete a WU sucessfully ???'

Thread 'How to complete a WU sucessfully ???'

Message boards : Number crunching : How to complete a WU sucessfully ???
Message board moderation

To post messages, you must log in.

AuthorMessage
old_user530580

Send message
Joined: 3 Aug 08
Posts: 7
Credit: 7,146
RAC: 0
Message 36387 - Posted: 15 Mar 2009, 17:32:26 UTC

Hello !!!
After 723h43m33s CPU time without any problem, there are remaining 2s of work to complete the file hadsm3fub_jvua_..... ( That is like a remaining 0,75m step at the end of a 1000Km long trail. Not very much.) I didn´t find any way to restart this minimal work. If I restart the projekt, I get messages like:
15.03.2009 13:34:19|climateprediction.net|Restarting task hadsm3fub_jvua_005959397_4 using hadsm3 version 506
15.03.2009 13:34:23|climateprediction.net|Sending scheduler request: To send trickle-up message. Requesting 0 seconds of work, reporting 0 completed tasks
15.03.2009 13:34:28|climateprediction.net|Scheduler request succeeded: got 0 new tasks.
After that message it remains steady for prolonged time.

Just before the last working session was started, there where 1023.74MB of data from CPN on my disk seen by the BOINC Manager. Now there are 612MB of data. That would mean to me, that the difference of 412MB of data had been transfered to the server as I didn´t find them in the recycle bin. - I never observed, that my application of CPN transfered data back to the server. - In the folder \\BOINC\\projects\\climateprediction.net\\hadsm3......\\dataout on my disk, I still find 375 files summarizing to 735MB. I believe, some of them are of some importance for the project.
So my question: To whome and how should I send what data ?
I don´t have the possibility to send them by FTP.

Kind regards
Martin


ID: 36387 · Report as offensive     Reply Quote
ProfilePooh Bear 27
Avatar

Send message
Joined: 5 Feb 05
Posts: 465
Credit: 1,914,189
RAC: 0
Message 36388 - Posted: 15 Mar 2009, 17:42:29 UTC

You have completed graphs, so it was finished and uploaded. The report function may or may not have been done yet. The crediting system on CPDN runs approximately once a day so your crediting didn\'t finish.

There is data that can be left over if you do things in the watching of the WU you are crunching, as it can save data that you did not realized it was saving. Plus there are a lot of files that download, executables, supporting files, etc. that do not delete so you can reuse them for another WU of the same type.

Is the result completely gone (look if it\'s at \"Ready To Report\", or gone). If gone, once the credit program runs on CPDN it should do the clean up of the unit on the site to show it\'s complete.

Hope this helps.
ID: 36388 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 36394 - Posted: 16 Mar 2009, 23:32:40 UTC
Last modified: 16 Mar 2009, 23:33:02 UTC

This is the model\'s web page. As Pooh Bear says, all the graphs are there so the model completed successfully. You also have the correct number of credits for a completed model. Thank you very much for completing it. The graphs look good.

I think you probably clicked on the Reset button which is button #4 in the Projects section. You probably did this before the model reported. (Models first transfer their files; they report later.) I think this is why the model shows the -197 code meaning aborted by you. This doesn\'t matter because the model had already uploaded its final file for phase 3. But you shouldn\'t normally use the project Reset button. In the case of a model still running, you would lose the rest of the model. To reset a project is to abort the project.

Do you want a new model? If you do want another and climateprediction has disappeared from your Boinc Manager Projects list, you will need to attach again. Boinc Manager > Tools > Attach to project. Say you are an existing member, not a new member and give the same email address as before.
Cpdn news
ID: 36394 · Report as offensive     Reply Quote
old_user530580

Send message
Joined: 3 Aug 08
Posts: 7
Credit: 7,146
RAC: 0
Message 36401 - Posted: 17 Mar 2009, 8:40:31 UTC - in response to Message 36388.  
Last modified: 17 Mar 2009, 8:48:11 UTC

Thanks Pooh Bear!
You have completed graphs, so it was finished and uploaded.

As all indications in the graphs showed, that the task was finished ( Timestep 259248 of 259248; Your Progress 100,00%; Temperature graph showed a blue planet; Simulation Time 0:0:0:0 ), so I decided to \"Abort\" this task and \"Update\"ed the project.

There is data that can be left over if you do things in the watching of the WU you are crunching,

The question remains: What type if data became obsolete and can be cleared? It seems to be by far the majority.

Is the result completely gone (look if it\'s at \"Ready To Report\", or gone). If gone, once the credit program runs on CPDN it should do the clean up of the unit on the site to show it\'s complete.

After aborting the task and \"Update\" the project, the task diapeared from the tasklist and the disk usage in BOINC Manager reduced to 575,5MB But in the WinExplorer it stays at 735MB as before. So I believe, these 575.5 MB are obsolete data and the difference of about 160MB may be of some concerne for further tasks. But more than 0,5GB of obsolete data should not be ignored.


Hope this helps.
Yes, thanks!
Martin
ID: 36401 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 36402 - Posted: 17 Mar 2009, 9:03:57 UTC

Under projects/climateprediction.net, there are folders for each of the models being run. Sometimes these don\'t get deleted after the model is finished, such as when a model crashes.
If you have a folder with the same name as the model that you just finished, you can delete it. And any other model\'s folder that you no longer have listed in the Tasks tab.


Backups: Here
ID: 36402 · Report as offensive     Reply Quote
old_user530580

Send message
Joined: 3 Aug 08
Posts: 7
Credit: 7,146
RAC: 0
Message 36403 - Posted: 17 Mar 2009, 9:31:20 UTC - in response to Message 36394.  

Hallo mo.v !
As Pooh Bear says, all the graphs are there so the model completed successfully. The graphs look good.

It´s nice to know, that all results are well transfered.

I think you probably clicked on the Reset button. You probably did this before the model reported.

No, I clicked the abort buttom last night, before reading your message, as there was no further reaction of the task, what ever I tried and after becoming fairly shure, that the task has ended almost successfull.

I´m very much astonished, that CPN let stay these valuable results at the participants computer. Here the used disk space by CPN increased by 10,45 +/- 0,082[MB/%], with jumps at about every 1% of progress. - I could send you a graph of that by email. - I find this behavior highly risky. On the other hand, if you transfer intermediate results back to the server, why don´t you delete these now obsolete data from the participants disk ???

I would be pleased to get an answer on my questions also in my answer to Pooh Bears reply.

With kind regards
Martin
ID: 36403 · Report as offensive     Reply Quote
ProfileIain Inglis

Send message
Joined: 9 Jan 07
Posts: 467
Credit: 14,549,176
RAC: 317
Message 36404 - Posted: 17 Mar 2009, 10:59:41 UTC

Martin,

Here is an example of how the stored data increases as a \'slab\' model (i.e. HADSM3) progresses.



When the model finishes successfully, all the data for the model is deleted. However, some crashes do leave some data behind.

To destroy all data for a project, then \'detach\' from the project. Only do this when you\'re not running any models, because detaching destroys running models as well as the traces of any finished/crashed models. You can then immediately \'attach\' again, identifying yourself as an existing user and using the same e-mail address and password. All the data will have gone.

Iain
ID: 36404 · Report as offensive     Reply Quote
old_user530580

Send message
Joined: 3 Aug 08
Posts: 7
Credit: 7,146
RAC: 0
Message 36410 - Posted: 18 Mar 2009, 11:21:37 UTC - in response to Message 36404.  
Last modified: 18 Mar 2009, 11:31:22 UTC


Here is an example of how the stored data increases as a \'slab\' model (i.e. HADSM3) progresses.

Thanks for this graph. At me it looks different. The jumps are only 8 - 8.5[MB] high, but every 2.32 +/- 0.31 [%] of progress - our abzissa differ -. I did this WU in portions of 0.147[%] of progress only.

But still my question: Is it realy neccessary for the calculations, to keep the result data available at the participants computer ?

Martin
ID: 36410 · Report as offensive     Reply Quote
ProfileIain Inglis

Send message
Joined: 9 Jan 07
Posts: 467
Credit: 14,549,176
RAC: 317
Message 36411 - Posted: 18 Mar 2009, 11:37:00 UTC - in response to Message 36410.  
Last modified: 18 Mar 2009, 11:38:47 UTC

[Martin wrote:] But still my question: Is it realy neccessary for the calculations to keep the result data available at the participants computer ?

No, it is not necessary. If the model finishes normally, then the data is deleted. Data will remain on your machine only as the result of an error of some kind (in the model, changed file permissions etc.).

The temporary data accumulated during the run is processed into compressed Zip files and uploaded to the project\'s server. It should then be deleted automatically. Nothing that remains on the computer as the result of an error is of use to the project.

If you have a growing amount of data accumulating from models that didn\'t finish correctly, then use the \'detach\' procedure described in my earlier post - that will remove all data for the project.

Iain
ID: 36411 · Report as offensive     Reply Quote
old_user530580

Send message
Joined: 3 Aug 08
Posts: 7
Credit: 7,146
RAC: 0
Message 36412 - Posted: 18 Mar 2009, 13:12:18 UTC - in response to Message 36411.  

Hallo Iain !
Thanks for your answer.
The temporary data accumulated during the run is processed into compressed Zip files and uploaded to the project\'s server.

When does this upload happen? By the program in intermediate steps or only once at the end? In BOINC Manager/Messages on my pc I never saw any hint about uploaded result data. But there must be some exchange of information between my pc and the server, as about every 1[%] of progress the \"Work done\" became updated. I never watched projects/climateprediction.net until now.

Sorry that I´m somewhat penetrant, but it´s of some interest about the behavior of this project.

Martin
ID: 36412 · Report as offensive     Reply Quote
ProfileIain Inglis

Send message
Joined: 9 Jan 07
Posts: 467
Credit: 14,549,176
RAC: 317
Message 36413 - Posted: 18 Mar 2009, 16:03:02 UTC
Last modified: 18 Mar 2009, 16:11:10 UTC

[Martin wrote:] When does this upload happen?

Here, for example, is the contents of the message log for a slab model (i.e. HADSM3), which is taken from the messages tab of BOINC Manager (in advanced view) or from stdoutdae.txt in the BOINC data folder.

[size=9]17-Feb-2009 08:30:52 [climateprediction.net] Started upload of hadsm3fub_k9z3_005977714_3_1.zip
17-Feb-2009 08:30:52 [climateprediction.net] Sending scheduler request: To send trickle-up message.  Requesting 0 seconds of work, reporting 0 completed tasks
17-Feb-2009 08:30:57 [climateprediction.net] Scheduler request succeeded: got 0 new tasks
17-Feb-2009 08:32:06 [climateprediction.net] Finished upload of hadsm3fub_k9z3_005977714_3_1.zip[/size]

[size=9]27-Feb-2009 17:27:20 [climateprediction.net] Started upload of hadsm3fub_k9z3_005977714_3_2.zip
27-Feb-2009 17:27:24 [climateprediction.net] Sending scheduler request: To send trickle-up message.  Requesting 0 seconds of work, reporting 0 completed tasks
27-Feb-2009 17:27:29 [climateprediction.net] Scheduler request succeeded: got 0 new tasks
27-Feb-2009 17:27:56 [climateprediction.net] Finished upload of hadsm3fub_k9z3_005977714_3_2.zip[/size]

[size=9]15-Mar-2009 07:34:08 [climateprediction.net] Started upload of hadsm3fub_k9z3_005977714_3_3.zip
15-Mar-2009 07:34:09 [climateprediction.net] Sending scheduler request: To send trickle-up message.  Requesting 0 seconds of work, reporting 0 completed tasks
15-Mar-2009 07:34:11 [climateprediction.net] Computation for task hadsm3fub_k9z3_005977714_3 finished
15-Mar-2009 07:34:13 [climateprediction.net] Scheduler request succeeded: got 0 new tasks
15-Mar-2009 07:34:42 [climateprediction.net] Finished upload of hadsm3fub_k9z3_005977714_3_3.zip
16-Mar-2009 07:34:46 [climateprediction.net] Sending scheduler request: To report completed tasks.  Requesting 0 seconds of work, reporting 1 completed tasks
16-Mar-2009 07:34:51 [climateprediction.net] Scheduler request succeeded: got 0 new tasks[/size]

So, a slab model uploads data at the end of each of its three phases. Other model types upload data at different intervals (HADSM3MH at the end of each of its four phases, HADCM3 every 10 model years, HADAM3 and HADAM3P at the end of the model).

You should have similar entries in your message log.

Iain

PS Note how BOINC waited a day after the upload of the final Zip file before automatically reporting the task as complete. If another model had been downloaded immediately then the report would have been submitted earlier, or the report button in BOINC Manager can be pressed. The model is only completely finished when the data has been uploaded and the model has been reported.
ID: 36413 · Report as offensive     Reply Quote

Message boards : Number crunching : How to complete a WU sucessfully ???

©2024 cpdn.org