climateprediction.net (CPDN) home page
Thread 'Multiple Crunching of Same Thread'

Thread 'Multiple Crunching of Same Thread'

Message boards : Number crunching : Multiple Crunching of Same Thread
Message board moderation

To post messages, you must log in.

AuthorMessage
Lockleys

Send message
Joined: 13 Jan 07
Posts: 195
Credit: 10,581,566
RAC: 0
Message 31110 - Posted: 25 Oct 2007, 7:25:03 UTC

May I question why the same model is issued to more than one cruncher?

I understand the stats that say that many crunches fail to complete and the project needs to ensure that some models are completed. But I have a slab model going through 60% just now and it appears that this model has already run to full completion by two other crunchers with identical parameters and apparently identical results graphs. So why am I continuing to struggle towards the finish line with my slow laptop to produce a 3rd identical result?

Would it not be better to abort still-running models when the same model returns a completed result elsewhere rather than have us waste our resources? Or have I stupidly misunderstood?
ID: 31110 · Report as offensive     Reply Quote
ProfileMikeMarsUK
Volunteer moderator
Avatar

Send message
Joined: 13 Jan 06
Posts: 1498
Credit: 15,613,038
RAC: 0
Message 31111 - Posted: 25 Oct 2007, 7:58:46 UTC


If my understanding is correct, each individual model within the group is actually very slightly different. IIRC the \'initial condition parameter\' acts like a random number, so the models in the work unit form an \'ensemble\' of similar models. This provides a spread of outcomes for a given climate scenario.

If you listen to a detailed weather forecast, sometimes they give probabilities for things such as rain - the \'ensemble\' is now they work out what the probability is (i.e., 10 out of 25 models in an ensemble had rain, therefore the probability of rain is 40%).

http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=6835271

I'm a volunteer and my views are my own.
News and Announcements and FAQ
ID: 31111 · Report as offensive     Reply Quote
Lockleys

Send message
Joined: 13 Jan 07
Posts: 195
Credit: 10,581,566
RAC: 0
Message 31112 - Posted: 25 Oct 2007, 9:34:55 UTC - in response to Message 31111.  

Ah, right. I could see no discernable difference in the list of visible parameters, but if their is a hidden coefficient of nurdling, it makes more sense. It might be good to make it visible so that we-all realise our model is unique. On the other hand, the team have more valuable things to be doing.


If my understanding is correct, each individual model within the group is actually very slightly different. IIRC the \'initial condition parameter\' acts like a random number, so the models in the work unit form an \'ensemble\' of similar models. This provides a spread of outcomes for a given climate scenario.

If you listen to a detailed weather forecast, sometimes they give probabilities for things such as rain - the \'ensemble\' is now they work out what the probability is (i.e., 10 out of 25 models in an ensemble had rain, therefore the probability of rain is 40%).

http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=6835271

ID: 31112 · Report as offensive     Reply Quote
crandles
Volunteer moderator

Send message
Joined: 16 Oct 04
Posts: 692
Credit: 277,679
RAC: 0
Message 31113 - Posted: 25 Oct 2007, 9:58:41 UTC

Yes there are different initial conditions in models. However your model, the two that have finished and the other models listed have initial condition parameter 0.05 so the listed models are all identical. Models with different initial conditions will be listed under a different work unit reference.

Just because the models handed out are identical does not mean the results will be identical. If the processor or operating system is different then a different maths library is likely to be in use and the results are likely to be fairly similar but slightly different.

There is a recent paper Knight et al (see climate science, publications, scientific papers for link) which examined the effects. The changes created were larger than changes caused by an initial condition but fairly similar (IIRC less than double the sizes of variations) This is much smaller than the differences caused by many parameters. The effect is that these different models can sensibly be used as an initial condition ensemble.

If you look very carefully at the graphs you may detect slight variations. If so your work is not quite a repeat and is useful to increase the initial condition ensemble size which is wanted.

If it is absolutely identical, then some of these are wanted. (Or at least before the Knight et al paper was published they were. I don\'t really know whether there are more papers like Knight et al planned or being prepared. But I would assume it is possible.)

Even if you are sure it is identical, I don\'t think this is enough reason to abort it.
Visit BOINC WIKI for help

And join BOINC Synergy for all the news in one place.
ID: 31113 · Report as offensive     Reply Quote
crandles
Volunteer moderator

Send message
Joined: 16 Oct 04
Posts: 692
Credit: 277,679
RAC: 0
Message 31114 - Posted: 25 Oct 2007, 10:14:49 UTC

Another thing to say is that at various times there have been changes to the number of models being sent out. This appears to me to indicate there is some management of the numbers being sent out to get the inintial condition ensemble sizes that they want.

More management in terms of aborting excess exact duplicates if they are getting more than they want might be possible but I imagine time consuming for the scientists who want to work on the science or preparing different models and so on.
Visit BOINC WIKI for help

And join BOINC Synergy for all the news in one place.
ID: 31114 · Report as offensive     Reply Quote
Lockleys

Send message
Joined: 13 Jan 07
Posts: 195
Credit: 10,581,566
RAC: 0
Message 31115 - Posted: 25 Oct 2007, 14:58:01 UTC - in response to Message 31114.  

Another thing to say is that at various times there have been changes to the number of models being sent out. This appears to me to indicate there is some management of the numbers being sent out to get the inintial condition ensemble sizes that they want.

More management in terms of aborting excess exact duplicates if they are getting more than they want might be possible but I imagine time consuming for the scientists who want to work on the science or preparing different models and so on.


Yes, I take the point. With a larger (e.g. 160 year coupled) model, it would be intensely tempting to abort it and move to a new one - though I shall try to resist this temptation next time I spot it. But since my slab model should be complete in about 19 days (IT and supernatural forces willing) crunching will continue.
ID: 31115 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 31119 - Posted: 25 Oct 2007, 21:15:15 UTC


And sometimes the researchers have issued identical models several times where they want to make sure that at least one gets completed. Their reasons for this are varied, and you\'d need to be part of their team to know what they\'re \'up to\' from time to time.
So, keep crunching.


Backups: Here
ID: 31119 · Report as offensive     Reply Quote
Lockleys

Send message
Joined: 13 Jan 07
Posts: 195
Credit: 10,581,566
RAC: 0
Message 31120 - Posted: 25 Oct 2007, 21:42:57 UTC - in response to Message 31119.  


And sometimes the researchers have issued identical models several times where they want to make sure that at least one gets completed. Their reasons for this are varied, and you\'d need to be part of their team to know what they\'re \'up to\' from time to time.
So, keep crunching.



Yes. I guess my original thought process relates to the 160 year coupled models, some of which have taken me up to 18 months to complete. This is an awful lot of parallel burn if it\'s just in case someone else doesn\'t complete it.
ID: 31120 · Report as offensive     Reply Quote

Message boards : Number crunching : Multiple Crunching of Same Thread

©2024 cpdn.org