climateprediction.net (CPDN) home page
Thread 'wah2_eu25 work units are BIG'

Thread 'wah2_eu25 work units are BIG'

Message boards : Number crunching : wah2_eu25 work units are BIG
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
John Eric Hopkinson

Send message
Joined: 27 Jan 05
Posts: 74
Credit: 1,047,809
RAC: 0
Message 54284 - Posted: 11 Jun 2016, 14:19:25 UTC - in response to Message 54280.  

Les and geophi, two answers, thank you.
I have that map saved and the enlarged picture does indeed indicate very large areas assigned to the work units.
One region of personal interest is the Pacific Northwest which extends into Canadas west coast. This area is also under study by the Pacific Climate Impacts Consortium of University of Victoria,BC:
http://www.PacificClimate.org
A recent presentation by Dr.Charles Curry at the university :
"Does increased resolution alone add value in regional climate model simulations of climate extremes? A multi-scale study over western Canada".
Nice stuff, right?
No doubt Oxford is aware of PCIC, but I have not seen any reference to CPDN in any of the updates from PCIC.
The Australian region is huge, and a layman wonders what can be determined from a mix of so many weather patterns. No doubt the models are subjected to further concentrated studies of actual conditions in smaller sectors and the overlapped zones for confirmations of model predictions.
So much to learn, so little time.

ID: 54284 · Report as offensive     Reply Quote
DJStarfox

Send message
Joined: 27 Jan 07
Posts: 300
Credit: 3,288,263
RAC: 26,370
Message 54302 - Posted: 14 Jun 2016, 15:27:47 UTC
Last modified: 14 Jun 2016, 15:28:21 UTC

One behavior I've noticed.... If each WU requires 4GB of disk space and I have allocated 5GB to BOINC, then it will download 2 WU... resulting in out of disk space when trying to get more work for additional cores...

And seriously... 4GB per WU? Are you guys kidding me?

BOINC 6.10.45
ID: 54302 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 54303 - Posted: 14 Jun 2016, 16:15:11 UTC - in response to Message 54302.  

And seriously... 4GB per WU? Are you guys kidding me?


Not kidding - If you run high resolution models over a large area that is a lot of data being generated. The alternative would be to split things up so uploads to the server happened more frequently but if internet access was down at either your or the server end the size would still carry on increasing presumably up the the 4GB max which would be a completed model waiting to upload.

Interestingly http://www.climateprediction.net/support/technical-faq/#How_much_disk_space_do_the_models_take_up The faq on disk space is out of date and states that the largest models are the hadcm3n taking up a maximum of 2GB.

Biggest I have currently is 3GB. I have had problems in the past with the automagically allocated amount of disk space allocated to BOINC. On my quad core laptop I am currently giving it 50GB to play with between two projects, though it isn't often it runs anything but CPDN. Every once in a while one can get tasks that crash and don't clean up after themselves resulting in the task folder remaining - I find 50GB is enough that I always find out about the problem in time to delete the offending folders before they cause problems.

If I have missed anything important I am sure a moderator will post something as well.
ID: 54303 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54309 - Posted: 14 Jun 2016, 20:29:18 UTC

Also, it's UP TO 4 gigs.
Data accumulates until it's time to zip up the data for return to the project. After this, the original files are deleted, and the total drops back down.
Only to slowly increase again.

How much data is accumulated depends on how much info the researcher(s) need.
Some zips have been around 70 Megs, whereas lately there's a batch with zips of between 1 and 2 Megs.

The increasing size of data was foretold by me centuries ago, in the post at the top of this section, right under the News thread. :)

ID: 54309 · Report as offensive     Reply Quote
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,824,485
RAC: 4,956
Message 54355 - Posted: 21 Jun 2016, 9:49:42 UTC - in response to Message 54274.  

[Les Bayliss wrote:]Mexico has just joined up.
It's mentioned on the front page, bottom right, under
Experiments Coming Soon
[Iain Inglis wrote:]Just got a mex50 model - wah2_mex50_fo0v_193812_12_407_010607167_1. New one on me ...

The Mexico model completed nicely: 12 Zips of ~17 MB and a final one of 54 MB.
ID: 54355 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,033,903
RAC: 14,766
Message 54399 - Posted: 28 Jun 2016, 18:27:14 UTC - in response to Message 54355.  

There are a couple of new eu batches that are only a one month run and they do not produce any trickles. They are 412 and 413 as far as I can make out.
ID: 54399 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54400 - Posted: 28 Jun 2016, 18:50:33 UTC - in response to Message 54399.  

No trickles as in: No trickles mentioned in the Event log,
or
No trickles as in the last few posts in this thread ?

ID: 54400 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,033,903
RAC: 14,766
Message 54401 - Posted: 28 Jun 2016, 22:30:45 UTC - in response to Message 54400.  

No trickles listed on the task page in "my account" but the tasks show as completed. Tasks are 10621786 and 10619445 - both completed - and 10614217 in progress. Estimated run time is 18hrs on my machine.
ID: 54401 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54403 - Posted: 29 Jun 2016, 0:36:29 UTC - in response to Message 54401.  
Last modified: 29 Jun 2016, 0:37:37 UTC

That's a new problem which started recently.
I've sent an email about it.

It applies to the trickles of ALL model types.
ID: 54403 · Report as offensive     Reply Quote
Profile[B@H] tomcat

Send message
Joined: 28 Nov 05
Posts: 24
Credit: 3,784,363
RAC: 0
Message 54415 - Posted: 4 Jul 2016, 13:41:05 UTC

I've got neither any trickles nor credits for wah2!
Why?
Is this problem known already? (search didn't find any hint)

examples:
ID 19701200, reported 11.Jun, http://climateapps2.oerc.ox.ac.uk/cpdnboinc/result.php?resultid=19701200
ID 19678778 reported 14.Jun, http://climateapps2.oerc.ox.ac.uk/cpdnboinc/result.php?resultid=19678778
ID 19603818 reported 29.Jun, http://climateapps2.oerc.ox.ac.uk/cpdnboinc/result.php?resultid=19603818

System:
Mac (2 different) / Mac OS X 10.11 / BOINC 7.6.22



ID: 54415 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54416 - Posted: 4 Jul 2016, 15:07:36 UTC - in response to Message 54415.  

The post about it is in the Mac section: here

And then there's this about why credits take a while to appear.

ID: 54416 · Report as offensive     Reply Quote
ojum-le

Send message
Joined: 5 May 07
Posts: 27
Credit: 6,369,307
RAC: 0
Message 54595 - Posted: 1 Aug 2016, 8:08:21 UTC - in response to Message 53057.  
Last modified: 1 Aug 2016, 8:12:12 UTC

Think about HadCM3-Model some years ago.
A 160-year-simulation takes round about 6 month of calculating-time on a i7-920-machine. CPDN always was a project who need endless PC-power.

I was shut down my CPDN-computing because of the hich energy-prices in germany :-( An i7-920 takes 260W always. Over the year it causes about 1000€/1100$ of energy-costs every year.

The only senseful way to use CPDN in a private case is to use high-efficient Power-platform. Broadwell-EP (LGA2011-3) in a low-power design would be the efficient way to participate CPDN for credit-hunting.

The 2-channel-platform (1151) is not really recommend for CPDN i mean. The FPU-Power is too much less for the energy-consumption. CPDN is a typical high-FPU-based computation. Look at spec.org to identify the best performance of "481.wrf" in the SPEC2006-FP-Rates-benchmark. These benchmark induces directly the CPDN-performance of the system. There you can see, that an actual Skylake 1151-CPU only has 80 Percent more FPU-power over an i7-920. Than take a look to an Broadwell/Haswell-EP-platform and u can see a much stronger increasement of FPU-power - depends on core-numbers - but the whole performance and efficiency in not comparable to S1151-platform.
ID: 54595 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 54596 - Posted: 1 Aug 2016, 8:46:54 UTC - in response to Message 54595.  

Or to have your own renewables. - My electricity during the day all comes from our roof. At some point I may upgrade to a system where I have battery back up/storage to reduce what I use from the mains still further.
ID: 54596 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : wah2_eu25 work units are BIG

©2024 cpdn.org