climateprediction.net (CPDN) home page
Thread '8.09 UK Met Office HadAM4 at N144 resolution'

Thread '8.09 UK Met Office HadAM4 at N144 resolution'

Questions and Answers : Unix/Linux : 8.09 UK Met Office HadAM4 at N144 resolution
Message board moderation

To post messages, you must log in.

AuthorMessage
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60333 - Posted: 17 Jun 2019, 14:30:47 UTC

Wonders never cease. A few work units are available.
I am running a bake-off on my i7-8700, i7-4770 and Ryzen 2600, all on Ubuntu 18.04. It will be only one work unit on each machine at first, just to give an idea of the run times.

But the first one running on the i7-8700 is using just 625 MB of memory after 10 minutes, so nothing unusual about that.
ID: 60333 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,016,442
RAC: 21,024
Message 60334 - Posted: 17 Jun 2019, 15:17:47 UTC - in response to Message 60333.  

But the first one running on the i7-8700 is using just 625 MB of memory after 10 minutes, so nothing unusual about that.


The new more demanding ones when they come will be using open ifs which is an EU collaboration program rather than a Met Office one. On the positive side, they are all 64bit which means no need for 32bit libs. On the negative side they will use a LOTmore memory.
ID: 60334 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 60336 - Posted: 17 Jun 2019, 15:34:00 UTC - in response to Message 60333.  

Yeah, the memory requirements of these are not too much different than the higher res WAH2 Windows tasks. There was a small batch of these run on this site back in early February (batch 785), but they had the problem of not being able to recover from restarts. That problem has been fixed.

Tests have been run on the beta site of a higher resolution version of hadam4 (N216) which took a little over twice the RAM per task, and a workstation version of the ECMWF called openIFS which has required 4 to 6 or more times the RAM of these hadam4 N144 tasks. Who knows when one or the other of these two higher requirements models will make it over to the main cpdn site.
ID: 60336 · Report as offensive     Reply Quote
mngn

Send message
Joined: 13 Jul 18
Posts: 38
Credit: 62,933,508
RAC: 84,702
Message 60337 - Posted: 17 Jun 2019, 15:45:22 UTC

ID: 60337 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,016,442
RAC: 21,024
Message 60339 - Posted: 17 Jun 2019, 16:14:30 UTC - in response to Message 60337.  

With 1GB/Pi (could be a complex formula to come out of that eventually) some of the models in testing recently would require six Pis run one task. (Assuming the memory requirement could be spread across six of the devices.) I don't know how closely the type of tasks CPDN will be using the openifs software for mirror the work done in the link you provided so have no idea whether this is possible or not.

If the work and memory could be spread in such a way, 24 cores at 1.4GHz may well be faster and quite a lot cheaper than the motherboard, cpu and memory upgrade I am considering at the moment!
ID: 60339 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 60342 - Posted: 17 Jun 2019, 21:58:38 UTC - in response to Message 60333.  
Last modified: 17 Jun 2019, 22:00:44 UTC

But the first one running on the i7-8700 is using just 625 MB of memory after 10 minutes, so nothing unusual about that.


I have four that are up to about nine hours. They each are taking aobut 4% of my 16 GBytes of RAM, which is not too bad. mine seem to need 632 to 635 MebaBytes RAM. They claim to be 32-bit applications:

UK Met Office HadAM4 at N144 resolution v8.09 i686-pc-linux-gnu

USER PR NI S VIRT RES SHR %MEM %CPU TIME+ P COMMAND

boinc 39 19 R 652m 635m 9.8m 4.0 95.9 531:42.45 1 /home/boinc/projects/climateprediction.net/hadam4_um_8.09_i686-pc-linux-gnu 177305 /home
boinc 39 19 R 652m 635m 9.8m 4.0 96.9 531:41.31 3 /home/boinc/projects/climateprediction.net/hadam4_um_8.09_i686-pc-linux-gnu 173955 /home
boinc 39 19 R 652m 635m 9.8m 4.0 98.9 531:57.64 0 /home/boinc/projects/climateprediction.net/hadam4_um_8.09_i686-pc-linux-gnu 174225 /home
boinc 39 19 R 652m 632m 9.8m 4.0 98.1 486:22.36 2 /home/boinc/projects/climateprediction.net/hadam4_um_8.09_i686-pc-linux-gnu 177375 /home
ID: 60342 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60343 - Posted: 17 Jun 2019, 22:00:44 UTC

These are the small ones. The big ones are still being tested.
ID: 60343 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 25,620,508
RAC: 4,981
Message 60385 - Posted: 20 Jun 2019, 18:40:10 UTC - in response to Message 60343.  

These are the small ones. The big ones are still being tested.


Will there be any system requirements published for all models? I currently run 8 N144 on 16 Gb machine (assuming 2Gb per model) but if it won't be the case for the future some sticky would be appreciated (or messages via boinc) - whatever works
ID: 60385 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60386 - Posted: 20 Jun 2019, 22:06:32 UTC

Yes there will.
They're still in development/testing.
Please be patient.
ID: 60386 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 25,620,508
RAC: 4,981
Message 60537 - Posted: 1 Jul 2019, 18:53:42 UTC

Some memory debug from 2 WUs running on Core 2 Duo E7300 @ 2.66GHz with 4GB RAM

Mon 01 Jul 2019 08:53:49 PM EEST | climateprediction.net | [mem_usage] hadam4_a01z_200610_12_825_011882426_0: WS 617.70MB, smoothed 617.70MB, swap 654.29MB, 0.00 page faults/sec, user CPU 896670.400, kernel CPU 1932.130
Mon 01 Jul 2019 08:53:49 PM EEST | climateprediction.net | [mem_usage] hadam4_a033_200710_12_825_011882466_0: WS 613.43MB, smoothed 613.43MB, swap 652.59MB, 0.00 page faults/sec, user CPU 684316.660, kernel CPU 1561.980
Mon 01 Jul 2019 08:53:49 PM EEST | | [mem_usage] BOINC totals: WS 1235.54MB, smoothed 1235.54MB, swap 1312.59MB, 0.00 page faults/sec
Mon 01 Jul 2019 08:53:49 PM EEST | | [mem_usage] All others: WS 2146.10MB, swap 12725.41MB, user 71628.510s, kernel 42740.070s
Mon 01 Jul 2019 08:53:49 PM EEST | | [mem_usage] non-BOINC CPU usage: 6.01%
ID: 60537 · Report as offensive     Reply Quote
Hal Bregg

Send message
Joined: 20 Nov 18
Posts: 20
Credit: 816,342
RAC: 1,139
Message 60607 - Posted: 6 Jul 2019, 19:30:42 UTC

Hello,

I snatched one of those tasks on my Linux host but after 2 days of crunching WU failed with no credit granted.

Was WU faulty?

https://www.cpdn.org/cpdnboinc/result.php?resultid=21733569
ID: 60607 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,016,442
RAC: 21,024
Message 60608 - Posted: 6 Jul 2019, 19:39:38 UTC - in response to Message 60607.  

Hello,

I snatched one of those tasks on my Linux host but after 2 days of crunching WU failed with no credit granted.

Was WU faulty?

https://www.cpdn.org/cpdnboinc/result.php?resultid=21733569


Three trickles have been reported so I imagine you will get credit for them by end of play tomorrow when the weekly credit script has run. (assuming all goes to plan!)


The -ve theta detected means the model has produced an impossible climate for example a negative air pressure. Some tasks in a batch often push the limits of what might happen and if they are too far out this happens. That result is still useful.
ID: 60608 · Report as offensive     Reply Quote
Hal Bregg

Send message
Joined: 20 Nov 18
Posts: 20
Credit: 816,342
RAC: 1,139
Message 60617 - Posted: 8 Jul 2019, 8:15:28 UTC - in response to Message 60608.  

Hello,

I snatched one of those tasks on my Linux host but after 2 days of crunching WU failed with no credit granted.

Was WU faulty?

https://www.cpdn.org/cpdnboinc/result.php?resultid=21733569


Three trickles have been reported so I imagine you will get credit for them by end of play tomorrow when the weekly credit script has run. (assuming all goes to plan!)


The -ve theta detected means the model has produced an impossible climate for example a negative air pressure. Some tasks in a batch often push the limits of what might happen and if they are too far out this happens. That result is still useful.


Great to hear it was nothing to do with host. I will carry on crunching.
ID: 60617 · Report as offensive     Reply Quote

Questions and Answers : Unix/Linux : 8.09 UK Met Office HadAM4 at N144 resolution

©2024 cpdn.org