Message boards : Number crunching : wah2_eu25 work units are BIG
Message board moderation
Author | Message |
---|---|
Send message Joined: 15 Jan 06 Posts: 637 Credit: 26,751,529 RAC: 653 |
I place the BOINC Data folder on a 14 GB ramdisk on an i7-3770 machine where I run a maximum of six wah2_eu25 work units at a time, but that was not enough memory. One of them errored out with an insufficient storage space message. Looking into it, I see that each work unit takes at least 2 GB, and I have seen up to 3.6 GB. So be forewarned, they are big. |
Send message Joined: 6 Aug 04 Posts: 264 Credit: 965,476 RAC: 0 |
They are also slow, at least on my Windows 10 PC with 20 GB RAM and 2 TB disk. It is true that it hosts also VirtualBox 5.0.10 and 4 BOINC CERN projects, plus the CERN 2015 Challenge outside BOINC. Tullio |
Send message Joined: 15 Jan 06 Posts: 637 Credit: 26,751,529 RAC: 653 |
They will take about 9 1/2 days on the virtual cores of my i7-3770 machines, and about 6 1/2 days on the full cores of my i5-3550. That is OK; they a presumably getting some real work done, and I run the machines 24/7 anyway. I can deal with the memory too; after this group is finished, I will increase the ramdisk to 24 GB; that should do it. But it comes as a bit of a surprise, and I think there should have been some warning. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
I did "sort of" mention this at the bottom, in the first post of this thread. with latest land cover model And I've mentioned a lot earlier that the newer models are/will be, using the newest version of the MOSES land scheme (which is version II), and also TRIFFID Both of these are land use schemes, and details can be found by a web search. They are also used for one of the "Linux only" models. (See the Server Status page.) These newest modelling versions are much more detailed than those used in earlier work here, say up to 2 years ago, so they take a lot longer to calculate, and produce lots more data to return to the projects using them. And then there's this at the end of the MaRIUS project page: This will in particular involve the inclusion of an improved land-surface scheme into both the global and regional model, and a higher-resolution regional model at 25km. I'm afraid that I'm often guilty of assuming that those who post here regularly, also read every new post, and then looks for more info when something new is mentioned. Later, after I've gotten some of today out of the way, I'll make a News item, repeating a lot of this, plus anything else that I think of. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
I've decided to make it a separate sticky post at the top of Number crunching. It's called: Models are getting more detailed, and therefore bigger Also, don't forget that failed models don't clean up after them selves. And neither do the "short" models whether Success or Fail. edit But it's possible that the "short" models MAY clean up on Windows. The best way to deal with shrinking file space is to have a look for ANY left overs. Keeping a human in the loop can help with cpdn. :) |
Send message Joined: 15 May 09 Posts: 4540 Credit: 19,039,635 RAC: 18,944 |
And neither do the "short" models whether Success or Fail. I thought Les from reading posts on this that the short ones do clean up on windows and it was only the Linux versions which got pulled anyway that didn't? |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
I'm not sure about it. You could be right. I'll change the post a bit. |
Send message Joined: 15 May 09 Posts: 4540 Credit: 19,039,635 RAC: 18,944 |
Are some bigger than others? http://climateapps2.oerc.ox.ac.uk/cpdnboinc/show_host_detail.php?hostid=1384830 This machine is running 4 tasks under WINE. Two of them are from batch no. 366. One is progressing at a rate of about 4% a day while the other is only managing 1% a day. Machine info as follows. GenuineIntel Intel(R) Pentium(R) CPU N3540 @ 2.16GHz [Family 6 Model 55 Stepping 8] Number of processors 4 Coprocessors --- Operating System Microsoft Windows 8.1 Ultimate N x64 Edition, , (06.03.9600.00) BOINC client version 7.6.22 Memory 7863.95 MB Cache 1024 KB Swap space 7893.95 MB Total disk space 458.32 GB Free Disk Space 388.98 GB Measured floating point speed 472.1 million ops/sec Measured integer speed 3486.25 million ops/sec Nothing else running on machine 95% of time and during the 5% or less only a little light browsing. |
Send message Joined: 27 Jan 05 Posts: 74 Credit: 1,047,809 RAC: 0 |
Dave: How does this compare? :On a WIN10 laptop XPS14 Four wah2_eu25 WUs, wah2_eu25_Odbv.....etc 86.316% progress @ 12d 04:14:06 three others range from 83.6% to 85.7% same elapsed time. Question: do trickles show up during progress or only on upload of finished WU, and are there graphics associated with these tasks? ( I am not having much luck with installing CPDN on WINE Linux Mint, but have not yet given up} Created 12 May 2016 15:34:01 UTC "(same date as restarting with CPDN)" Total credit 15,278 Average credit 1,170.27 Cross project credit BOINCstats.com Free-DC CPU type GenuineIntel Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz [Family 6 Model 58 Stepping 9] Number of processors 4 Coprocessors NVIDIA GeForce GT 630M (1024MB) driver: 35362 Operating System Microsoft Windows 10 Core x64 Edition, (10.00.10586.00) BOINC client version 7.6.22 Memory 8069.75 MB Cache 256 KB Swap space 9349.75 MB Total disk space 457.85 GB Free Disk Space 197.06 GB Measured floating point speed 3098.05 million ops/sec Measured integer speed 8580.18 million ops/sec Average upload rate 64.51 KB/sec Average download rate 126.52 KB/sec Average turnaround time 0 days Maximum daily WU quota per CPU 0/day Tasks 4 Number of times client has contacted server 53 Last time contacted server 4 Jun 2016 2:59:54 UTC % of time BOINC client is running 77.9231 % While BOINC running, % of time host has an Internet connection 90.4781 % While BOINC running, % of time work is allowed 99.967 % Task duration correction factor |
Send message Joined: 15 May 09 Posts: 4540 Credit: 19,039,635 RAC: 18,944 |
Yours all seem to show broadly similar percentages for elapsed time. What seems particularly strange to me about this one is that they are both from the same batch (366) and one of them seems to be progressing three to four times as fast as the other. Sadly no graphics on WAH2 tasks to look for clues. Will have a look at event log for checkpoint times and possible email someone who knows about what is happening at project level. |
Send message Joined: 7 Aug 04 Posts: 2187 Credit: 64,822,615 RAC: 5,275 |
Yours all seem to show broadly similar percentages for elapsed time. What seems particularly strange to me about this one is that they are both from the same batch (366) and one of them seems to be progressing three to four times as fast as the other. If you are talking about the two tasks that were downloaded on May 5th, one trickled first on May 7, and the other trickled first on May 14. It looks like one didn't have to wait very long to start after downloading, but the other had to wait quite awhile for another task to finish before it could start? Unless I'm mixing up the tasks you are comparing? |
Send message Joined: 27 Jan 05 Posts: 74 Credit: 1,047,809 RAC: 0 |
Dave and Geophi: The three workunits referenced below will finish in about ten hours. I have set preferences to prevent new downloads because I may be able to improve performance with some adjustments to preferences. However, I dont know how to evaluate the performance. Can you look at these in the log file and provide some guidance. And, will there be more workunits provided for Linux in the near future? Thanks for the help. |
Send message Joined: 16 Jan 10 Posts: 1084 Credit: 7,824,485 RAC: 4,956 |
Just got a mex50 model - wah2_mex50_fo0v_193812_12_407_010607167_1. New one on me ... |
Send message Joined: 27 Jan 05 Posts: 74 Credit: 1,047,809 RAC: 0 |
Interesting: First, because I have not seen Iain Inglis for ten years, and second because I have been waiting for downloads after 4 completions for 18 hours, with server stats showing more than 67000 units ready for blastoff. However, I did see in Applications that Windows 10 is not listed. So maybe I have some kind of anomaly here. Hold on, let me look at my Linux rig. Les Bayliss, Can you add anything here? |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
Iain Mexico has just joined up. It's mentioned on the front page, bottom right, under Experiments Coming Soon |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
John The only thing wrong with Windows 10, is that MicroSoft has taken over people's computers with that version. Different problems with different versions, and with how it's set up. (And apparently you can't do anything about it with the Home version.) Your tasks list shows that you have 4 new models there. Which probably happened 5 seconds after you posted. :) You'll need to look at your Event log, stdoutdae.txt, and stderrdae.txt files etc, to see why they were delayed. I'll email "them" about adding "10" to the applications list. |
Send message Joined: 27 Jan 05 Posts: 74 Credit: 1,047,809 RAC: 0 |
Righto mate, thanks. Now, I found a small map a while back which showed all the wah2 regions, but now I dont know where that map is. However, Northeastern North America e.g. Quebec to Newfoundland and out to Greenland was not included. That may be a useful study area because the most recent weather has been absolutely haywire. Who decides what areas are to be studied and how is all this coordinated. This is not clear in the CPDN Home page info etc. I will follow up on Experiments Coming Soon as you recommended. Thanks Les. P.S. Win10 does try to run your life if you let it, and just for relief, I have fired up an old IBM756CSE laptop which has OS2 and WIndows 3.1 loaded. Life was simpler then, but we survived, and spent less time messing with the hardware. |
Send message Joined: 7 Aug 04 Posts: 2187 Credit: 64,822,615 RAC: 5,275 |
The map of regions is in this post: http://climateapps2.oerc.ox.ac.uk/cpdnboinc/forum_thread.php?id=8069&nowrap=true#54016 |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
Hi John The regions map is in the page about the Mac versions All new models can now be run on Macs It's a bit small and gloomy, but if you save it, and then use Windows explorer to open it, you can expand it. Areas to be studied are decided by a research centre interested in that area, mostly by people who are in it I guess. How they get to know about cpdn, is probably via conferences and such. But the big problem is money - they need to convince the people paying their wages that it's a good idea, part of which is the cost to lease space on a big server somewhere, and also pay for the large net connections to it. This is what caused the long delay between testing the ANZ models, and them going into production. Edit I was a bit slow, but now you have two answers. :) |
Send message Joined: 4 Jul 15 Posts: 63 Credit: 3,223,760 RAC: 0 |
Re: Windows 10. Yes it is a quagmire for the privacy-minded. There are a number of guides on the web re: how to configure to decrease exposure to MS snooping. Also, some tools are available to help reduce MS tracking on Win10. This one seems to work and is fairly easy to use: see DisableWinTracking at https://github.com/10se1ucgo/DisableWinTracking |
©2024 cpdn.org