climateprediction.net (CPDN) home page
Thread 'New work Discussion'

Thread 'New work Discussion'

Message boards : Number crunching : New work Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 32 · 33 · 34 · 35 · 36 · 37 · 38 . . . 91 · Next

AuthorMessage
ProfileBonsai911

Send message
Joined: 9 Sep 04
Posts: 228
Credit: 30,756,611
RAC: 3,303
Message 61160 - Posted: 3 Oct 2019, 10:25:51 UTC - in response to Message 61159.  

Yep, I catched 15 wu,

have a nice day!
Bonsai911
ID: 61160 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 61161 - Posted: 3 Oct 2019, 11:44:21 UTC - in response to Message 61154.  


Pretty sure in my case it was the disk writes slowing things down. Though this was around the time when the temperature in Cambridge set a new UK record. The fan in my laptop is quieter now with 4 cores running than it was then with two! With the IFS I know it was swapping data out to the swap partition from main memory.


Swapping, depending on your paging rate, can kill you.

Most Linux systems can run an app called xosview that will show lots of things going on in your system in essentially real time. It will show how many pages per second are going to and from swap, for example; also the amount of swap actually used. For example: my system has been up 24/7 for 9 days, 12:58; this is not a lot, but it shows it is probably in steady state as it would not be had I just booted it. Boinc comes up automatically when I boot my system. Right now it is running two hadcm3s work units and two hadam4 work units. My system has Swap: 4095996k total, 191512k used, 3904484k free which you might think is quite a lot (though I do not). It would depend on what is out there. I just watched the paging and in the last minute, it paged in 17 pages and paged out none. Pages are 4096 bytes on my system, and my swap is on a 7200rpm SATA hard drive.

My room temperature is 78F; in winter when the heat is on, it is set to 69F. Currently, the hard drive where boinc keep all its files reports:

Oct  3 05:57:52 smartd[11576]: Device: /dev/sdc [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 55 to 54
Oct  3 05:57:52 smartd[11576]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 45 to 46


One might say that that is the internal temperature of my machine.
ID: 61161 · Report as offensive
ian

Send message
Joined: 23 Feb 06
Posts: 20
Credit: 9,217,950
RAC: 3,460
Message 61164 - Posted: 3 Oct 2019, 15:43:11 UTC - in response to Message 61159.  

Oh well missed those too. I now have one task still in progress over 4 machines in intermittent use. There does seem to be a serious problem on the " other side" of this project. As a windows only user the work availability has been poor to say the least - has it been better for linux etc users?. If so why? Is there something in the modelling which is dissuading research teams from producing windows models? Given that we are back to full academic activity across both US and Europe we should be seeing more work forthcoming ( yes I know the bit about postgrads are supposed to work outside of " termtime " I was one once so lets be realistic). I suppose the real question is IF teams are not using boinc cpnd models what are they using? Or, perhaps more radically, has this subject become academically "stale" with little enthusiasm for new entrants to the research field? I.hope it is not the latter.

Is there any indication from the "other side" on work input reduction?
ID: 61164 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 61165 - Posted: 3 Oct 2019, 17:18:40 UTC

The windows tasks, batches 840 and 841 are 28 and 16 month Pacific North West 25km resolution tasks so a bit longer than some recently.
ID: 61165 · Report as offensive
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,826,970
RAC: 5,066
Message 61166 - Posted: 3 Oct 2019, 17:35:27 UTC - in response to Message 61164.  

[ian wrote:]... Is there something in the modelling which is dissuading research teams from producing windows models? ...

My impression is that it’s mostly a question of new model types. The new OpenIFS application, for example, is a big port — and they want to get that working. Likewise the HADAM4 is new.

The recent HADCMS was Linux-only, which is unusual as that application is not new.

I hope a mix appears again or I’ll have to convert a Windows machine to Linux. We’ll see.
ID: 61166 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 61168 - Posted: 3 Oct 2019, 18:01:07 UTC - in response to Message 61161.  

Swapping, depending on your paging rate, can kill you.

When setting up a new machine, I always set:
sudo sysctl vm.swappiness=0

That doesn't eliminate the use of swap entirely, but it discourages it. What you really need for that is a lot of memory.
I always have at least 1 GB per core, or 16 to 64 GB per machine these days. That does the job.
ID: 61168 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 61169 - Posted: 3 Oct 2019, 18:39:52 UTC - in response to Message 61168.  

I always have at least 1 GB per core, or 16 to 64 GB per machine these days. That does the job.


Oh! Goody! I have 4 GBytes per core.
ID: 61169 · Report as offensive
wolfman1360

Send message
Joined: 18 Feb 17
Posts: 81
Credit: 14,026,949
RAC: 972
Message 61170 - Posted: 3 Oct 2019, 18:56:24 UTC - in response to Message 61166.  

[ian wrote:]... Is there something in the modelling which is dissuading research teams from producing windows models? ...

My impression is that it’s mostly a question of new model types. The new OpenIFS application, for example, is a big port — and they want to get that working. Likewise the HADAM4 is new.

The recent HADCMS was Linux-only, which is unusual as that application is not new.

I hope a mix appears again or I’ll have to convert a Windows machine to Linux. We’ll see.

Has anyone tried the Linux subsystem for Windows?
I know it is only for Windows 10, but it might be the best of both worlds. I have a Ryzen that is my daily driver and really don't want to dual boot. I'm just not sure if Boinc has to be installed through Linux or Windows to take advantage of that. Hmm.
The majority of my fleet is Windows but I'll be converting another to Linux in the coming days.
ID: 61170 · Report as offensive
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 61171 - Posted: 3 Oct 2019, 20:09:01 UTC

I've been running Linux as a VMWare guest on my native Windows PCs for awhile now. That is one alternative to converting a whole PC to Linux or dual booting. You may not have as many cores available as you would otherwise, but if that doesn't bother you, it's certainly an option.

In terms of performance, similar to running BOINC in Wine on Linux, I've seen no real degradation in model speed when running in a VM.
ID: 61171 · Report as offensive
wolfman1360

Send message
Joined: 18 Feb 17
Posts: 81
Credit: 14,026,949
RAC: 972
Message 61172 - Posted: 3 Oct 2019, 20:57:21 UTC - in response to Message 61171.  

I've been running Linux as a VMWare guest on my native Windows PCs for awhile now. That is one alternative to converting a whole PC to Linux or dual booting. You may not have as many cores available as you would otherwise, but if that doesn't bother you, it's certainly an option.

In terms of performance, similar to running BOINC in Wine on Linux, I've seen no real degradation in model speed when running in a VM.

That's actually a good option too, especially since I probably wouldn't want to run full out anyway.
Unbelievably, this Lenovo doesn't have AMD V in the bios. Now time to check for a bios update.
Fortunately everything else does, which should add a productivity boost.
ID: 61172 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 61173 - Posted: 3 Oct 2019, 21:08:13 UTC - in response to Message 61164.  

Ian

Andy was taking advantage of the quiet time to upgrade the servers, first on the test site, then the main site.
This wasn't rushed, so that the moderators could check things as he progressed.

************

Work from the external researchers is sent to a holding server at Oxford, where the data is checked for problems before being issued.
So there needs to be someone at Oxford who can do that.

***********

Only a guess, but I think that the researchers will know by now about the constant Windows updates at inconvenient times.
And the bugs that sometimes crop up and cause problems.
So using Linux to do their work is reasonable.

The BIG OpenIFS models are Linux too, if and when they show up.

***********

There's only one way to be sure of getting work with this project these days, and that's to NOT have any other projects on the same computer.
Not going to suit everyone, but the researchers don't care who runs the tasks, just as long as they get run. :)
ID: 61173 · Report as offensive
ian

Send message
Joined: 23 Feb 06
Posts: 20
Credit: 9,217,950
RAC: 3,460
Message 61179 - Posted: 4 Oct 2019, 13:46:46 UTC

Thanks Les and as if by magic I have 8 new tasks!!!
ID: 61179 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 61182 - Posted: 4 Oct 2019, 14:18:17 UTC

Has everyone seen the post in the Linux section about some new Linux work about to be shovelled into the download server?
ID: 61182 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 61183 - Posted: 4 Oct 2019, 14:51:32 UTC - in response to Message 61182.  

Has everyone seen the post in the Linux section about some new Linux work about to be shovelled into the download server?


https://www.cpdn.org/forum_thread.php?id=8841#61180
ID: 61183 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 61225 - Posted: 13 Oct 2019, 20:28:30 UTC - in response to Message 61183.  

I uave seen the post, but not the work units.
ID: 61225 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 61226 - Posted: 14 Oct 2019, 1:54:24 UTC - in response to Message 61225.  

It'll be a little while longer yet.
The post is just to prepare people for some possible outcomes.
ID: 61226 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 61227 - Posted: 14 Oct 2019, 3:37:12 UTC - in response to Message 61226.  

The post is just to prepare people for some possible outcomes.

You are beginning to sound like The Telegraph.
ID: 61227 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 61228 - Posted: 14 Oct 2019, 5:52:35 UTC - in response to Message 61227.  

Has everyone seen the post in the Linux section about some new Linux work about to be shovelled into the download server?


I think they broke the shovel and are picking each one out with a pair of tweezers to put them in the hopper.
ID: 61228 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 61229 - Posted: 14 Oct 2019, 7:15:16 UTC - in response to Message 61228.  

The last estimate of the time for doing these by hand was another two days from now but that was a week ago so things may have changed.
ID: 61229 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 61230 - Posted: 14 Oct 2019, 11:31:30 UTC - in response to Message 61226.  

It'll be a little while longer yet.
The post is just to prepare people for some possible outcomes.


That is OK: I have three hadam4 work units running on my 4-core machine, and a WCG work unit soaking up the spare time in the fourth core.
ID: 61230 · Report as offensive
Previous · 1 . . . 32 · 33 · 34 · 35 · 36 · 37 · 38 . . . 91 · Next

Message boards : Number crunching : New work Discussion

©2024 cpdn.org