climateprediction.net (CPDN) home page
Thread 'New work Discussion'

Thread 'New work Discussion'

Message boards : Number crunching : New work Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 39 · 40 · 41 · 42 · 43 · 44 · 45 . . . 91 · Next

AuthorMessage
lazlo_vii

Send message
Joined: 11 Dec 19
Posts: 108
Credit: 3,012,142
RAC: 0
Message 62013 - Posted: 22 Jan 2020, 21:19:48 UTC - in response to Message 62011.  
Last modified: 22 Jan 2020, 21:31:03 UTC

...One thing that I've learned, is that 12 models are too many for an AMD 3900X. :(
Next batch will be 8.



You got a 3900X? Congrats and welcome to the Red Team.

Edit: Les, I just took a look at the trickles from that system and I wanted to say that 45 sec/TS is more than double what I am getting on my systems. You might be better off pausing half of this first batch just to speed things up.
ID: 62013 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,042,229
RAC: 14,816
Message 62014 - Posted: 22 Jan 2020, 23:43:30 UTC - in response to Message 62013.  

For comparison I'm running 3 cores on an i5 in VMware and they are taking about 35sec/ts. That's with 8Gb allocated RAM and 6Mb L3 cache.
ID: 62014 · Report as offensive
lazlo_vii

Send message
Joined: 11 Dec 19
Posts: 108
Credit: 3,012,142
RAC: 0
Message 62016 - Posted: 23 Jan 2020, 0:04:32 UTC - in response to Message 62014.  
Last modified: 23 Jan 2020, 0:05:20 UTC

For comparison I'm running 3 cores on an i5 in VMware and they are taking about 35sec/ts. That's with 8Gb allocated RAM and 6Mb L3 cache.


Alan, that is impressive when you consider the system overhead for virtualization. What version ESXi are you running? I haven't used it since about 2010 and while it was OK at raw computing back then I still lost over 15% of the CPU cycles to overhead. The real killer back then was IO though. Disk access was horrible and it took 8 hours (yes, 8 hours) to install CentOS in a VM. Once it was done I could just clone the VM instead of reinstalling, thank god.
ID: 62016 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 62017 - Posted: 23 Jan 2020, 5:57:06 UTC

I'm running them. It takes a little time to get between checkpoints on these N216's though. I think that is why the Server Status page is under reporting the number of active users in the last 24 hours. I could be wrong though.


I think that figure is the number of users who have completed tasks.
ID: 62017 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,042,229
RAC: 14,816
Message 62018 - Posted: 23 Jan 2020, 23:17:02 UTC - in response to Message 62016.  

VMware workstation 15 player with UBUNTU 18.4 LTS as the guest OS. Native OS is Windows10 1903 edition.
ID: 62018 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,042,229
RAC: 14,816
Message 62020 - Posted: 24 Jan 2020, 8:57:29 UTC - in response to Message 62018.  

Correction - it is UBUNTU 18.0.4 LTS.
ID: 62020 · Report as offensive
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 62024 - Posted: 24 Jan 2020, 15:47:07 UTC - in response to Message 62008.  

Will likely be gone fast.

They seem to have disappeared before you could speak the words.
Linux, on the other hand, is hopeless. There used to be people around here who could run them.
Gone with the Wind.


Its starting to look like the researchers have overreached what the average home computer can handle. There don’t seem to be that many home boxes that are running Linux and can meet the huge memory and L2 requirement out there as they expected. So they Linux work sits and sits. Meantime for Windows users the cupboard is usually bare. Keep this up and they could kill this project.
ID: 62024 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 62025 - Posted: 24 Jan 2020, 15:58:13 UTC - in response to Message 62024.  
Last modified: 24 Jan 2020, 16:00:50 UTC

Its starting to look like the researchers have overreached what the average home computer can handle. There don’t seem to be that many home boxes that are running Linux and can meet the huge memory and L2 requirement out there as they expected.

True, though I still see a lot of failures at 0 seconds, which probably means lack of 32-bit libraries.
Also, I think they should tell us what is going on. It seems that the Windows machines have better descriptions of what they are working on.

EDIT: I have sized my machines to handle the OpenIFS, so that will be more interesting for me, if it ever gets here.
ID: 62025 · Report as offensive
lazlo_vii

Send message
Joined: 11 Dec 19
Posts: 108
Credit: 3,012,142
RAC: 0
Message 62026 - Posted: 24 Jan 2020, 17:19:56 UTC - in response to Message 62024.  

According to https://www.cpdn.org/host_stats.php there are 1235 Linux systems with recent credit (the page doesn't define "recent" but I would guess that means credit within 30 or 60 days) and it's been at about 1200 since I started crunching last month. I can't say how many of them are working on the current batch of N216's but I am sure some of them are and others have a backlog of N144's they downloaded.

At any rate most systems built within the last 10 years will be able to run either project as long as they have the 32bit libraries installed. The people that just "set it and forget" might not be getting optimal performance out of their hardware but I am sure these are things the researchers take into account before the public launch of any new model.
ID: 62026 · Report as offensive
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,828,627
RAC: 4,993
Message 62027 - Posted: 24 Jan 2020, 17:42:12 UTC

Maybe with Microsoft pulling support for Windows 7 there will be people thinking of a PC upgrade - I am, though my current machine is perfectly alright for my own and CPDN’s normal purposes. I’ve tried Windows 10 upgrades but just get blue screens of death, so I can see why Microsoft recommends a new PC, though I don’t see why they couldn’t have worked a bit harder on the upgrades. A new machine with enough memory to run a virtual Linux should be easy enough: I’ll run Linux models on that when they’re available, keep my current Windows 7 machine until it becomes risky, and retire my oldest Windows 7 PC (which is already struggling).
ID: 62027 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 62028 - Posted: 24 Jan 2020, 17:43:24 UTC

Its starting to look like the researchers have overreached what the average home computer can handle. There don’t seem to be that many home boxes that are running Linux and can meet the huge memory and L2 requirement out there as they expected.


Both my boxes will run them and the desktop is over ten years old. It just doesn't like the 216's if I try to run them on both cores at once. They are really unsuitable for the set and forget brigade.

And first post asking for help installing the 32 bit libs in a while today so with luck one less cruncher with boxes crashing everything!
ID: 62028 · Report as offensive
Bryn Mawr

Send message
Joined: 28 Jul 19
Posts: 150
Credit: 12,830,559
RAC: 228
Message 62030 - Posted: 24 Jan 2020, 18:59:51 UTC - in response to Message 62024.  

Will likely be gone fast.

They seem to have disappeared before you could speak the words.
Linux, on the other hand, is hopeless. There used to be people around here who could run them.
Gone with the Wind.


Its starting to look like the researchers have overreached what the average home computer can handle. There don’t seem to be that many home boxes that are running Linux and can meet the huge memory and L2 requirement out there as they expected. So they Linux work sits and sits. Meantime for Windows users the cupboard is usually bare. Keep this up and they could kill this project.


Then why are there 10 times the number of Windows WUs sitting on users computers waiting to be processed than are sitting on Linux boxes?
ID: 62030 · Report as offensive
Harri Liljeroos

Send message
Joined: 9 Dec 05
Posts: 116
Credit: 12,547,934
RAC: 2,738
Message 62031 - Posted: 24 Jan 2020, 22:35:17 UTC

If they really need to get those Linux tasks done, they could think about the VirtualBox environment of Boinc. Do what LHC@home is doing and create Linux VirtualBox machines with everything set up the way they want and need, and send them to the users for running. This could solve the 32 bit libraries problem as well.
ID: 62031 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,042,229
RAC: 14,816
Message 62032 - Posted: 24 Jan 2020, 23:35:14 UTC - in response to Message 62018.  

I think it also helps that these are effectively "liote" versions of both VMware and UBUNTU. Also the machine is basically 5years old so not the newest by any means.
ID: 62032 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 62033 - Posted: 24 Jan 2020, 23:55:21 UTC

The researcher using these hadam4's is aware of the slowness, and doesn't mind a bit of a wait.
He can also see the statistics of the runs; waiting/running/failing/successful, so he knows how things are.

And the project has several "irons in the fire" about the 32 bit problem.

Relax. Do something soothing, like fishing. :)
ID: 62033 · Report as offensive
wolfman1360

Send message
Joined: 18 Feb 17
Posts: 81
Credit: 14,029,434
RAC: 1,176
Message 62034 - Posted: 25 Jan 2020, 0:46:04 UTC

The n216 tasks will definitely take a while.
My poor core 2 duo e8400 was one of the machines I forgot to run an app_config on. Consequently, as a prime example of set it and forget it, 36% of an n216 completed after 8 days of work.
An n144 on a w3530 has taken 8 days to reach 70%, on the other hand.
Of course these are examples of 10 year old hardware at this point, so this is worst case, though the machines in question do not get shut down or work suspended.
Remember there used to be single workunits completed on single core machines that in some cases took months. So long as the work is getting done and being sent up to the servers when it is completed in a reasonable (of course there is the subjective part) amount of time and can be of help to the researchers, this is what is important to me.
I may invest in a Ryzen 3600 or 3700x at some point. Not right now. May even wait for the 4000.
ID: 62034 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 62035 - Posted: 25 Jan 2020, 2:02:50 UTC - in response to Message 62034.  

I may invest in a Ryzen 3600 or 3700x at some point. Not right now. May even wait for the 4000.

Might as well save money and get the 3600. You can use only about four cores anyway (maybe five) on N216.
Then get the 4000 later as well.
ID: 62035 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,042,229
RAC: 14,816
Message 62042 - Posted: 25 Jan 2020, 19:40:30 UTC - in response to Message 62009.  

Mine are taking about 45mins between checkpoints.
ID: 62042 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 62043 - Posted: 25 Jan 2020, 22:45:57 UTC - in response to Message 62042.  

That is interesting. I have never used checkpoints for that before, and don't know how consistent they are. But my Ryzen 3600 is checkpointing in about 20 minutes. That is running BOINC on only six of the twelve cores: four of them are on N216, and the other two are on Rosetta. You fall off a cliff somewhere above that.
ID: 62043 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 62045 - Posted: 26 Jan 2020, 16:11:10 UTC - in response to Message 62043.  

And on my i7-9700 (which has eight full cores), it checkpoints at 23 minutes. But that is again with limiting the N216 to running on only four cores. The other four cores are on TN-Grid, which seems to be an easy project for this purpose.

In general, I find that I need to limit any of my CPUs (Intel coffee lake or Ryzen) to four cores for the N216, but can put just about anything else on the other cores without much ill effect. Beyond four cores, it drops off a cliff.
ID: 62045 · Report as offensive
Previous · 1 . . . 39 · 40 · 41 · 42 · 43 · 44 · 45 . . . 91 · Next

Message boards : Number crunching : New work Discussion

©2024 cpdn.org