climateprediction.net (CPDN) home page
Thread 'New work discussion - 2'

Thread 'New work discussion - 2'

Message boards : Number crunching : New work discussion - 2
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 17 · 18 · 19 · 20 · 21 · 22 · 23 . . . 42 · Next

AuthorMessage
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 66975 - Posted: 20 Dec 2022, 12:17:21 UTC - in response to Message 66974.  

"Soon if all goes well." Is what I have seen but whose definition of soon? At least another day before the next OIFS tasks is likely.


Ready when you are....

If you send me a bunch of OpenIFS 43r3 Perturbed Surface and a bunch of OpenIFS 43r3 Baroclinic Lifecycle I will run three of each at a time. I have had no failures yet. I do not suppose they have to be 1.05 or 1.07, but those are the versions I have most recently run.
ID: 66975 · Report as offensive
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,432,494
RAC: 17,331
Message 66977 - Posted: 20 Dec 2022, 13:44:58 UTC
Last modified: 20 Dec 2022, 13:53:57 UTC

OpenIFS Perturbed Surface batches

The next set of batches for the OpenIFS Perturbed Surface application are going out today/tomorrow. CPDN have asked me to write a short item on them as the scientist running the project is away. He will write something more complete for the CPDN website in due course.

The aim of this project is to understand how predictable the weather events over Europe were in 2021. The forecasts for 2021 have already gone out. The next batches will cover the 39 yrs from 2020 backwards. These are called 'hindcasts' and are used to establish a baseline in order to compare with the forecasts of 2021 to establish whether differences in the forecasts are significant or not, in a statistical sense.

For each of the 39 years, there will be 1000 forecasts. Each forecast modifies the way the Earth's surface is treated in the model by small amounts. This means each forecast creates a slightly different outcome. By using 39 years, this allows for variability in the climate (interannual variability). In total, there will be 39,000 tasks.

These 'perturbed surface' forecasts will be compared with a set of control forecasts without perturbations. These are being run offline outside of CPDN.

From a technical point of view, each batch will be the same as has been sent out before for the year 2021; same model runtime, same output sizes. There were a few of these forecasts that failed due to the model being perturbed in a way that was no longer physical realizable. Although a 'fail' this is still useful information for the scientist.

I hope I've not misrepresented anything - I'm sure the scientist will write something better in due course!

@moderators - should I post this on any other forum threads? Or wait until the scientist involved writes something? (which I'd prefer).
ID: 66977 · Report as offensive
klepel

Send message
Joined: 9 Oct 04
Posts: 82
Credit: 69,921,875
RAC: 8,353
Message 68653 - Posted: 15 Apr 2023, 22:14:23 UTC - in response to Message 66977.  

I need your help!
I updated Lubuntu to version 22.04 the day before yesterday.
As I have already read in the forums, that boinc won’t work anymore, however I hoped it would.
I know that Gianfranco´s ppa is the solution (Boinc version: 7.20.5, my old version is 7.14.2), that it runs again. However I would like to install boincmgr, boinccmd, boinc in my old folder under// home/…/boinc and not in the standard folder – where ever it is.
So my question: How do I down-load the ppa without installing it in the default folder or where can I find the program files of new versions after installing as ppa? So I am able to transfer those to the desired location.
Thanks a lot!
ID: 68653 · Report as offensive
klepel

Send message
Joined: 9 Oct 04
Posts: 82
Credit: 69,921,875
RAC: 8,353
Message 68654 - Posted: 15 Apr 2023, 22:23:29 UTC - in response to Message 68653.  
Last modified: 15 Apr 2023, 22:44:43 UTC

I need your help!
I updated Lubuntu to version 22.04 the day before yesterday.
As I have already read in the forums, that boinc won’t work anymore, however I hoped it would.
I know that Gianfranco´s ppa is the solution (Boinc version: 7.20.5, my old version is 7.14.2), that it runs again. However I would like to install boincmgr, boinccmd, boinc in my old folder under// home/…/boinc and not in the standard folder – where ever it is.
So my question: How do I down-load the ppa without installing it in the default folder or where can I find the program files of new versions after installing as ppa? So I am able to transfer those to the desired location.
Thanks a lot!

Solved: Found the programms, transfered these and it works somehow.
ID: 68654 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4536
Credit: 18,999,047
RAC: 21,571
Message 68695 - Posted: 9 May 2023, 19:39:43 UTC

Just to say, all four of the East Asia tasks I was running from testing completed successfully. The researcher will be generating some Windows tasks from those spinups. "Soon" Was the word she used but I don't know what that means in her context.
ID: 68695 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 68696 - Posted: 10 May 2023, 3:43:28 UTC - in response to Message 68695.  

Thanks Dave, I have 7 Windows and 1 Linux machine waiting for things to get going.
ID: 68696 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 68697 - Posted: 10 May 2023, 4:19:24 UTC - in response to Message 68696.  
Last modified: 10 May 2023, 4:20:47 UTC


Thanks Dave, I have 7 Windows and 1 Linux machine waiting for things to get going.


Not quite the same. My main machine is a Linux one like this.
CPU type 	GenuineIntel
Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7]
Number of processors 	16

Operating System 	Linux Red Hat Enterprise Linux
Red Hat Enterprise Linux 8.7 (Ootpa) [4.18.0-425.13.1.el8_7.x86_64|libc 2.28]
BOINC version 	7.20.2
Memory 	125.34 GB
Cache 	16896 KB
Swap space 	15.62 GB
Total disk space 	488.04 GB
Free Disk Space 	475.73 GB
Measured floating point speed 	6.05 billion ops/sec
Measured integer speed 	25.61 billion ops/sec
Average upload rate 	212.24 KB/sec
Average download rate 	18546.14 KB/sec
Average turnaround time 	0.65 days


And my other one is a Windows machine.
CPU type 	GenuineIntel
11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz [Family 6 Model 140 Stepping 1]
Number of processors 	8

Operating System 	Microsoft Windows 10
Core x64 Edition, (10.00.19045.00)
BOINC version 	7.20.2
Memory 	15.64 GB
Cache 	256 KB
Swap space 	18.02 GB
Total disk space 	460.73 GB
Free Disk Space 	329.54 GB
Measured floating point speed 	4.27 billion ops/sec
Measured integer speed 	17.82 billion ops/sec
Average upload rate 	121.25 KB/sec
Average download rate 	4970.77 KB/sec
Average turnaround time 	7.75 days


Funny that their upload and download speeds are so different since they both go through the same router to my fiber-optic Internet connection.
ID: 68697 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 68698 - Posted: 10 May 2023, 5:03:49 UTC - in response to Message 68697.  

Not quite the same.
I do go overboard. I'm obsessed with collecting stuff and repairing old hardware. Way too much electricity is used here.

Funny that their upload and download speeds are so different since they both go through the same router to my fiber-optic Internet connection.
Are you looking at the ones on the CPDN page? Is that for CPDN downloads? Those will be affected by the server speed at the time. Even if it's cross project, it depends as much on the project as yourself. Mine are also random, even two identical xeons connected to the same router on adjacent ports. The slowest machine I have (Picard) reports 3MB/sec, twice one of the xeons. Also, I run a squid cache on my main computer, so Boinc has seen 800Mbit downloads even though my internet is only capable of 32Mbit. I put that in there because I run LHC, and their virtualbox programs are stupid, they don't keep a local store outside themselves, so I can have over 100 VB instances running across 8 machines, all downloading the same dataset. To save load on the LHC servers, I store everything (for all Boinc) on my main machine.
ID: 68698 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4536
Credit: 18,999,047
RAC: 21,571
Message 68735 - Posted: 15 May 2023, 12:42:55 UTC

Just seen a request for the next spin up runs for East Asia to be prepared. Not sure if they are for main site or testing. Have set up my Windows client for both main site and testing in case they arrive while i am out.
ID: 68735 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 68736 - Posted: 15 May 2023, 13:14:01 UTC
Last modified: 15 May 2023, 13:14:19 UTC

Thanks Dave. My 8 Windows machines always have CPDN checking for new work (they do other stuff but since CPDN has rare work, they're always trying to get that first), although they probably only check about twice a day since Boinc backs off when there's no work available. Between 8 computers I might get something. I'm not a tester though. One of them has a Linux virtual machine to get those type of tasks too, not had much luck with that either though. This is like going fishing! Usually ends up as several hours of idle chatter with friends and no fish. I think I asked before and you don't need more testers :-(
ID: 68736 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4536
Credit: 18,999,047
RAC: 21,571
Message 68737 - Posted: 15 May 2023, 15:29:31 UTC - in response to Message 68736.  

I have asked whether these are main or testing branch tasks. Probably won't get a chance to check on whether there is an answer till tomorrow.
ID: 68737 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4536
Credit: 18,999,047
RAC: 21,571
Message 68738 - Posted: 15 May 2023, 16:38:04 UTC - in response to Message 68737.  

I have asked whether these are main or testing branch tasks. Probably won't get a chance to check on whether there is an answer till tomorrow.


They will be shorter runs, Maximum 2 years (model years) to run on the main site.
ID: 68738 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4536
Credit: 18,999,047
RAC: 21,571
Message 68754 - Posted: 16 May 2023, 16:44:13 UTC - in response to Message 68738.  

It may well be that the East Asia tasks are waiting for the last issues from the server move to be sorted out.
ID: 68754 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 68755 - Posted: 16 May 2023, 17:47:06 UTC - in response to Message 68754.  

Well, I do have a Windows machine that is willing to try. But it is a pipsqueak.

CPU type 	GenuineIntel
11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz [Family 6 Model 140 Stepping 1]
Number of processors 	8
Operating System 	Microsoft Windows 10
Core x64 Edition, (10.00.19045.00)
BOINC version 	7.20.2
Memory 	15.64 GB
Cache 	  256 KB
Swap space 	18.02 GB
Total disk space 	460.73 GB
Free Disk Space 	329.54 GB
Measured floating point speed  4.27 billion ops/sec
Measured integer speed 	17.82 billion ops/sec
Average upload rate 	 121.25 KB/sec
Average download rate 	4970.77 KB/sec
Average turnaround time 7.75 days

ID: 68755 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 68756 - Posted: 16 May 2023, 19:15:08 UTC

And I have two 24 core Ryzen 9 3900XTs waiting. One will run Linux or Windows, the other is only Windows. Just upgraded them to 96GB RAM each.

Gimme something or LHC continues to get my power :-P
ID: 68756 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 68793 - Posted: 22 May 2023, 17:25:50 UTC - in response to Message 68756.  

My main (Linux) machine is kind-of busy. This is what is is doing for Boinc. Working hard on those hadam4 tasks.

    PID    PPID USER      PR  NI S    RES  %MEM  %CPU  P     TIME+ COMMAND                                                                   
 917749   18695 boinc     39  19 R   3.2g   2.5  98.8 15 144:19.87 ../../projects/einstein.phys.uwm.edu/einstein_O3MD1_1.03_x86_64-pc-linux+ 
 783091  783081 boinc     39  19 R   1.4g   1.1  98.8  5   2236:09 /var/lib/boinc/projects/climateprediction.net/hadam4_um_8.52_i686-pc-lin+ 
 778206  778196 boinc     39  19 R   1.4g   1.1  99.1  3   2303:44 /var/lib/boinc/projects/climateprediction.net/hadam4_um_8.52_i686-pc-lin+ 
 865645  865423 boinc     39  19 R   1.4g   1.1  98.8  6 940:50.61 /var/lib/boinc/projects/climateprediction.net/hadam4_um_8.52_i686-pc-lin+ 
 912483   18695 boinc     39  19 R   1.3g   1.0  98.7  2 231:14.63 ../../projects/boinc.bakerlab.org_rosetta/rosetta_4.20_x86_64-pc-linux-g+ 
 904124   18695 boinc     39  19 R 213984   0.2  98.8  1 345:39.97 ../../projects/einstein.phys.uwm.edu/einsteinbinary_BRP4G_1.33_x86_64-pc+ 
 924235   18695 boinc     39  19 R  72996   0.1  98.8 13  44:16.64 ../../projects/www.worldcommunitygrid.org/wcgrid_mcm1_map_7.61_x86_64-pc+ 
 924280   18695 boinc     39  19 R  72996   0.1  98.8  7  43:05.77 ../../projects/www.worldcommunitygrid.org/wcgrid_mcm1_map_7.61_x86_64-pc+ 
 926133   18695 boinc     39  19 S  54848   0.0  98.9 10  10:24.16 ../../projects/www.worldcommunitygrid.org/wcgrid_scc1_vina_7.18_x86_64-p+ 
 923175   18695 boinc     39  19 S  52152   0.0  98.7 15  62:52.00 ../../projects/www.worldcommunitygrid.org/wcgrid_scc1_vina_7.18_x86_64-p+ 
 926385   18695 boinc     39  19 S  50952   0.0  98.9 14   6:54.91 ../../projects/www.worldcommunitygrid.org/wcgrid_scc1_vina_7.18_x86_64-p+ 
 924314   18695 boinc     39  19 S  47728   0.0  98.9 10  42:15.33 ../../projects/www.worldcommunitygrid.org/wcgrid_scc1_vina_7.18_i686-pc-+ 
  18695       1 boinc     30  10 S  46084   0.0   0.1  4 152917:44 /usr/bin/boinc                                                            
 865423   18695 boinc     39  19 S  17908   0.0   0.0 14   0:34.94 ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu hadam+ 
 783081   18695 boinc     39  19 S  17796   0.0   0.1 10   0:58.87 ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu hadam+ 
 778196   18695 boinc     39  19 S  17368   0.0   0.1 14   1:00.92 ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu hadam+ 

ID: 68793 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 68794 - Posted: 23 May 2023, 1:13:47 UTC

WARNING, the current user has no knowledge of Linux! Please speak in English to me only.

But I am logged on as root :-)

I thought I might be getting the 0 bytes file bug, so I deleted the four 0 byte files, then restarted Boinc, but it tried to get them again and failed. Problem still not fixed at CPDN? Will those files sort themselves or do I have to keep deleting them to try again?

33 Tue, 23/5/2023 02:10:05 AM Checking presence of 10 project files
34 climateprediction.net Tue, 23/5/2023 02:10:05 AM Resetting file projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu: permanent HTTP error
35 climateprediction.net Tue, 23/5/2023 02:10:05 AM Resetting file projects/climateprediction.net/hadam4_data_8.09_i686-pc-linux-gnu.zip: permanent HTTP error
36 climateprediction.net Tue, 23/5/2023 02:10:05 AM Resetting file projects/climateprediction.net/hadam4_se_8.09_i686-pc-linux-gnu.zip: permanent HTTP error
37 climateprediction.net Tue, 23/5/2023 02:10:05 AM Resetting file projects/climateprediction.net/hadam4_um_8.09_i686-pc-linux-gnu.zip: permanent HTTP error
38 climateprediction.net Tue, 23/5/2023 02:10:05 AM Sending scheduler request: To fetch work.
39 climateprediction.net Tue, 23/5/2023 02:10:05 AM Requesting new tasks for CPU
40 climateprediction.net Tue, 23/5/2023 02:10:06 AM Started download of hadam4_8.09_i686-pc-linux-gnu
41 climateprediction.net Tue, 23/5/2023 02:10:06 AM Started download of hadam4_data_8.09_i686-pc-linux-gnu.zip
42 climateprediction.net Tue, 23/5/2023 02:10:08 AM Giving up on download of hadam4_8.09_i686-pc-linux-gnu: permanent HTTP error
43 climateprediction.net Tue, 23/5/2023 02:10:08 AM Giving up on download of hadam4_data_8.09_i686-pc-linux-gnu.zip: permanent HTTP error
44 climateprediction.net Tue, 23/5/2023 02:10:08 AM Started download of hadam4_se_8.09_i686-pc-linux-gnu.zip
45 climateprediction.net Tue, 23/5/2023 02:10:08 AM Started download of hadam4_um_8.09_i686-pc-linux-gnu.zip
46 climateprediction.net Tue, 23/5/2023 02:10:08 AM Scheduler request completed: got 0 new tasks
47 climateprediction.net Tue, 23/5/2023 02:10:08 AM Not sending work. Last request too recent. Please wait 3161 seconds.
48 climateprediction.net Tue, 23/5/2023 02:10:08 AM Project requested delay of 3636 seconds
49 climateprediction.net Tue, 23/5/2023 02:10:10 AM Giving up on download of hadam4_se_8.09_i686-pc-linux-gnu.zip: permanent HTTP error
50 climateprediction.net Tue, 23/5/2023 02:10:10 AM Giving up on download of hadam4_um_8.09_i686-pc-linux-gnu.zip: permanent HTTP error
ID: 68794 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 68795 - Posted: 23 May 2023, 2:06:09 UTC - in response to Message 68794.  

WARNING, the current user has no knowledge of Linux! Please speak in English to me only.

But I am logged on as root :-)

I thought I might be getting the 0 bytes file bug, so I deleted the four 0 byte files, then restarted Boinc, but it tried to get them again and failed. Problem still not fixed at CPDN? Will those files sort themselves or do I have to keep deleting them to try again?


On another project (Rosetta) it was like that for a long while. Gave it up, Using the Boincmgr, I did a reset project, and then tried to restart the project. It tried to download 4 files, one fantastically big
.database_357d5d93529_n_methyl/minirosetta_database
but they were all zero length. After trying for several days, I replaced them with correct files I got from one of my backups. I tried again, but it still would not download the rest of the stuff. I waited many days? weeks? ???, and finally, with no help from me, they fixed their encryption certificate or whatever it was and everything started working again.

So my guess is that it may start working by itself if you do nothing, possess yourself of patience (Advice I could not manage for myself), and wait until their sys-admin gets it straightened out.

Trouble is, I am currently having no trouble getting tasks from CPDN these days; Currently, i have three of them. They are all re-runs like this one:
hadam4h_a05p_200011_5_931_012138767_2
ID: 68795 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 68796 - Posted: 23 May 2023, 2:17:19 UTC - in response to Message 68795.  
Last modified: 23 May 2023, 2:17:42 UTC

Maybe you already have the files I don't, and the server is currently unable to hand them out. Do you have these?

hadam4_8.09_i686-pc-linux-gnu
hadam4_data_8.09_i686-pc-linux-gnu.zip
hadam4_se_8.09_i686-pc-linux-gnu.zip
hadam4_um_8.09_i686-pc-linux-gnu.zip

If so, can you put them on dropbox/etc and I'll see if I get tasks?
ID: 68796 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 68797 - Posted: 23 May 2023, 4:05:48 UTC - in response to Message 68796.  


Maybe you already have the files I don't, and the server is currently unable to hand them out. Do you have these?

hadam4_8.09_i686-pc-linux-gnu
hadam4_data_8.09_i686-pc-linux-gnu.zip
hadam4_se_8.09_i686-pc-linux-gnu.zip
hadam4_um_8.09_i686-pc-linux-gnu.zip

If so, can you put them on dropbox/etc and I'll see if I get tasks?


I do not know anything about dropbox.

I have these hadam4 ones..

The first number is the size in bytes. The date is probably when it was put onto my machine.


-rwxr-xr-x. 1 boinc boinc 3634637 Dec 18  2021 hadam4_8.09_i686-pc-linux-gnu
-rwxr-xr-x. 1 boinc boinc 3635629 Dec 18  2021 hadam4_8.52_i686-pc-linux-gnu
-rwxr-xr-x. 1 boinc boinc   75539 Dec 18  2021 hadam4_data_8.09_i686-pc-linux-gnu.zip
-rwxr-xr-x. 1 boinc boinc   75539 Dec 18  2021 hadam4_data_8.52_i686-pc-linux-gnu.zip

-rwxrwxr-x. 1 boinc boinc 3736644 May  1  2019 hadam4_se_8.09_i686-pc-linux-gnu.so
-rwxr-xr-x. 1 boinc boinc 1166623 Dec 18  2021 hadam4_se_8.09_i686-pc-linux-gnu.zip
-rwxrwxr-x. 1 boinc boinc 3736644 May  1  2019 hadam4_se_8.52_i686-pc-linux-gnu.so
-rwxr-xr-x. 1 boinc boinc 1166623 Dec 18  2021 hadam4_se_8.52_i686-pc-linux-gnu.zip
-rwxrwxr-x. 1 boinc boinc 6574073 May  1  2019 hadam4_um_8.09_i686-pc-linux-gnu
-rwxr-xr-x. 1 boinc boinc 2723666 Dec 18  2021 hadam4_um_8.09_i686-pc-linux-gnu.zip
-rwxrwxr-x. 1 boinc boinc 6574073 May  1  2019 hadam4_um_8.52_i686-pc-linux-gnu
-rwxr-xr-x. 1 boinc boinc 2725214 Dec 18  2021 hadam4_um_8.52_i686-pc-linux-gnu.zip

ID: 68797 · Report as offensive
Previous · 1 . . . 17 · 18 · 19 · 20 · 21 · 22 · 23 . . . 42 · Next

Message boards : Number crunching : New work discussion - 2

©2024 cpdn.org