climateprediction.net (CPDN) home page
Thread 'New work discussion - 2'

Thread 'New work discussion - 2'

Message boards : Number crunching : New work discussion - 2
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 14 · 15 · 16 · 17 · 18 · 19 · 20 . . . 42 · Next

AuthorMessage
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,716,561
RAC: 8,355
Message 66782 - Posted: 5 Dec 2022, 11:28:13 UTC

There were two fundamental technologies in the early years of the internet: copper wire (the PSTN - public switched telephone network), and coaxial cable (hence 'cable TV').

When I moved to my current house in 1993, cable TV was in its infancy, with separate and distinct franchises in each local government area that wanted it. It had reached the street past my old flat, but not yet the house I moved to - though the survey work had been done, judging by the green spray paint on the pavements.

So I stuck with copper wire. My first modem was switchable between 300/300 baud and 1200/75 baud, and I then progressed up to 56 Kb, ISDN, and finally ADSL - when 512 Kb 'extended reach' broadband reached the village in 2001. The 'A' in ADSL stands for 'asymmetric', which is a clue to the current conversation.

Domestic cable TV still hasn't reached the village - they never built out the green survey marks (presumably satellite TV cut the ground from under their feet - but that's rarely appropriate for two-way broadband, because of the latency). I think cable TV is much more widespread in the USA, and is more likely to be symmetric.

I'm currently on FTTC - fibre to the cabinet - ADSL, which is still asymmetric, but I'm on the waiting list for fibre to the premises. We'll see how that is implemented when it arrives.
ID: 66782 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 66784 - Posted: 5 Dec 2022, 12:53:57 UTC - in response to Message 66782.  

Domestic cable TV still hasn't reached the village - they never built out the green survey marks (presumably satellite TV cut the ground from under their feet - but that's rarely appropriate for two-way broadband, because of the latency). I think cable TV is much more widespread in the USA, and is more likely to be symmetric.
Latency of Starlink [1] is pretty good, way better than the BT one I tried about 2002. It's reported to give 20-88ms. My fibre to the cabinet gives 17ms.

I'm currently on FTTC - fibre to the cabinet - ADSL, which is still asymmetric, but I'm on the waiting list for fibre to the premises. We'll see how that is implemented when it arrives.
Am I mixing you up with somebody else? I thought you were the guy in Oxford/Cambridge with a terrible connection?

[1] ROFL, I looked up Skynet by mistake and found the international government spy satellites watching every move we make. That really needs to be shot down. Oops, they probably read that.
ID: 66784 · Report as offensive
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,716,561
RAC: 8,355
Message 66785 - Posted: 5 Dec 2022, 14:39:30 UTC - in response to Message 66784.  
Last modified: 5 Dec 2022, 14:39:46 UTC

Am I mixing you up with somebody else?
Yes, that's Dave the moderator.
ID: 66785 · Report as offensive
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,716,561
RAC: 8,355
Message 66786 - Posted: 5 Dec 2022, 14:56:12 UTC - in response to Message 66784.  

Starlink uses a swarm of low earth orbit, fast-moving, reflective, satellites - much to the annoyance of visible-light astronomers.

Broadcast TV uses distant geostationary satellites, hence the higher latency. And you'd need a much more powerful transmitter to send a fast reply up there.

In 1993 you'd have been better off with a service like INMARSAT - if you could afford the price.
ID: 66786 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 66788 - Posted: 5 Dec 2022, 17:45:57 UTC - in response to Message 66786.  

In 1993 you'd have been better off with a service like INMARSAT - if you could afford the price.
In 1993 I didn't have an internet. I got it dial up at my parents' house in 1995, and dialup in my own house in 2000, and whatever BT Openworld used for satellite in 2002.
ID: 66788 · Report as offensive
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 66790 - Posted: 5 Dec 2022, 19:11:55 UTC - in response to Message 66781.  

They need them ASAP. So if some people can do them in 4 days and some in 4 weeks, they want them done in 4 days. You are not the centre of their universe, their work is. Why not leave your machine on 24/7?


I know, and if the deadlines drop far enough I can't return results, I'll run something else. I prefer to run these tasks as it's literally running off surplus energy most of the year that otherwise doesn't get generated, but I recognize that the project does set the stuff. I don't care if it's shorter, but I don't see a point in making it needlessly shorter.

Short answer as to why I can't leave the machines on 24/7 is that I don't have nearly enough battery to do that, nor (in some winter conditions like the heavy fog today) nearly enough panel area to do so. I do the bulk of the compute in a standalone off-grid office, using the surplus energy in the summer, but days like today I can't even keep up with my own loads on solar alone. Not running CPDN compute on generator, sorry. ;) Rather counterproductive. I do have some 24/7 compute in the house (grid tied) that runs in the winter, but that VM is quite empty now.

Is Starlink the Musk one?! I thought that was gonna be the fastest thing ever.


Yes. It's fine, and a big improvement over the rural WISP, most of the time. Right now it's returning 80/6. It varies wildly. It's probably iced up right now, which isn't helping things.

Starlink uses a swarm of low earth orbit, fast-moving, reflective, satellites - much to the annoyance of visible-light astronomers.


And vague panic of anyone who sees the sky being broken with one of the launches strung out in their boost phase, because that's just not in the slightest a "normal" thing to see in the sky. But, yes, I hear that even the low-reflectivity ones are a royal pain if you're doing sky photography. :(
ID: 66790 · Report as offensive
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 66822 - Posted: 8 Dec 2022, 20:02:05 UTC
Last modified: 8 Dec 2022, 20:02:27 UTC

Those new Mac tasks are... crashy.

Just lit up one of my Legacy 32-bit Intel VMs to grab some on an otherwise idle box, and more than half crashed immediately.

https://www.cpdn.org/result.php?resultid=22249709 crashed with "Model crashed: INITTIME: Atmosphere basis time mismatch"

Same for https://www.cpdn.org/result.php?resultid=22249652, https://www.cpdn.org/result.php?resultid=22249619, https://www.cpdn.org/result.php?resultid=22249546, https://www.cpdn.org/result.php?resultid=22249525, https://www.cpdn.org/result.php?resultid=22249524.

https://www.cpdn.org/result.php?resultid=22249505 crashed with "Model crashed: INANCLA: Error opening file tmp/pipe_dummy; Model crashed: ATM_DYN : INVALID THETA DETECTED."

Looks like three are running OK, though.
ID: 66822 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 66823 - Posted: 8 Dec 2022, 21:26:10 UTC - in response to Message 66822.  

https://www.cpdn.org/result.php?resultid=22249505 crashed with "Model crashed: INANCLA: Error opening file tmp/pipe_dummy; Model crashed: ATM_DYN : INVALID THETA DETECTED."


Those with this "error message" are actually OK. The wrapper and the executable task are both OK. It is just that the starting conditions for the particular instance (task) do not work as expected.
ID: 66823 · Report as offensive
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,880,738
RAC: 19,368
Message 66824 - Posted: 8 Dec 2022, 22:07:44 UTC

Have to wait for some LHC ATLAS tasks to finish in WSL2 on the 2nd PC, which can take some time, before I can reconfigure it to fire up the Mac VM. There aren't a lot of tasks but not many older Macs are around so maybe I'll have a chance at some.
ID: 66824 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 66827 - Posted: 9 Dec 2022, 8:19:35 UTC - in response to Message 66822.  

Those new Mac tasks are... crashy.
How dare you? Macs do not crash.
ID: 66827 · Report as offensive
xii5ku

Send message
Joined: 27 Mar 21
Posts: 79
Credit: 78,306,092
RAC: 230
Message 66855 - Posted: 10 Dec 2022, 19:31:06 UTC - in response to Message 66771.  
Last modified: 10 Dec 2022, 19:31:53 UTC

Glenn Carver wrote:
xii5ku wrote:
But alas, and as usual, the one thing which cannot be done at CPDN is to follow a plan.
And what plan exactly are you referring to? I've been attending regular meetings, and working closely with them for many years (they are a very small team as you may or may not know), so I know very well what's going on. There are multiple projects on the go and CPDN have to wait until the scientist/s is/are ready. That is often where the wait time is. I find that comment annoying and plainly inaccurate.
I meant that I cannot follow a plan which I make. (I described a particular plan of mine: Maximizing my December contribution by using time without CPU availability for uploads.) In the post which you responded to, I described in some detail what I had in mind, so there was _some_ context to infer _who_ I was claiming cannot follow _whose_ plans. I am not a native English speaker though and now take from your response that I sounded as if I claimed that CPDN didn't follow any plans. Sorry for causing this misunderstanding.

Glenn Carver wrote:
CPDN is following the needs of the science projects, thankfully not the needs of your computers. They survive on a shoestring budget from the science projects they are able to attract.
I know. It is prominently explained in the FAQ, which I have read, and recently re-read. So I was redundantly stating the obvious and the well documented. So in addition I am sorry for lowering signal to noise ratio here.
ID: 66855 · Report as offensive
xii5ku

Send message
Joined: 27 Mar 21
Posts: 79
Credit: 78,306,092
RAC: 230
Message 66856 - Posted: 10 Dec 2022, 20:07:53 UTC - in response to Message 66770.  
Last modified: 10 Dec 2022, 20:21:49 UTC

Mr. P Hucker wrote:
xii5ku wrote:
It's got a slow upstream link, but one which is still above average in the country where I live.
Mine is slow compared to everyone I know, I only get 7 Mbit up, but 32 (was 54, they claim the trunk line got too busy, probably they don't like what I download) Mbit down. Everyone else seems to have symmetrical connections.

Do you like in the UK? Your team is British.
I live in Germany. I mentioned in another thread that I have an 8 Mbit/s uplink = 84 GB/day. Current OpenIFS tasks produce 1.72 GB result files per task, therefore: 84 GB/day translate to 48.8 results/day which can be uploaded at most. On hardware which spends 16.5 hours per OpenIFS task, this translates to 33 simultaneously running tasks which would saturate that 8 Mbit/s link. Deduct a little overhead, and it's about 30 simultaneously running tasks which can be run without building an upload backlog on this link of mine. (Not considering downtimes of the connection or of the upload server.)

There are of course much faster Internet connections available to consumers in some areas of Germany. But in my area, the top offer is 25 Mbit/s upload, from what I could find. Even that wouldn't be enough to let even just two of my better computers run OpenIFS on 100% of their cores or during 100% of their time. A bit of a luxury problem. (Edit: In rural areas, it can be a lot worse, of course.)
ID: 66856 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 66857 - Posted: 10 Dec 2022, 20:12:56 UTC - in response to Message 66856.  
Last modified: 10 Dec 2022, 20:13:19 UTC

There are of course much faster Internet connections available to consumers in some areas of Germany. But in my area, the top offer is 25 Mbit/s upload, from what I could find. Even that wouldn't be enough to let even just two of my better computers run OpenIFS on 100% of their cores or during 100% of their time. A bit of a luxury problem.
I wonder if you could get two? Back in the days of modems, I got two phone lines (double the cost of course), and connected twice. You could even bond them with the right software.

There's also that satellite thing from Elon Musk, anyone can get that.

And depending where the nearest masts are, 4/5G.
ID: 66857 · Report as offensive
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,880,738
RAC: 19,368
Message 66858 - Posted: 11 Dec 2022, 4:57:23 UTC - in response to Message 66822.  

Those new Mac tasks are... crashy.

Yes, they are. Just fired up a Mojave VM and the entire initial batch of 7 crashed within a minute. We'll see how it goes with the next batch in an hour. From looking a bit, it seems like a very common reason for crashes is:
Model crashed: INITTIME: Atmosphere basis time mismatch
ID: 66858 · Report as offensive
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,880,738
RAC: 19,368
Message 66859 - Posted: 11 Dec 2022, 6:03:31 UTC - in response to Message 66858.  

More Mac crashes, this one with a new error I haven't seen before:
Model crashed: SETCONA: missing data as exponential in SOILB
ID: 66859 · Report as offensive
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,476,460
RAC: 15,681
Message 66860 - Posted: 11 Dec 2022, 9:51:10 UTC - in response to Message 66859.  

Like like they have not been set up correctly. I'll report back to the scientist. Thanks for the reports.
More Mac crashes, this one with a new error I haven't seen before:
Model crashed: SETCONA: missing data as exponential in SOILB
ID: 66860 · Report as offensive
bullschuck

Send message
Joined: 22 May 21
Posts: 39
Credit: 1,194,332
RAC: 3,964
Message 66862 - Posted: 11 Dec 2022, 20:33:05 UTC - in response to Message 66860.  

Anyone else having issues getting the results to upload? I'm just getting timeouts.

Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: Found bundle for host upload11.cpdn.org: 0x7fb24da212b0 [serially]
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#17] Info: Trying 192.171.139.103...
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#17] Info: TCP_NODELAY set
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: Hostname 'upload11.cpdn.org' was found in DNS cache
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: Trying 192.171.139.103...
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: TCP_NODELAY set
Sun Dec 11 14:30:31 2022 | climateprediction.net | [http] [ID#17] Info: connect to 192.171.139.103 port 80 failed: Operation timed out
Sun Dec 11 14:30:31 2022 | climateprediction.net | [http] [ID#17] Info: Failed to connect to upload11.cpdn.org port 80: Operation timed out

Thanks!
ID: 66862 · Report as offensive
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 66863 - Posted: 11 Dec 2022, 20:54:20 UTC - in response to Message 66862.  

Anyone else having issues getting the results to upload? I'm just getting timeouts.

Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: Found bundle for host upload11.cpdn.org: 0x7fb24da212b0 [serially]
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#17] Info: Trying 192.171.139.103...
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#17] Info: TCP_NODELAY set
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: Hostname 'upload11.cpdn.org' was found in DNS cache
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: Trying 192.171.139.103...
Sun Dec 11 14:29:16 2022 | climateprediction.net | [http] [ID#18] Info: TCP_NODELAY set
Sun Dec 11 14:30:31 2022 | climateprediction.net | [http] [ID#17] Info: connect to 192.171.139.103 port 80 failed: Operation timed out
Sun Dec 11 14:30:31 2022 | climateprediction.net | [http] [ID#17] Info: Failed to connect to upload11.cpdn.org port 80: Operation timed out

Thanks!
I'm not uploading anything just now, but I can't get to upload11.cpdn.org:80 in a web browser. "upload11.cpdn.org took too long to respond". I assume these servers would respond in a browser since they're http based.
ID: 66863 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 66864 - Posted: 11 Dec 2022, 20:56:41 UTC - in response to Message 66862.  

Anyone else having issues getting the results to upload? I'm just getting timeouts.


Oh! Yes!

The last I checked, I had about 750 files waiting to upload.
I hope they fix it before my deadlines expire.
They will probably fix the problem tomorrow (Monday).
ID: 66864 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 66865 - Posted: 11 Dec 2022, 21:26:31 UTC

Anyone else having issues getting the results to upload? I'm just getting timeouts.
See this post in OpenIFS discussion thread.
ID: 66865 · Report as offensive
Previous · 1 . . . 14 · 15 · 16 · 17 · 18 · 19 · 20 . . . 42 · Next

Message boards : Number crunching : New work discussion - 2

©2024 cpdn.org