the model is capable of producing data faster than it could be uploaded over an average broadband connection in a reasonable time.

Glenn, could you please explain that a bit more. Is this going to be something like LHC ATLAS and Theory subprojects where you need constant internet access, because the data is going to be going back and forth? It kind of sounds like you're going to need results almost real-time? I'm having a hard time visualizing how it's going to work.

If the work units are of the same size as the ones that have been through testing, the final uploads have been up to 1.5GB. If for example, tasks increase in size by a factor of four so that they are still completed in the same time running on more cores, that could see uploads of over 6GB the tasks don't send up data continuously while they run.

If the work units are of the same size as the ones that have been through testing, the final uploads have been up to 1.5GB. If for example, tasks increase in size by a factor of four so that they are still completed in the same time running on more cores, that could see uploads of over 6GB the tasks don't send up data continuously while they run.

How long do the work units run? I don't recall seeing that.

My cable modem is rated at 15 Mbps upload, but in practice I see 20 Mbps. That should upload 6GB (48Gb) in 40 minutes if the other infrastructure supports it.
But that is probably the real limitation, not the local speed. I see very good speeds across the Atlantic, but never more than 10 Mbps.
So in practice, it would take more like 1 1/2 hours. And with uploads from my other machines, it could be longer than that.

How long do the work units run? I don't recall seeing that.

The last ones that had the big uploads took about 10 hours on my Ryzen7.

I can get 1 Gigabit up and down if I want to pay for it. My "NIC" can do it already:

Network specifications
Features Specifications
Integrated Intel i219 Gigabit Ethernet controllers with Intel Remote Wake UP, PXE and Jumbo frames support

I think my Verizon-supplied router will go up to 100 MBits/sec.
If I upgrade to 940 MBits/sec, I think the rates are about double what they are now. I cannot tell. Their stupid web site says FiOS is not available in my neighborhood, but I have had it since about 2004.

---

Where are you in the UK that you can't get semi-fibre? They've been rolling out full fibre to the home for years, I thought semi-fibre was long since finished.

I could get fibre to the box by swapping to Virgin but I happen to not like Branson's business practices so will not change from the BT infrastructure. On Faceache they keep advertising that they are upgrading in Cambridge but whenever I check, it suggests I check again in three months time.

Wow, I'm surprised in Cambridge (a large city) BT haven't given everyone fibre. Perhaps it's because Virgin have snuck in. A friend in Hull had big problems with some little local telephone company taking over. I have principles too, but I'd sell my sister to get fast internet.

Wow, I'm surprised in Cambridge (a large city) BT haven't given everyone fibre. Perhaps it's because Virgin have snuck in. A friend in Hull had big problems with some little local telephone company taking over. I have principles too, but I'd sell my sister to get fast internet.

I was born in Cambridge, and I spent four years of my adult life there. It's a small place, I doubt even if legally a city. More like a large town - and an important one.

And it has a large rural hinterland. Dave is free to choose how much he wants to disclose of his distance from the centre: my four years were spent "within three miles of Great St. Mary's". And there wasn't any broadband.

Wow, I'm surprised in Cambridge (a large city) BT haven't given everyone fibre. Perhaps it's because Virgin have snuck in. A friend in Hull had big problems with some little local telephone company taking over. I have principles too, but I'd sell my sister to get fast internet.

I was born in Cambridge, and I spent four years of my adult life there. It's a small place, I doubt even if legally a city. More like a large town - and an important one.

And it has a large rural hinterland. Dave is free to choose how much he wants to disclose of his distance from the centre: my four years were spent "within three miles of Great St. Mary's". And there wasn't any broadband.

I assumed it must be big because of the famous university. A quick check on a map shows the diameter of various cities to be:

Cambridge 6km across
Oxford 8km across
Clackmannan (where I live, called a town) 1.25km across
Alloa (where I'd go for most shopping, they have 6 supermarkets, called a town) 3km across
Stirling (if I need a B&Q etc, not sure if that's a town or city) 4.5km across
Dundee (where I was born, a city) 6km across
Edinburgh (obviously a city) 11km across
Glasgow (obviously a city) 12km across

So the same size as Dundee, everyone calls that a city. Most pubs per square mile in the UK! And most police cars set on fire.... not that I ever did that, although I will admit to puncturing some tyres. Long time ago, statute of limitations and all that :-P

Huh. I guess I'd better route those units out Starlink, because my terrestrial WISP is only good for 2-3Mbit up, and if that's saturated, the rest of the internet doesn't exactly work well...

Huh. I guess I'd better route those units out Starlink, because my terrestrial WISP is only good for 2-3Mbit up, and if that's saturated, the rest of the internet doesn't exactly work well...

Does Linux have a problem sharing the internet between programs? Because in Windows I've never noticed a problem. Boinc cannot "saturate" the uplink, it's shared evenly between whatever is trying to use it. If you do have a problem, just tell Boinc to only use 50% of your connection.

Good idea. Just make Boinc sit in the naughty corner (I wish someone else also) and talk to it.

Good idea. Just make Boinc sit in the naughty corner (I wish someone else also) and talk to it.

And just who do you wish to give a jolly good spanking to?

Peter. "City" is a specific legal status - ask Southend

On 18 October 2021, it was announced that Southend would be granted city status, as a memorial to ...

Southend was granted city status by letters patent dated 26 January 2022.

And it has a large rural hinterland. Dave is free to choose how much he wants to disclose of his distance from the centre: my four years were spent "within three miles of Great St. Mary's". And there wasn't any broadband.

Only two miles from the station via the misguided busway. (If I cycle rather than getting the misguided bus which detours around Addenbrooke's hospital. Before they decided to build on the fields we were on the Southern edge of the city. From Wikipedia,

The first town charters were granted in the 12th century, although modern city status was not officially conferred until 1951.

And the station is a long way from the town centre. I remember that schlep!

And the station is a long way from the town centre. I remember that schlep!

But Microsoft, Amazon and some other tech giant I forget which all have massive operations in Station Road. The MS UK headquarters is within a stones throw of the station and that means a stone's throw for me, not for someone who can throw a cricket ball from the boundary to the wicket!

the model is capable of producing data faster than it could be uploaded over an average broadband connection in a reasonable time.

Glenn, could you please explain that a bit more. Is this going to be something like LHC ATLAS and Theory subprojects where you need constant internet access, because the data is going to be going back and forth? It kind of sounds like you're going to need results almost real-time? I'm having a hard time visualizing how it's going to work.

Sorry for the confusion on the terminology. I'm used to an operational forecasting environment where the 'raw' model output is processed in real-time to produce the forecast products and there's Tb of data. We're not doing that here. For OpenIFS@home, the CPDN team are well aware of networking contraints. The initial download is the only download for the task, once the model starts running there's a trickle of data as it produces output to avoid a big upload at the end. The total output size is also limited to avoid uploads that might prove too long on slower broadband speeds. I'll stop talking about this now. Best to see what it does in action.

If the work units are of the same size as the ones that have been through testing, the final uploads have been up to 1.5GB. If for example, tasks increase in size by a factor of four so that they are still completed in the same time running on more cores, that could see uploads of over 6GB the tasks don't send up data continuously while they run.

I can add more detail. There will be a variety of workunits, some for shorter forecasts, say 10 days, others for seasonal (several months). The model output IS continuously uploaded as soon as output is ready - previous tests didn't have this in place. Workunits at higher resolutions will have the potential to create more data but the CPDN team do restrict model output to what they regard as reasonable limits - if needed, the same workunit could be run but with different output variables.

Bottom line, if the model upload starts causing a problem, let the CPDN team know and we can adjust.

Peter. "City" is a specific legal status - ask Southend

On 18 October 2021, it was announced that Southend would be granted city status, as a memorial to ...

Southend was granted city status by letters patent dated 26 January 2022.

Oh. i thought it was all done by population size. That's what i was taght in geography class at school. Hamlet, village, town, city. The changeover points were something like 100, 1000, 10000.

If the work units are of the same size as the ones that have been through testing, the final uploads have been up to 1.5GB. If for example, tasks increase in size by a factor of four so that they are still completed in the same time running on more cores, that could see uploads of over 6GB the tasks don't send up data continuously while they run.

I can add more detail. There will be a variety of workunits, some for shorter forecasts, say 10 days, others for seasonal (several months). The model output IS continuously uploaded as soon as output is ready - previous tests didn't have this in place. Workunits at higher resolutions will have the potential to create more data but the CPDN team do restrict model output to what they regard as reasonable limits - if needed, the same workunit could be run but with different output variables.

Bottom line, if the model upload starts causing a problem, let the CPDN team know and we can adjust.

It seems a pity to limit things due to the internet speed of some people. I wonder if Boinc has knowledge of your internet speed, and could send different types of task to different people? If not, there could always be a tickbox in preferences to accept enormous tasks.

Or quite commonly by the presence of a cathedral.