climateprediction.net (CPDN) home page
Thread 'Upload pending'

Thread 'Upload pending'

Message boards : Number crunching : Upload pending
Message board moderation

To post messages, you must log in.

AuthorMessage
old_user191209

Send message
Joined: 26 Jun 06
Posts: 7
Credit: 158,935
RAC: 0
Message 44692 - Posted: 13 Aug 2012, 7:02:10 UTC

Hi! Running your files have created a bunch of hadam3p_eu_xxxx...zip files. These apparently won't upload. They've been sitting in "Transfers" for days, a few saying "100 %" and "Upload: pending (project backoff.." most saying "0 %" Upload: pending ... ". "Retry now" has no apparent effect.

Same in "Tasks" finished runs just sitting there saying "100 %" and "Uploading" but nothing is happening.

Running BOINC 7.0.28 (64x) on win 64.

Server down? Line down?
ID: 44692 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 44693 - Posted: 13 Aug 2012, 7:36:47 UTC - in response to Message 44692.  

Server is full.
We're waiting for the working week to start in the UK. :)

Backups: Here
ID: 44693 · Report as offensive     Reply Quote
old_user191209

Send message
Joined: 26 Jun 06
Posts: 7
Credit: 158,935
RAC: 0
Message 44695 - Posted: 13 Aug 2012, 10:25:42 UTC - in response to Message 44693.  

Hi!

Errrmmm... server full...? That would do it, I guess. How... Why don't... oh, never mind. :)

We're too productive for them!

Thanks!
ID: 44695 · Report as offensive     Reply Quote
old_user668187

Send message
Joined: 18 Dec 11
Posts: 4
Credit: 64,484
RAC: 0
Message 44696 - Posted: 13 Aug 2012, 17:20:47 UTC

I've got the same problem (4 zip files waiting at the moment). Any idea how long the server will be down for?
ID: 44696 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 44698 - Posted: 13 Aug 2012, 20:54:13 UTC

See the News and Announcements in Number crunching.


Backups: Here
ID: 44698 · Report as offensive     Reply Quote
old_user191209

Send message
Joined: 26 Jun 06
Posts: 7
Credit: 158,935
RAC: 0
Message 44699 - Posted: 14 Aug 2012, 9:49:14 UTC - in response to Message 44698.  

Even moving terabytes shouldn't take days...
ID: 44699 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 44700 - Posted: 14 Aug 2012, 11:40:50 UTC - in response to Message 44699.  

Even moving terabytes shouldn't take days...


On one occasion recently when stuff had to be moved it was a couple of Terabytes.

The speed all depends on the speed of the network. if 100MB/second=6GB/min=360GB/hour=8.64GB/day. My guess is that 100MB/s is optimistic even if the data isn't being verified. Add on verification and I don't know how much time it adds.
ID: 44700 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 44701 - Posted: 14 Aug 2012, 14:48:52 UTC

Wait and see.

And this thread should have been in Number Crunching, not the Science section, as it's nothing to do with the science of climate modelling.

Backups: Here
ID: 44701 · Report as offensive     Reply Quote
old_user668187

Send message
Joined: 18 Dec 11
Posts: 4
Credit: 64,484
RAC: 0
Message 44707 - Posted: 14 Aug 2012, 17:15:35 UTC - in response to Message 44701.  

Sorry Les. :-)
ID: 44707 · Report as offensive     Reply Quote
old_user191209

Send message
Joined: 26 Jun 06
Posts: 7
Credit: 158,935
RAC: 0
Message 44711 - Posted: 15 Aug 2012, 12:48:28 UTC - in response to Message 44701.  

My bad. Sorry. Haven't posted here before and was unclear on the areas. Will try to do right in the future :)
ID: 44711 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 44714 - Posted: 15 Aug 2012, 21:11:50 UTC

It turns out to be terabytes, not gigabytes. And it's live data, to which researchers have access for running data extraction queries. So care has to be taken in moving it.

Because some of the download files are also on that server, they've had to re-enable access to it, and not all of the terabytes of upload data have been moved.

Expect things to be slow, and for the server to quickly re-fill.


Backups: Here
ID: 44714 · Report as offensive     Reply Quote
old_user191209

Send message
Joined: 26 Jun 06
Posts: 7
Credit: 158,935
RAC: 0
Message 44721 - Posted: 16 Aug 2012, 9:03:05 UTC - in response to Message 44714.  

All this talk of cumbersome moving of files makes one ponder the marvels of server area networks and clustered solutions.
ID: 44721 · Report as offensive     Reply Quote
ProfileastroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 44722 - Posted: 16 Aug 2012, 15:49:49 UTC - in response to Message 44721.  

All this talk of cumbersome moving of files makes one ponder the marvels of server area networks and clustered solutions.

All it requires is a huge budget, for hardware, for staff ...

"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 44722 · Report as offensive     Reply Quote
old_user191209

Send message
Joined: 26 Jun 06
Posts: 7
Credit: 158,935
RAC: 0
Message 44726 - Posted: 16 Aug 2012, 17:48:20 UTC

Well yes. There is always that.

The opposite strategy would be shutting down the server, plug a cheap drive into a physical port, copy the files, have someone physically run it over to wherever it should go and call and say whether it went allright. Then delete the originals. In my darker moments I wistfully wonder which is really the most efficient and safe.

If against all odds it hadn't already occurred to you, I'm neither complaining nor being very serious. Not my intent to po anyone. Please feel free to ignore, terminate, delete or move this thread as you see fit. :)

Have a good one. Time for bed on my end.
ID: 44726 · Report as offensive     Reply Quote
old_user191209

Send message
Joined: 26 Jun 06
Posts: 7
Credit: 158,935
RAC: 0
Message 44730 - Posted: 17 Aug 2012, 13:57:47 UTC

More than a GBs worth of upload files have magically disappeared from my transfer tab in BOINC. All good things to those who wait :)
ID: 44730 · Report as offensive     Reply Quote
ggrinton

Send message
Joined: 24 Jan 06
Posts: 5
Credit: 435,756
RAC: 0
Message 44917 - Posted: 28 Sep 2012, 10:07:57 UTC - in response to Message 44730.  

Do we have a problem again? I have had a bunch of files waiting for nearly two weeks now.
ID: 44917 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 44923 - Posted: 28 Sep 2012, 13:12:45 UTC - in response to Message 44917.  

Do we have a problem again? I have had a bunch of files waiting for nearly two weeks now.



Check the news section at the top of number crunching.
ID: 44923 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 44938 - Posted: 30 Sep 2012, 22:40:30 UTC

This thread has been moved here from the climateprediction.net Science section.

ID: 44938 · Report as offensive     Reply Quote

Message boards : Number crunching : Upload pending

©2024 cpdn.org