climateprediction.net (CPDN) home page
Thread 'ANOTHER UPLOAD PROBLEM'

Thread 'ANOTHER UPLOAD PROBLEM'

Message boards : Number crunching : ANOTHER UPLOAD PROBLEM
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 22 · 23 · 24 · 25 · 26 · 27 · 28 . . . 33 · Next

AuthorMessage
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54138 - Posted: 17 May 2016, 20:32:38 UTC - in response to Message 54106.  

Further to my post referenced at the top right of this one, batches 396 & 397 are 3 month models, and go to you know where. Shhhh.

They're taking 10 hours on my Ivy Bridge.

ID: 54138 · Report as offensive     Reply Quote
Kevin

Send message
Joined: 5 Jul 09
Posts: 63
Credit: 6,091,274
RAC: 0
Message 54150 - Posted: 20 May 2016, 7:47:52 UTC

Its doing it again:-(

4 afr50 zips in my upload queue.


Kevin
ID: 54150 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54151 - Posted: 20 May 2016, 8:01:55 UTC - in response to Message 54150.  

Yes, I've got a miscellaneous collection of different batches going there, and another moderator has a bunch of PNWs.

Andy was emailed several hours ago, and as it should be about 9am there now, hopefully it won't be much longer. Unless they're going to have a long discussion and think about it.

ID: 54151 · Report as offensive     Reply Quote
MartinNZ

Send message
Joined: 22 Mar 06
Posts: 144
Credit: 24,695,428
RAC: 0
Message 54159 - Posted: 21 May 2016, 23:06:04 UTC

Oh look, it's the weekend again :-) Who needs a calendar?

21/05/2016 14:21:28 | climateprediction.net | [error] Error reported by file upload server: can't open file /storage/incoming/uploader/hadam3p_pnw_j7lf_201412_12_405_010579265_0_2.zip: No such file or directory


Out of interest, are these issues automatically flagged on the servers, or do the IT guys rely on feedback?
ID: 54159 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54161 - Posted: 22 May 2016, 2:33:16 UTC - in response to Message 54159.  

There should be an errors list somewhere for all of the computers, but the IT people need to remember to look at it soon after getting to work, rather than get distracted by people telling them there's a meeting, etc.

Going by the messages that we're getting, the upload server is OK, it's the NAS getting disconnected/full/huffy. :) (Upload reaches 100%, and then errors out.)

Andy was told way back about this latest problem.

ID: 54161 · Report as offensive     Reply Quote
MartinNZ

Send message
Joined: 22 Mar 06
Posts: 144
Credit: 24,695,428
RAC: 0
Message 54163 - Posted: 22 May 2016, 4:25:45 UTC - in response to Message 54161.  

Thanks Les,

I had noticed that the servers were OK on the status page & this seems to the case 95% of the time with upload errors.

From your reply I take it we keep dropping a line here with issues. We know the IT staff will get it back up and running asap.
ID: 54163 · Report as offensive     Reply Quote
ProfileVicki

Send message
Joined: 28 Nov 15
Posts: 50
Credit: 4,099,809
RAC: 0
Message 54167 - Posted: 22 May 2016, 10:35:52 UTC
Last modified: 22 May 2016, 10:54:48 UTC

Hi all
"it seams a shame the walrus said.... " lines from a well known piece of prose
My shame is battles on 2 fronts; windows 10 and 3 tasks unable to upload:

Hadam39_eu_iaao batch 396 trickle #1
Hadam39_eu_i4s2 batch 397 trickle #1
Hadam39_eu_i9w4 batch 396 trickle #1

For those of you with a sense of humor, this picture is rather apt
https://onedrive.live.com/redir?resid=DA0E938BD05FB3F3!614&authkey=!ANroR2Wd4rnFZIk&v=3&ithint=photo%2cjpg

Looks for the any key on the malfunctioning server and flicks a coin, heads cold reboot, tails simply boot the server until it works, that method worked in star trek, Blakes 7 and other sci fis

Having posted the error, and vented the steam, I now return to Lady Mode.

Good Night from here where its a bit after 10 pm

Vicki
ID: 54167 · Report as offensive     Reply Quote
rob

Send message
Joined: 5 Jun 09
Posts: 97
Credit: 3,736,855
RAC: 4,073
Message 54168 - Posted: 22 May 2016, 12:54:23 UTC

Today is a somewhat sad day in the land of the CPDN servers - the work (tricle=ups?) gets sent back, or at least the transmit report says 100%, then stalls. Typical message is:

22/05/2016 13:50:31 | climateprediction.net | [error] Error reported by file upload server: can't open file /storage/incoming/uploader/hadam3p_pnw_j3aj_201412_12_405_010573689_0_10.zip: No such file or directory
22/05/2016 13:50:31 | climateprediction.net | Temporarily failed upload of hadam3p_pnw_j3aj_201412_12_405_010573689_0_10.zip: transient upload error
22/05/2016 13:50:31 | climateprediction.net | Backing off 00:02:29 on upload of hadam3p_pnw_j3aj_201412_12_405_010573689_0_10.zip



Oh well, the server might get better without a kick in the nodes, but I suspect it is going to need the well directed application of a size-ten reboot....
ID: 54168 · Report as offensive     Reply Quote
jrapdx

Send message
Joined: 4 Jul 15
Posts: 63
Credit: 3,223,760
RAC: 0
Message 54175 - Posted: 23 May 2016, 5:51:23 UTC - in response to Message 54168.  

Getting the same error messages here, at least on this machine. However my other computers aren't reporting errors probably because they're running different WUs.

AFAICT the current upload failure is affecting hadam3p_eu/pnw, but not wah2_eu/pnw (which uploaded OK). IIRC it's been the other way around before when it was the wah2 WUs that were not getting through. I guess it depends on exactly which server(s) are malfunctioning.

Lately the project seems to be having a hard time keeping all its plates spinning simultaneously. Let's hope these problems are resolved soon!
ID: 54175 · Report as offensive     Reply Quote
ProfileVicki

Send message
Joined: 28 Nov 15
Posts: 50
Credit: 4,099,809
RAC: 0
Message 54176 - Posted: 23 May 2016, 9:53:37 UTC - in response to Message 54175.  

Hi all
without wanting to jinx anything, i am happy to say the stuck tasks have just sucessfully uploaded (Hadam3p_eu ) along with some wah2_pnw25 trickles.

Thank you to all who had a hand in sorting the issue(s)

Good night

Vicki (gmt +12)

ID: 54176 · Report as offensive     Reply Quote
Kevin

Send message
Joined: 5 Jul 09
Posts: 63
Credit: 6,091,274
RAC: 0
Message 54177 - Posted: 23 May 2016, 14:51:12 UTC

that server needs another prod

afr50 getting stuck again.


Kevin
ID: 54177 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 54179 - Posted: 23 May 2016, 20:31:53 UTC

Looks like it cleared. Two of my PCs uploaded a lot of files and reported completed models.

ID: 54179 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54180 - Posted: 23 May 2016, 21:03:46 UTC

It looks as though not even Upload2 itself is working from either of my 2 clogged up machines.

ID: 54180 · Report as offensive     Reply Quote
Legoman

Send message
Joined: 26 Aug 04
Posts: 1
Credit: 2,448,843
RAC: 710
Message 54182 - Posted: 24 May 2016, 5:27:06 UTC

Upload2 definitely isn't working. I've 25 files stacked up in the transfer queue and they aren't progressing beyond 0%.
ID: 54182 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54188 - Posted: 24 May 2016, 18:41:32 UTC

I've been talking to Andy, and it's good news / bad news.
Good: the old server is getting replaced.
Bad: this is going to take several days at least.

Ommmmmm


ID: 54188 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,000,748
RAC: 14,638
Message 54192 - Posted: 24 May 2016, 22:05:26 UTC - in response to Message 54188.  

Progress I suppose.
ID: 54192 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,026,382
RAC: 20,431
Message 54196 - Posted: 25 May 2016, 18:02:23 UTC

From Andy,

"The switch-over has now been made. The link 'upload2.cpdn.org' is now pointing to a new upload server."

I don't have any of the right sort of tasks to assess whether it is working or not.
ID: 54196 · Report as offensive     Reply Quote
ProfileastroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 54198 - Posted: 25 May 2016, 19:08:21 UTC - in response to Message 54196.  

From Andy,

"The switch-over has now been made. The link 'upload2.cpdn.org' is now pointing to a new upload server."

I don't have any of the right sort of tasks to assess whether it is working or not.


One of my machines uploaded ~30 files, so far -- close to 600 (overall) to go, so it will be a l-o-n-g time via DSL.

Even better news, they are all going up smoothly -- no hiccups due to server overload.

"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 54198 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54200 - Posted: 25 May 2016, 21:03:50 UTC

Mine are uploading OK too.
They're mostly just under 13 Megs each on this machine, with the occasional 30+ Meg re-start file, so it's not taking long for each one to disappear.

ID: 54200 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54202 - Posted: 25 May 2016, 22:52:49 UTC

Possibly more important - as soon as an upload reaches 100% it disappears immediately, which means that the upload server is passing the data to the NAS straight away.

ID: 54202 · Report as offensive     Reply Quote
Previous · 1 . . . 22 · 23 · 24 · 25 · 26 · 27 · 28 . . . 33 · Next

Message boards : Number crunching : ANOTHER UPLOAD PROBLEM

©2024 cpdn.org