climateprediction.net (CPDN) home page
Thread 'Upload problems'

Thread 'Upload problems'

Message boards : Number crunching : Upload problems
Message board moderation

To post messages, you must log in.

AuthorMessage
MVeiga

Send message
Joined: 14 Oct 07
Posts: 6
Credit: 1,567,569
RAC: 2,974
Message 49752 - Posted: 18 Aug 2014, 9:33:30 UTC

Hello everyone.
I don't know if is a problem of BOINC but im not able to upload complete tasks.
I have done some tasks but they don't upload.

Can anyone tells me if there is a problem???
ID: 49752 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,008,987
RAC: 21,524
Message 49753 - Posted: 18 Aug 2014, 10:10:40 UTC - in response to Message 49752.  

Yes, one of the servers is down shown in red on the server status page fifth up from the bottom in the links on the left. There is an extensive thread on this subject in the number crunching section. relevant server is actually back to yellow now still not accepting zip files. Worth keeping an eye on the thread I mentioned and trying uploads again when it is mentioned in there that it is working. When it comes back on line there are likely to be problems as the server will get hammered by everyone trying to upload their zips at once which will result in temporary http errors for some.
ID: 49753 · Report as offensive     Reply Quote
MVeiga

Send message
Joined: 14 Oct 07
Posts: 6
Credit: 1,567,569
RAC: 2,974
Message 49755 - Posted: 18 Aug 2014, 11:26:42 UTC

Thanks for the info.
I will keep the uploads on hold.
This problems at least for me has already 2 weeks
ID: 49755 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,008,987
RAC: 21,524
Message 49756 - Posted: 18 Aug 2014, 11:36:00 UTC - in response to Message 49755.  

The server was back on then it went off again. Currently one person has posted that zips staring to go but the transient errors are likely to be around for an hour or so till the backlog clears.
ID: 49756 · Report as offensive     Reply Quote
MVeiga

Send message
Joined: 14 Oct 07
Posts: 6
Credit: 1,567,569
RAC: 2,974
Message 49764 - Posted: 18 Aug 2014, 16:46:34 UTC - in response to Message 49756.  

Nop, mine are at 0%
Tried to force manually but keeps doing the same
ID: 49764 · Report as offensive     Reply Quote
Profileritterm
Avatar

Send message
Joined: 29 May 08
Posts: 128
Credit: 6,289,876
RAC: 0
Message 49765 - Posted: 18 Aug 2014, 17:28:44 UTC - in response to Message 49764.  
Last modified: 18 Aug 2014, 17:29:09 UTC

Miguel Veiga wrote:
Tried to force manually but keeps doing the same...

Uploads are slowly getting through as the server is able to take them. There is almost certainly a very large back up of uploads and we just need to be patient. The files will eventually get uploaded and manually "forcing" uploads may even make the problem worse.
ID: 49765 · Report as offensive     Reply Quote
wateroakley

Send message
Joined: 6 Aug 04
Posts: 195
Credit: 28,336,682
RAC: 10,407
Message 49766 - Posted: 18 Aug 2014, 19:09:23 UTC - in response to Message 49765.  

I can confirm that the 1GB backlog of upload files has cleared off the PC here.

Unfortunately the laptop HDD that objected to the CPDN 10 year anniversary is still being a pain and up-chucked the latest set of work. A replacement HDD is being imaged and we'll see how that behaves.
ID: 49766 · Report as offensive     Reply Quote
MVeiga

Send message
Joined: 14 Oct 07
Posts: 6
Credit: 1,567,569
RAC: 2,974
Message 49769 - Posted: 18 Aug 2014, 21:26:55 UTC - in response to Message 49765.  

Miguel Veiga wrote:
Tried to force manually but keeps doing the same...

Uploads are slowly getting through as the server is able to take them. There is almost certainly a very large back up of uploads and we just need to be patient. The files will eventually get uploaded and manually "forcing" uploads may even make the problem worse.



I will not force manually anymore.
The problem is that i have at least 6 files os 35mb each to upload.
Well ive now and 4 of them have been uploaded. The other 2 are stopped but i will just wait
ID: 49769 · Report as offensive     Reply Quote
Niall

Send message
Joined: 18 Dec 13
Posts: 62
Credit: 1,078,935
RAC: 0
Message 49770 - Posted: 18 Aug 2014, 23:17:00 UTC - in response to Message 49769.  

Miguel

Patience is sometimes key around here. If you are worried about a couple of hundred Mb on your hard drive, you probably need to see this in the context of the sheer amount of work getting done here (and how much of your hard drive it's taking up already).

A CPDN model, as has been discussed elsewhere around here, is in the million lines of Fortran range, and will be taking up Gigabytes of hard drive space. Between us we're at the tail end of crunching 80,000 hadam3p_eu work units. Each of those (assuming they don't crash/get aborted by the user, in which case they will be reassigned to another user) will upload 13 zip files, at around 34Mb each. A quick back of an envelope calculation tells me that means the servers will be getting back in the range of (13x34x80,000)/1024= 34531 Gigabytes of data, and change, just from those hadam3p_eu work units.

Then there are the Pacific North-West (not sure how many of those are being worked on at the moment, but it looks like it's into 5 figures) and ANZ attribution projects and two different coupled model projects (over 11,000 work units ready to be sent plus more on computers just from the geoengineering programme).

What were are probably talking about is a team of ace programmers and techs filling a terabyte drive on a more-or-less daily basis, while keeping the whole lot running.

To my mind, the best thing to do is to report glitches and wait patiently while they fix them. Servers are going to overload and crash, glitches are going to happen, and the credit script might not run reliably.

Under the circumstances, it's a testament to the competence and hard work of these people that we don't have more problems.
ID: 49770 · Report as offensive     Reply Quote
MVeiga

Send message
Joined: 14 Oct 07
Posts: 6
Credit: 1,567,569
RAC: 2,974
Message 49771 - Posted: 19 Aug 2014, 0:51:04 UTC - in response to Message 49770.  

Thanks for the explanation Nial.
I was not worried at all about the disk in space that the units will take.
I fully understand what you mean, to much network activity.
I manage some servers in my work with some TB, so i know how a database or some incremental backup can grow in one day.

My only worry was to know if the problem was getting fixed.
My answer is given.

Thanks for all the info and good luck in receiveng all that huge amount of data.
Keep up the god work
ID: 49771 · Report as offensive     Reply Quote
Niall

Send message
Joined: 18 Dec 13
Posts: 62
Credit: 1,078,935
RAC: 0
Message 49772 - Posted: 19 Aug 2014, 1:03:40 UTC - in response to Message 49771.  

To clarify: this is not me doing this work. It is me having an appreciation of the people who do.
ID: 49772 · Report as offensive     Reply Quote
MVeiga

Send message
Joined: 14 Oct 07
Posts: 6
Credit: 1,567,569
RAC: 2,974
Message 49773 - Posted: 19 Aug 2014, 10:10:32 UTC - in response to Message 49772.  

Anyway it should be more or less the way youve told
ID: 49773 · Report as offensive     Reply Quote

Message boards : Number crunching : Upload problems

©2024 cpdn.org