climateprediction.net (CPDN) home page
Thread 'NZ25 file upload server problems?'

Thread 'NZ25 file upload server problems?'

Message boards : Number crunching : NZ25 file upload server problems?
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65808 - Posted: 11 Aug 2022, 23:42:55 UTC

Anyone else getting file upload errors on these nz25 tasks? Mine uploads to 100% but then gives an error:

8/11/2022 5:52:48 PM | climateprediction.net | Started upload of wah2_nz25_a141_199505_25_936_012151203_0_r55943290_1.zip
8/11/2022 5:58:10 PM | climateprediction.net | [checkpoint] result wah2_nz25_a141_199505_25_936_012151203_0 checkpointed
8/11/2022 6:06:07 PM | climateprediction.net | [checkpoint] result wah2_nz25_a141_199505_25_936_012151203_0 checkpointed
8/11/2022 6:08:26 PM | climateprediction.net | [error] Error reported by file upload server: EOF on socket read : asked for 262144, got 150376
8/11/2022 6:08:26 PM | climateprediction.net | Temporarily failed upload of wah2_nz25_a141_199505_25_936_012151203_0_r55943290_1.zip: transient upload error
8/11/2022 6:08:26 PM | climateprediction.net | Backing off 01:12:41 on upload of wah2_nz25_a141_199505_25_936_012151203_0_r55943290_1.zip

This is reminiscent of previous problems with the ANZ model uploads that go to Tasmania/New Zealand servers. If others chime in with a problem, I'll notify Andy and Suzanne Rosier about it so someone can kick the server.
ID: 65808 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65809 - Posted: 11 Aug 2022, 23:44:20 UTC - in response to Message 65808.  

Hmmm. On the 3rd retry, it finally went up so either they fixed it, or the problem is intermittent.
ID: 65809 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,835,158
RAC: 19,825
Message 65811 - Posted: 12 Aug 2022, 6:53:36 UTC

I'm having issues too but it does seem that eventually things upload as there are already 2 trickles uploaded, the third one is currently having upload issues though..
ID: 65811 · Report as offensive     Reply Quote
Albert H.

Send message
Joined: 18 Feb 06
Posts: 73
Credit: 61,612,794
RAC: 47,770
Message 65813 - Posted: 12 Aug 2022, 8:01:23 UTC - in response to Message 65811.  

I have suspended network activity for the moment. Wait and see.
ID: 65813 · Report as offensive     Reply Quote
ProfileThyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 65814 - Posted: 12 Aug 2022, 8:36:11 UTC - in response to Message 65808.  

My task's third upload has had 7 failed attempts, the first starting at 0409 UTC, the last 20 minutes ago.
"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 65814 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,014,785
RAC: 20,946
Message 65815 - Posted: 12 Aug 2022, 9:50:54 UTC - in response to Message 65814.  

Also getting problems on my two testing site tasks from this project. Have left a message with extracts from log. If action in NZ needed, it may not get looked at for a few hours yet!
ID: 65815 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,014,785
RAC: 20,946
Message 65816 - Posted: 12 Aug 2022, 11:01:51 UTC - in response to Message 65815.  
Last modified: 12 Aug 2022, 11:15:13 UTC

My uploads have started to upload again but still waiting to see if they finish. No idea if anyone has kicked server or not.

Getting stuck at 100%.
ID: 65816 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,014,785
RAC: 20,946
Message 65817 - Posted: 12 Aug 2022, 20:11:39 UTC

Andy says the server appears to be up and he thinks it may be getting flooded with the number of machines trying to send zips. I will try again in the morning to see if anything has changed.
ID: 65817 · Report as offensive     Reply Quote
rob

Send message
Joined: 5 Jun 09
Posts: 97
Credit: 3,736,855
RAC: 4,073
Message 65818 - Posted: 12 Aug 2022, 21:19:07 UTC

The arthritic snail that is uploads slides slowly up the sandpaper.....
It's very patchy, sometimes a zip will go up up smoothly, but most of the time the snail is being very careful, if not stopped and soothing its painful foot.
ID: 65818 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,014,785
RAC: 20,946
Message 65821 - Posted: 13 Aug 2022, 6:13:17 UTC

I turned network activity off overnight and this morning, no data going through rather than getting to 100% and then giving the end of file error. Have turned it off again.
ID: 65821 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,705,793
RAC: 9,655
Message 65822 - Posted: 13 Aug 2022, 7:54:59 UTC - in response to Message 65821.  

Has anyone used a file manager to look at the actual output data file on their hard disk (it'll be in the CPDN project folder in their BOINC data tree). Does the byte count match the "asked for" number in the error message, the "got" number, or something else?

It's possible - though unlikely - that a program or data error is mangling the file, and thus the error might be local, rather than in New Zealand. Best to eliminate the possibility, just to be on the safe side.
ID: 65822 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,014,785
RAC: 20,946
Message 65823 - Posted: 13 Aug 2022, 8:20:07 UTC - in response to Message 65822.  

Has anyone used a file manager to look at the actual output data file on their hard disk (it'll be in the CPDN project folder in their BOINC data tree). Does the byte count match the "asked for" number in the error message, the "got" number, or something else?


Just checked. On at least one of my testing site tasks it matches at 90,375,444bytes. That doesn't guarantee no problems elsewhere but given that George's task finished and uploaded OK from testing that suggests that it certainly isn't a universal problem. Also, I am now getting internet access OK, project servers may be down.
ID: 65823 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,705,793
RAC: 9,655
Message 65824 - Posted: 13 Aug 2022, 8:24:39 UTC - in response to Message 65823.  
Last modified: 13 Aug 2022, 8:24:56 UTC

Thanks - just checking. The next step would be http_debug (NOT xfer debug), to see what the server is actually telling you.
ID: 65824 · Report as offensive     Reply Quote
Harri Liljeroos

Send message
Joined: 9 Dec 05
Posts: 116
Credit: 12,547,934
RAC: 2,738
Message 65826 - Posted: 13 Aug 2022, 9:41:29 UTC

All of my uploads have gone thru, but the web site is missing all the trickles. The trickles were there on Thursday evening.
ID: 65826 · Report as offensive     Reply Quote
rob

Send message
Joined: 5 Jun 09
Posts: 97
Credit: 3,736,855
RAC: 4,073
Message 65827 - Posted: 13 Aug 2022, 9:44:21 UTC - in response to Message 65824.  
Last modified: 13 Aug 2022, 9:46:31 UTC

Richard - I just set http_debug....
"Instantly" all the tasks in transfer vanished from view, some of them reappeared about half a minute later.
One thing I noticed was a number of blank lines - here's a sample @ 10:23:23:
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Received header from server: Server: Apache/2.4.7 (Ubuntu)
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Received header from server: Vary: Accept-Encoding
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Received header from server: Content-Encoding: gzip
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Received header from server: Content-Length: 84
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Received header from server: Content-Type: text/plain
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Received header from server:
13/08/2022 10:23:23 | climateprediction.net | 
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Info:  Connection #57 to host upload4.cpdn.org left intact
13/08/2022 10:23:23 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [can pipeline]
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Info:  Re-using existing connection! (#57) with host upload4.cpdn.org
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Info:  Connected to upload4.cpdn.org (131.217.169.79) port 80 (#57)
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Sent header to server: Host: upload4.cpdn.org
13/08/2022 10:23:23 | climateprediction.net | [http] [ID#228] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.16.20)



Scrolling back in time to just before I set http_debug:
13/08/2022 10:13:36 | climateprediction.net | [sched_op] Starting scheduler request
13/08/2022 10:13:37 | climateprediction.net | Sending scheduler request: To send trickle-up message.
13/08/2022 10:13:37 | climateprediction.net | Not requesting tasks: don't need (CPU: job cache full; NVIDIA GPU: )
13/08/2022 10:13:37 | climateprediction.net | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
13/08/2022 10:13:37 | climateprediction.net | [sched_op] NVIDIA GPU work request: 0.00 seconds; 0.00 devices
13/08/2022 10:13:41 | climateprediction.net | Scheduler request completed
13/08/2022 10:13:41 | climateprediction.net | [sched_op] Server version 715
13/08/2022 10:13:41 | climateprediction.net | Project requested delay of 3636 seconds
13/08/2022 10:13:41 | climateprediction.net | [sched_op] Deferring communication for 01:00:36
13/08/2022 10:13:41 | climateprediction.net | [sched_op] Reason: requested by project
13/08/2022 10:13:45 | climateprediction.net | Started upload of wah2_nz25_a1i4_199805_25_936_012151710_0_r2018106213_4.zip
13/08/2022 10:18:53 |  | Project communication failed: attempting access to reference site
13/08/2022 10:18:53 | climateprediction.net | Temporarily failed upload of wah2_nz25_a1i4_199805_25_936_012151710_0_r2018106213_4.zip: transient HTTP error
13/08/2022 10:18:53 | climateprediction.net | Backing off 00:03:35 on upload of wah2_nz25_a1i4_199805_25_936_012151710_0_r2018106213_4.zip



Then:
13/08/2022 10:20:09 |  | Internet access OK - project servers may be temporarily down.
13/08/2022 10:20:37 |  | Re-reading cc_config.xml
13/08/2022 10:20:37 |  | Using proxy info from GUI
13/08/2022 10:20:37 |  | Config: don't compute while Cities.exe is running
13/08/2022 10:20:37 |  | Config: event log limit 20000 lines
13/08/2022 10:20:37 |  | Config: use all coprocessors
13/08/2022 10:20:37 |  | log flags: file_xfer, sched_ops, task, http_debug, sched_op_debug
13/08/2022 10:20:37 | DENIS@home | Found app_config.xml
13/08/2022 10:20:37 | Einstein@Home | Found app_config.xml
13/08/2022 10:20:37 | LHC@home | Found app_config.xml
13/08/2022 10:20:40 |  | Re-reading cc_config.xml
13/08/2022 10:20:40 |  | Using proxy info from GUI
13/08/2022 10:20:40 |  | Config: don't compute while Cities.exe is running
13/08/2022 10:20:40 |  | Config: event log limit 20000 lines
13/08/2022 10:20:40 |  | Config: use all coprocessors
13/08/2022 10:20:40 |  | log flags: file_xfer, sched_ops, task, http_debug, sched_op_debug
13/08/2022 10:20:40 | DENIS@home | Found app_config.xml
13/08/2022 10:20:40 | Einstein@Home | Found app_config.xml
13/08/2022 10:20:40 | LHC@home | Found app_config.xml


Followed by a block like this (trying to upload the files that vanished from view?):
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:21:51 | climateprediction.net | Started upload of wah2_nz25_a1ga_199805_25_936_012151644_0_r2029078922_2.zip
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:21:51 | climateprediction.net | Started upload of wah2_nz25_a1ga_199805_25_936_012151644_0_r2029078922_3.zip
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:21:51 | climateprediction.net | Started upload of wah2_nz25_a1i4_199805_25_936_012151710_0_r2018106213_2.zip
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:21:51 | climateprediction.net | Started upload of wah2_nz25_a1k4_199905_25_936_012151782_0_r706971930_2.zip
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
13/08/2022 10:21:51 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:21:51 | climateprediction.net | Started upload of wah2_nz25_a1g2_199805_25_936_012151636_0_r1401480091_2.zip


And:
13/08/2022 10:21:51 | climateprediction.net | [http] [ID#220] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [serially]
13/08/2022 10:21:51 | climateprediction.net | [http] [ID#221] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [serially]
13/08/2022 10:21:51 | climateprediction.net | [http] [ID#222] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [serially]
13/08/2022 10:21:51 | climateprediction.net | [http] [ID#223] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [serially]
13/08/2022 10:21:51 | climateprediction.net | [http] [ID#224] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [serially]
13/08/2022 10:21:51 | climateprediction.net | [http] [ID#225] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [serially]
13/08/2022 10:21:51 | climateprediction.net | [http] [ID#226] Info:  Found bundle for host upload4.cpdn.org: 0x43d7230 [serially]


Then a lad of lines like these:
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#219] Info:    Trying 131.217.169.79...
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#220] Info:  Hostname was found in DNS cache
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#220] Info:    Trying 131.217.169.79...
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Info:  Hostname was found in DNS cache
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Info:    Trying 131.217.169.79...
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#222] Info:  Hostname was found in DNS cache


Then another block of lies like these:
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Info:  Connected to upload4.cpdn.org (131.217.169.79) port 80 (#56)
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: Host: upload4.cpdn.org
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.16.20)
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: Accept: */*
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: Accept-Encoding: deflate, gzip
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: Accept-Language: en_GB
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: Content-Length: 312
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: Content-Type: application/x-www-form-urlencoded
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server:
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: poll_debug>
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server:         <priority_debug>0</priority_debug>


Loads more lines - the last few are:
[code]
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Connection #55 to host upload4.cpdn.org left intact
13/08/2022 10:32:18 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:32:18 | climateprediction.net | Finished upload of wah2_nz25_a1i4_199805_25_936_012151710_0_r2018106213_3.zip
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Found bundle for host upload4.cpdn.org: 0x43d7230 [can pipeline]
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Re-using existing connection! (#55) with host upload4.cpdn.org
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Connected to upload4.cpdn.org (131.217.169.79) port 80 (#55)
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Host: upload4.cpdn.org
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.16.20)
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Accept: */*
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Accept-Encoding: deflate, gzip
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Accept-Language: en_GB
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Content-Length: 90401533
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Content-Type: application/x-www-form-urlencoded
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Expect: 100-continue
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server:
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Received header from server: HTTP/1.1 100 Continue
13/08/2022 10:37:36 | climateprediction.net | [http] [ID#228] Info: We are completely uploaded and fine
[/code]

I started with about 20 or so zip files to upload022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: poll_debug>
13/08/2022 10:21:52 | climateprediction.net | [http] [ID#221] Sent header to server: <priority_debug>0</priority_debug>
[/code]

Loads more lines - the last few are:
[code]
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Connection #55 to host upload4.cpdn.org left intact
13/08/2022 10:32:18 | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
13/08/2022 10:32:18 | climateprediction.net | Finished upload of wah2_nz25_a1i4_199805_25_936_012151710_0_r2018106213_3.zip
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Found bundle for host upload4.cpdn.org: 0x43d7230 [can pipeline]
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Re-using existing connection! (#55) with host upload4.cpdn.org
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Info: Connected to upload4.cpdn.org (131.217.169.79) port 80 (#55)
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Host: upload4.cpdn.org
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.16.20)
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Accept: */*
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Accept-Encoding: deflate, gzip
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Accept-Language: en_GB
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Content-Length: 90401533
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Content-Type: application/x-www-form-urlencoded
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server: Expect: 100-continue
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Sent header to server:
13/08/2022 10:32:18 | climateprediction.net | [http] [ID#246] Received header from server: HTTP/1.1 100 Continue
13/08/2022 10:37:36 | climateprediction.net | [http] [ID#228] Info: We are completely uploaded and fine
[/code]

I started with about 20 or so zip files to upload, now down to 7, plus one that's 100% progress, but in "upload: retry in 01:35:xx". Once the rest have cleared I might give that one a nudge....
Have fun
(I should say I'm running BOINC version 7.16.20 on Windows 10 with Virtual box.)
ID: 65827 · Report as offensive     Reply Quote
rob

Send message
Joined: 5 Jun 09
Posts: 97
Credit: 3,736,855
RAC: 4,073
Message 65828 - Posted: 13 Aug 2022, 9:59:59 UTC
Last modified: 13 Aug 2022, 10:01:58 UTC

Dialogue around a zip upload stalling:
13/08/2022 10:56:38 | climateprediction.net | 
13/08/2022 10:56:38 | climateprediction.net | [http] [ID#257] Info:  Connection #56 to host upload4.cpdn.org left intact
13/08/2022 10:56:39 | climateprediction.net | [error] Error reported by file upload server: EOF on socket read : asked for 262144, got 159040
13/08/2022 10:56:39 | climateprediction.net | Temporarily failed upload of wah2_nz25_a1g2_199805_25_936_012151636_0_r1401480091_4.zip: transient upload error
13/08/2022 10:56:39 | climateprediction.net | Backing off 00:02:05 on upload of wah2_nz25_a1g2_199805_25_936_012151636_0_r1401480091_4.zip
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Info:  We are completely uploaded and fine
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server: HTTP/1.1 200 OK
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server: Date: Sat, 13 Aug 2022 09:46:57 GMT
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server: Server: Apache/2.4.7 (Ubuntu)
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server: Vary: Accept-Encoding
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server: Content-Encoding: gzip
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server: Content-Length: 123
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server: Content-Type: text/plain
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Received header from server:
13/08/2022 10:57:01 | climateprediction.net |
 


Edit to add:
A bit more arrived as I was typing:
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Info:  Connection #72 to host upload4.cpdn.org left intact
13/08/2022 10:57:02 | climateprediction.net | [error] Error reported by file upload server: EOF on socket read : asked for 262144, got 157592
13/08/2022 10:57:02 | climateprediction.net | Temporarily failed upload of wah2_nz25_a1k4_199905_25_936_012151782_0_r706971930_4.zip: transient upload error
13/08/2022 10:57:02 | climateprediction.net | Backing off 00:04:34 on upload of wah2_nz25_a1k4_199905_25_936_012151782_0_r706971930_4.zip
ID: 65828 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,705,793
RAC: 9,655
Message 65830 - Posted: 13 Aug 2022, 12:09:00 UTC - in response to Message 65828.  

13/08/2022 10:18:53 | climateprediction.net | Temporarily failed upload of wah2_nz25_a1i4_199805_25_936_012151710_0_r2018106213_4.zip: transient HTTP error
That's the one I was hoping to explore. The answer seems to be

13/08/2022 10:56:39 | climateprediction.net | [error] Error reported by file upload server: EOF on socket read : asked for 262144, got 159040
13/08/2022 10:56:39 | climateprediction.net | Temporarily failed upload of wah2_nz25_a1g2_199805_25_936_012151636_0_r1401480091_4.zip: transient upload error
13/08/2022 10:56:39 | climateprediction.net | Backing off 00:02:05 on upload of wah2_nz25_a1g2_199805_25_936_012151636_0_r1401480091_4.zip
13/08/2022 10:57:01 | climateprediction.net | [http] [ID#265] Info: We are completely uploaded and fine
How big are your _4.zip files? The client seems to think it's sent it all, but neither 'asked for' nor 'got' match the numbers Dave posted earlier.

I think the process goes like this - it should all be logged, though it's easier to see if you can retry a single file at a time:

The client asks 'can I upload a file of 262144 bytes'?
The server replies 'OK, go ahead'
The client says 'here you are, then', and starts uploading what it finds on disk.

Which is - ???
ID: 65830 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,014,785
RAC: 20,946
Message 65831 - Posted: 13 Aug 2022, 12:31:50 UTC - in response to Message 65830.  
Last modified: 13 Aug 2022, 12:59:47 UTC

How big are your _4.zip files? The client seems to think it's sent it all, but neither 'asked for' nor 'got' match the numbers Dave posted earlier.


The one I posted was a 5.zip from one of the main site batches. The messages from my testing branch tasks before I started getting servers problem messages have now dropped off the back of the log.

Edit: now uploading again. will see what happens with the two uploads from testing and two from main site now running and check sizes of them if they fail

Enabled http debug and the 25.zip from testing promptly succeeded.
ID: 65831 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 65832 - Posted: 13 Aug 2022, 13:03:24 UTC

When the ANZ models first arrived years ago, the servers were at a big new data center in Hobart, the capitol of Tasmania.
And the south east of Australia has been having severe weather with flooding, on and off all the year, including in Tasmania.
The rain radar currently shows rain in the Hobart area.

So server "flooding" could have two meanings. :(
ID: 65832 · Report as offensive     Reply Quote
rob

Send message
Joined: 5 Jun 09
Posts: 97
Credit: 3,736,855
RAC: 4,073
Message 65833 - Posted: 13 Aug 2022, 13:07:37 UTC - in response to Message 65830.  
Last modified: 13 Aug 2022, 13:08:21 UTC

BOINC reported all the files to be in the mid to high 80Mb.

At the time of me seeing your comment I had one being uploaded, BOINC reported as ~86Mb, and was about the same when file was on the disc (needless to say it uploaded as I was looking at it and your message so I didn't catch the right sets of digits.....)
ID: 65833 · Report as offensive     Reply Quote
1 · 2 · 3 · Next

Message boards : Number crunching : NZ25 file upload server problems?

©2024 cpdn.org