climateprediction.net (CPDN) home page
Thread 'Uploads slow and overtake the system'

Thread 'Uploads slow and overtake the system'

Message boards : Number crunching : Uploads slow and overtake the system
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
frazil

Send message
Joined: 15 Jun 12
Posts: 10
Credit: 1,168,894
RAC: 0
Message 65851 - Posted: 17 Aug 2022, 15:20:15 UTC

I have roughly 40 files to upload to the project at the moment, and they are 86 mb each. The problem I am seeing is that they download really slowly, at like 50 kbs maximum (2 at a time), and somehow that seems to overload all other uses of my internet. That doesn't make sense, as on Speedtest.net, my standard and consistent upload speed is 600 kbs.

So...
1. Is this a normal upload speed for this project, or will it be able to receive uploads at a higher speed soon?
2. Why does this seem to restrict any other uses of my internet, when it is only using less that a 1/10 of what I can upload things at?
3. ESPECIALLY, why does this seem to cause problems for using my internet to download things, such as streaming shows and music?? I do not have this problem with any other BOINC projects uploading or downloading their data. Do they have something misconfigured with their project uploads somehow?
4. Someone mentioned Australia/New Zealand. Is that where the servers are located?
5. I have 5 files that say they are at 100%, but just sit in the transfer queue. Why don't those get fully completed, and removed from the queue?

If someone could fix this, or answer my questions, that would be great.
ID: 65851 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65852 - Posted: 17 Aug 2022, 16:30:52 UTC - in response to Message 65851.  

Hi, Your computers are hidden so I can't be sure. If your tasks are Windows ones then your zips will be going to AU/NZ. That server has a problem currently. Andy thinks it is being overloaded. Your current upload speed is what I get on a good day with a following wind, so I can't check that. I have three zips that are stuck, getting to 100% and then failing. I am turning internet access off between attempts to get the zips to upload.

I don''t know of any problem on the Linux batch at the moment which goes to a different server.

I am guessing here but is it possible that constant resends of data because of the server problems mean that far more data is going over your connection than what is shown by BOINC? If you have software to monitor your data that could answer that. The adjacent thread to this shows several people are having problems with zips for this batch so it is not a problem your end, again assuming windows tasks. And it is still in the small hours in NZ so Suzanne, whose tasks these are will probably not see my latest message for a few hours yet!
ID: 65852 · Report as offensive     Reply Quote
frazil

Send message
Joined: 15 Jun 12
Posts: 10
Credit: 1,168,894
RAC: 0
Message 65853 - Posted: 17 Aug 2022, 23:07:45 UTC - in response to Message 65852.  

Just looking at what I have to upload, it is 3.4 terabytes of data on the small side (and growing constantly). Is that really what is expected when doing these projects? I thought these projects focused more on using our computers for data crunching, and less about sending large files over the internet. Can you clarify? Is this typical?
ID: 65853 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 65854 - Posted: 18 Aug 2022, 0:53:48 UTC - in response to Message 65853.  

This is Climateprediction.
It is modelling the Earth's climate, and creates big files.
The newer models are modelling finer detail, and the data files are getting bigger.

Unfortunately, there is some sort of problem at the server end, which in this case is in a data center in Hobart, the capital of Tasmania.
ID: 65854 · Report as offensive     Reply Quote
KAMasud

Send message
Joined: 6 Oct 06
Posts: 204
Credit: 7,608,986
RAC: 0
Message 65855 - Posted: 18 Aug 2022, 9:52:08 UTC

HADSM4.
scheduler request failed; ssl peer certificate or ssh remote key was not ok.
Another problem is. on the first attempt to access the server page, I land somewhere else.
Intermittent service is another.
ID: 65855 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65856 - Posted: 18 Aug 2022, 10:44:03 UTC

Still getting this on the Windows tasks.
18/08/2022 11:14:26 | climateprediction.net | 
18/08/2022 11:14:26 | climateprediction.net | [http] [ID#16] Info:  Connection #25 to host upload4.cpdn.org left intact
18/08/2022 11:14:27 | climateprediction.net | [error] Error reported by file upload server: EOF on socket read : asked for 262144, got 144308
18/08/2022 11:14:27 | climateprediction.net | [error] Error reported by file upload server: EOF on socket read : asked for 262144, got 148628
18/08/2022 11:14:27 | climateprediction.net | Temporarily failed upload of wah2_nz25_a2r9_200905_25_936_012153335_0_r2007975233_7.zip: transient upload error
18/08/2022 11:14:27 | climateprediction.net | Backing off 03:32:50 on upload of wah2_nz25_a2r9_200905_25_936_012153335_0_r2007975233_7.zip
18/08/2022 11:14:27 | climateprediction.net | Temporarily failed upload of wah2_nz25_a2r9_200905_25_936_012153335_0_r2007975233_9.zip: transient upload error
18/08/2022 11:14:27 | climateprediction.net | Backing off 00:42:53 on upload of wah2_nz25_a2r9_200905_25_936_012153335_0_r2007975233_9.zip

I have informed the project. That still only three tasks showing as completed suggests most if not all are having this problem.
ID: 65856 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65857 - Posted: 18 Aug 2022, 15:04:14 UTC

A fourth task has just shown as completed, so someone has gotten through. I am retrying my uploads to see if any make it!
ID: 65857 · Report as offensive     Reply Quote
KAMasud

Send message
Joined: 6 Oct 06
Posts: 204
Credit: 7,608,986
RAC: 0
Message 65858 - Posted: 18 Aug 2022, 18:04:48 UTC

Not me.
18/08/2022 8:51:21 am | climateprediction.net | Sending scheduler request: To send trickle-up message.
18/08/2022 8:51:21 am | climateprediction.net | Not requesting tasks: don't need (CPU: job cache full; NVIDIA GPU: ; Intel GPU: )
18/08/2022 8:51:23 am | climateprediction.net | Scheduler request completed
18/08/2022 8:51:23 am | climateprediction.net | Project requested delay of 3636 seconds
18/08/2022 10:53:25 am | climateprediction.net | Started upload of wah2_nz25_a2db_200605_25_936_012152833_0_r1306077880_24.zip
18/08/2022 10:53:25 am | climateprediction.net | Started upload of wah2_nz25_a2db_200605_25_936_012152833_0_r1306077880_14.zip
18/08/2022 10:58:32 am | climateprediction.net | Temporarily failed upload of wah2_nz25_a2db_200605_25_936_012152833_0_r1306077880_14.zip: transient HTTP error
18/08/2022 10:58:32 am | climateprediction.net | Backing off 03:21:12 on upload of wah2_nz25_a2db_200605_25_936_012152833_0_r1306077880_14.zip
18/08/2022 10:58:32 am | climateprediction.net | Temporarily failed upload of wah2_nz25_a2db_200605_25_936_012152833_0_r1306077880_24.zip: transient HTTP error
18/08/2022 10:58:32 am | climateprediction.net | Backing off 00:05:37 on upload of wah2_nz25_a2db_200605_25_936_012152833_0_r1306077880_24.zip
18/08/2022 10:58:33 am | | Project communication failed: attempting access to reference site
18/08/2022 10:58:35 am | | Internet access OK - project servers may be temporarily down.
ID: 65858 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,884,880
RAC: 19,188
Message 65859 - Posted: 18 Aug 2022, 20:08:26 UTC

It seems like some of them go through after days of trying, I had one more go through last night, but new ones get created at a faster pace. I now have 5 files again trying to upload.
ID: 65859 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65862 - Posted: 18 Aug 2022, 21:16:12 UTC - in response to Message 65855.  

HADSM4.
scheduler request failed; ssl peer certificate or ssh remote key was not ok.
Another problem is. on the first attempt to access the server page, I land somewhere else.
Intermittent service is another.
Just had a hadsm4 zip go through with no problems. if you are still getting problems could you post the output in the event log with http debug enabled please.
ID: 65862 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 65863 - Posted: 18 Aug 2022, 23:25:25 UTC - in response to Message 65862.  
Last modified: 18 Aug 2022, 23:27:05 UTC

I have 3GB (I assume the other user meant GB not TB) waiting, related to about 15 tasks. Occasionally things work, but not often.

Having just tried to make an event log for you, all 6 of my computers are now uploading ok. Seems things are fixed. You're not the only project, I'm having trouble communicating with DENIS and WCG aswell.

I noticed no internet slowdown though, despite only having a connection 1.5 times faster than the other user.
ID: 65863 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 65864 - Posted: 18 Aug 2022, 23:53:48 UTC
Last modified: 18 Aug 2022, 23:57:00 UTC

They're uploading to 100% but never get acknowledged.

This is what I can see so far after turning http debug on (it's in the middle of several uploads, so you'll have to work out which is which).

Ryzen

10162	climateprediction.net	19-08-2022 12:51 AM	Started upload of wah2_nz25_a2dh_200605_25_936_012152839_0_r1231343251_13.zip	
10163	climateprediction.net	19-08-2022 12:51 AM	Started upload of wah2_nz25_a2dh_200605_25_936_012152839_0_r1231343251_20.zip	
10164	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Connection 2243 seems to be dead	
10165	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Closing connection 2243	
10166	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Connection 2245 seems to be dead	
10167	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Closing connection 2245	
10168	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Connection 2257 seems to be dead	
10169	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Closing connection 2257	
10170	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Connection 2258 seems to be dead	
10171	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Closing connection 2258	
10172	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Info:  Found bundle for host: 0x24687005c40 [serially]	
10173	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:    Trying 131.217.169.79:80...	
10174	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Info:  Hostname 'upload4.cpdn.org' was found in DNS cache	
10175	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Info:    Trying 131.217.169.79:80...	
10176	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Info:  Connected to upload4.cpdn.org (131.217.169.79) port 80 (#2262)	
10177	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
	
10178	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: Host: upload4.cpdn.org
	
10179	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.20.2)
	
10180	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: Accept: */*
	
10181	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: Accept-Encoding: deflate, gzip
	
10182	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: Accept-Language: en_GB
	
10183	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: Content-Length: 312
	
10184	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: Content-Type: application/x-www-form-urlencoded
	
10185	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Sent header to server: 
	
10186	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3070] Info:  We are completely uploaded and fine	
10187	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  Connected to upload4.cpdn.org (131.217.169.79) port 80 (#2261)	
10188	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
	
10189	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: Host: upload4.cpdn.org
	
10190	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.20.2)
	
10191	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: Accept: */*
	
10192	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: Accept-Encoding: deflate, gzip
	
10193	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: Accept-Language: en_GB
	
10194	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: Content-Length: 312
	
10195	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: Content-Type: application/x-www-form-urlencoded
	
10196	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Sent header to server: 
	
10197	climateprediction.net	19-08-2022 12:51 AM	[http] [ID#3069] Info:  We are completely uploaded and fine	

Ryzen

10198	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3069] Info:  Operation too slow. Less than 10 bytes/sec transferred the last 300 seconds	
10199	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3069] Info:  Closing connection 2261	
10200	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3070] Info:  Operation too slow. Less than 10 bytes/sec transferred the last 300 seconds	
10201	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3070] Info:  Closing connection 2262	
10202	climateprediction.net	19-08-2022 12:56 AM	[http] HTTP error: Timeout was reached	
10203	climateprediction.net	19-08-2022 12:56 AM	[http] HTTP error: Timeout was reached	
10204			19-08-2022 12:56 AM	Project communication failed: attempting access to reference site	
10205			19-08-2022 12:56 AM	[http] HTTP_OP::init_get(): https://www.google.com/	
10206	climateprediction.net	19-08-2022 12:56 AM	Temporarily failed upload of wah2_nz25_a2dh_200605_25_936_012152839_0_r1231343251_13.zip: transient HTTP error	
10207	climateprediction.net	19-08-2022 12:56 AM	Backing off 03:53:00 on upload of wah2_nz25_a2dh_200605_25_936_012152839_0_r1231343251_13.zip	
10208	climateprediction.net	19-08-2022 12:56 AM	Temporarily failed upload of wah2_nz25_a2dh_200605_25_936_012152839_0_r1231343251_20.zip: transient HTTP error	
10209	climateprediction.net	19-08-2022 12:56 AM	Backing off 02:24:59 on upload of wah2_nz25_a2dh_200605_25_936_012152839_0_r1231343251_20.zip	
10210	climateprediction.net	19-08-2022 12:56 AM	Started upload of wah2_nz25_a2f1_200605_25_936_012152895_0_r643599966_21.zip	
10211			19-08-2022 12:56 AM	[http] [ID#0] Info:  Found bundle for host: 0x24687005880 [serially]	
10212	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Info:    Trying 131.217.169.79:80...	
10213			19-08-2022 12:56 AM	[http] [ID#0] Info:    Trying 142.250.200.4:443...	
10214			19-08-2022 12:56 AM	[http] [ID#0] Info:  Connected to www.google.com (142.250.200.4) port 443 (#2263)	
10215			19-08-2022 12:56 AM	[http] [ID#0] Info:  schannel: disabled automatic use of client certificate	
10216			19-08-2022 12:56 AM	[http] [ID#0] Info:  ALPN: offers http/1.1	
10217			19-08-2022 12:56 AM	[http] [ID#0] Info:  ALPN: server accepted http/1.1	
10218			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: GET / HTTP/1.1
	
10219			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: Host: www.google.com
	
10220			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.20.2)
	
10221			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: Accept: */*
	
10222			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: Accept-Encoding: deflate, gzip
	
10223			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: Accept-Language: en_GB
	
10224			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: 
	
10225			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server: symstore></symstore>	
10226			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <user_name>Peter Hucker of the Scottish Boinc Team</user_name>	
10227			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <team_name>The Scottish Boinc Team</team_name>	
10228			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <host_venue></host_venue>	
10229			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <email_hash>37298d9897abcee1c2c6fedcff503fd0</email_hash>	
10230			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <cross_project_id>f061da9cc61c8bd5072599aed03def8c</cross_project_id>	
10231			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <external_cpid>53ed9d9b7d568cb7eb1ccc25a7dc4492</external_cpid>	
10232			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <cpid_time>1522841614.000000</cpid_time>	
10233			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <user_total_credit>5058783.265828</user_total_credit>	
10234			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <user_expavg_credit>205806.795552</user_expavg_credit>	
10235			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <user_create_time>1522841614.000000</user_create_time>	
10236			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <rpc_seqno>191</rpc_seqno>	
10237			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <userid>9605</userid>	
10238			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <teamid>502</teamid>	
10239			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <hostid>172697</hostid>	
10240			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <host_total_credit>1066446.201985</host_total_credit>	
10241			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <host_expavg_credit>69474.535774</host_expavg_credit>	
10242			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <host_create_time>1648022035.000000</host_create_time>	
10243			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <nrpc_failures>0</nrpc_failures>	
10244			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <master_fetch_failures>0</master_fetch_failures>	
10245			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <min_rpc_time>0.000000</min_rpc_time>	
10246			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <next_rpc_time>0.000000</next_rpc_time>	
10247			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <rec>8163.317348</rec>	
10248			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <rec_time>1660866970.366072</rec_time>	
10249			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <resource_share>100.000000</resource_share>	
10250			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <disk_usage>917201.000000</disk_usage>	
10251			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <disk_share>11104514685.629246</disk_share>	
10252			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <desired_disk_usage>0.000000</desired_disk_usage>	
10253			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <duration_correction_factor>1.000000</duration_correction_factor>	
10254			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <sched_rpc_pending>0</sched_rpc_pending>	
10255			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <send_time_stats_log>0</send_time_stats_log>	
10256			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <send_job_log>0</send_job_log>	
10257			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <njobs_success>178</njobs_success>	
10258			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <njobs_error>4</njobs_error>	
10259			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <elapsed_time>582687.788100</elapsed_time>	
10260			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <last_rpc_time>1656700372.447107</last_rpc_time>	
10261			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <dont_use_dcf/>	
10262			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <dont_request_more_work/>	
10263			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <rsc_backoff_time>	
10264			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:         <name>CPU</name>	
10265			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:         <value>0.000000</value>	
10266			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     </rsc_backoff_time>	
10267			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:     <rsc_backoff_interval>	
10268			19-08-2022 12:56 AM	[http] [ID#0] Sent header to server:         <name>CPU<	
10269			19-08-2022 12:56 AM	[http] [ID#0] Info:  Mark bundle as not supporting multiuse	
10270			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: HTTP/1.1 200 OK
	
10271			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Date: Thu, 18 Aug 2022 23:56:26 GMT
	
10272			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Expires: -1
	
10273			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Cache-Control: private, max-age=0
	
10274			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Content-Type: text/html; charset=ISO-8859-1
	
10275			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
	
10276			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Content-Encoding: gzip
	
10277			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Server: gws
	
10278			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: X-XSS-Protection: 0
	
10279			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: X-Frame-Options: SAMEORIGIN
	
10280			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Set-Cookie: AEC=AakniGOFILGj8tpFcaVvXjxs9OcsPhqC7R18LpH_ZE02Jfi16oZDu4RQFic; expires=Tue, 14-Feb-2023 23:56:26 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax
	
10281			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Set-Cookie: __Secure-ENID=6.SE=LrYQGweW0E599oBfCFzsUca4DAbUH4siM8jdxxnUMIZJi6Rdnp40jdkExyPbolbp0j_Ne94vmtCI5XYdSmCPHkL_9E9MjnUSrYWtn78ToF-jK6hpiBfx_M40GCeLmZeZ1npfI4_2kcbDs3sH468vNE1A2Ofyzn2q68cTsr3YbmA; expires=	
10282			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Set-Cookie: CONSENT=PENDING+058; expires=Sat, 17-Aug-2024 23:56:26 GMT; path=/; domain=.google.com; Secure
	
10283			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
	
10284			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: Transfer-Encoding: chunked
	
10285			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: 
	
10286			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: 00000001
	
10287			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: 
	
10288			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: 00000001
	
10289			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: ‹
	
10290			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: 00000001
	
10291			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: 
	
10292			19-08-2022 12:56 AM	[http] [ID#0] Received header from server: 00000001
	
10293			19-08-2022 12:56 AM	[http] [ID#0] Info:  Connection #2263 to host www.google.com left intact	
10294	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Info:  Connected to upload4.cpdn.org (131.217.169.79) port 80 (#2264)	
10295	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
	
10296	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: Host: upload4.cpdn.org
	
10297	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.20.2)
	
10298	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: Accept: */*
	
10299	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: Accept-Encoding: deflate, gzip
	
10300	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: Accept-Language: en_GB
	
10301	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: Content-Length: 311
	
10302	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: Content-Type: application/x-www-form-urlencoded
	
10303	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Sent header to server: 
	
10304	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3071] Info:  We are completely uploaded and fine	
10305			19-08-2022 12:56 AM	Internet access OK - project servers may be temporarily down.	
10306	climateprediction.net	19-08-2022 12:56 AM	Started upload of wah2_nz25_a2dh_200605_25_936_012152839_0_r1231343251_14.zip	
10307	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Info:  Found bundle for host: 0x24686edfd20 [serially]	
10308	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Info:  Hostname upload4.cpdn.org was found in DNS cache	
10309	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Info:    Trying 131.217.169.79:80...	
10310	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Info:  Connected to upload4.cpdn.org (131.217.169.79) port 80 (#2265)	
10311	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: POST /cgi-bin/file_upload_handler HTTP/1.1
	
10312	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: Host: upload4.cpdn.org
	
10313	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.20.2)
	
10314	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: Accept: */*
	
10315	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: Accept-Encoding: deflate, gzip
	
10316	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: Accept-Language: en_GB
	
10317	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: Content-Length: 312
	
10318	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: Content-Type: application/x-www-form-urlencoded
	
10319	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Sent header to server: 
	
10320	climateprediction.net	19-08-2022 12:56 AM	[http] [ID#3072] Info:  We are completely uploaded and fine	

ID: 65864 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65865 - Posted: 19 Aug 2022, 4:38:18 UTC - in response to Message 65864.  

Same here, both Oxford and researchers down under know about it. As someone a long time ago said to me, "hurry up and wait."
ID: 65865 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 65867 - Posted: 19 Aug 2022, 5:02:02 UTC - in response to Message 65865.  

Same here, both Oxford and researchers down under know about it. As someone a long time ago said to me, "hurry up and wait."
I only noticed it because I was in the transfers tab checking on the sticky WCG downloads. Otherwise, since the problem at CPDN is just trickle ups, it's not really causing a problem for the users.
ID: 65867 · Report as offensive     Reply Quote
frazil

Send message
Joined: 15 Jun 12
Posts: 10
Credit: 1,168,894
RAC: 0
Message 65868 - Posted: 19 Aug 2022, 6:28:08 UTC - in response to Message 65863.  

I have 3GB (I assume the other user meant GB not TB) waiting, related to about 15 tasks. Occasionally things work, but not often.

Having just tried to make an event log for you, all 6 of my computers are now uploading ok. Seems things are fixed. You're not the only project, I'm having trouble communicating with DENIS and WCG aswell.

I noticed no internet slowdown though, despite only having a connection 1.5 times faster than the other user.


I guess I did mean GB. Sorry brain not working late at night.
ID: 65868 · Report as offensive     Reply Quote
frazil

Send message
Joined: 15 Jun 12
Posts: 10
Credit: 1,168,894
RAC: 0
Message 65869 - Posted: 19 Aug 2022, 6:37:01 UTC

It still doesn't explain why it seems to override most of everything else on my system, such as download for other uses (stream tv for example).
ID: 65869 · Report as offensive     Reply Quote
frazil

Send message
Joined: 15 Jun 12
Posts: 10
Credit: 1,168,894
RAC: 0
Message 65872 - Posted: 19 Aug 2022, 7:26:56 UTC

I'm going to try putting a 20kbs limit on things in the options for this computer, and see if that improves things.

Of note, I just added it, and it looks like I have two tasks trying to report at a time, and each is limited to 20kbs each, and not a total of 20kbs. If that info is useful for others who want to try limiting things.
ID: 65872 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65880 - Posted: 19 Aug 2022, 9:55:40 UTC - in response to Message 65867.  

I only noticed it because I was in the transfers tab checking on the sticky WCG downloads. Otherwise, since the problem at CPDN is just trickle ups, it's not really causing a problem for the users.


The trickle ups are working fine. They go to Oxford. It is the zip files that are getting stuck and taking several attempts to get through and for anyone still on a metered connection lots of files close to 100MB being sent several times is going to eat into their allowance.
ID: 65880 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 65889 - Posted: 19 Aug 2022, 17:53:28 UTC - in response to Message 65869.  

It still doesn't explain why it seems to override most of everything else on my system, such as download for other uses (stream tv for example).
I've never had a problem with internet hogging since I went to semi-fibre (32 down 8 up), although I don't regularly stream HD TV. I usually download it in advance. I don't think Boinc was using anything like my whole connection, as it only retried every hour.
ID: 65889 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 690
Credit: 4,391,754
RAC: 6,918
Message 65890 - Posted: 19 Aug 2022, 17:54:40 UTC - in response to Message 65880.  

The trickle ups are working fine. They go to Oxford. It is the zip files that are getting stuck and taking several attempts to get through and for anyone still on a metered connection lots of files close to 100MB being sent several times is going to eat into their allowance.
Oh, I thought those were trickle ups. I haven't finished any tasks yet, and have 3GB queued to send.

People are still on metered? Ouch.
ID: 65890 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Uploads slow and overtake the system

©2024 cpdn.org