climateprediction.net (CPDN) home page
Thread 'Upload issue.'

Thread 'Upload issue.'

Questions and Answers : Windows : Upload issue.
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 5 · Next

AuthorMessage
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 69249 - Posted: 12 Jul 2023, 7:45:00 UTC
Last modified: 12 Jul 2023, 7:53:42 UTC

One of my tasks finished yesterday. One of the completion files did not upload. I have tried retrying the transfer, both last night and this morning. It /appears/ to transfer 100%, but never seems to get a completion status back from the server. It is roughly 118 Mega Bytes .zip file, so nothing dramatic, but stubbornly refusing to get an ack. New work units are downloading without issue.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 69249 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 69251 - Posted: 12 Jul 2023, 8:30:07 UTC - in response to Message 69249.  

Is it this one It has been granted full credit, the same as another task of yours that completed yesterday. If you let me know which file it is I can ask Andy to try and sort it as you may not be the only one affected. Of course with so few being completed for this batch you may be! The issue seems to occur if these large uploads get interrupted for any reason though not every time.
ID: 69251 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 69252 - Posted: 12 Jul 2023, 9:14:31 UTC - in response to Message 69251.  
Last modified: 12 Jul 2023, 9:35:53 UTC

The filename certainly starts like that, but after the "012216342_0", goes on with "_r1775509966_14.zip". The status on the BOINC Manager display shows the status of "Uploading". I've done several retry actions again now, it starts, get to 100% and stops there, goes no further as if it is waiting for an Ack from the server.

<edit>
I've subscribed to the thread so if any further info is wanted, just post it here, or mail, whatever is best for the circumstances.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 69252 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 69253 - Posted: 12 Jul 2023, 10:01:37 UTC - in response to Message 69252.  

Thanks for that, I should have asked what do the messages in the event log say? I have an email ready to click send for Andy.
ID: 69253 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 69255 - Posted: 12 Jul 2023, 11:13:24 UTC

The messages are spectacularly vague as usual. I can't cut and paste for some reason, there are four lines...

Failed to upload <filename> transient HTTP error
Backing off on upload <filename>
Project communication failed attempting access to reference site
Internet access OK - project servers may be down

The HTTP error may be interesting but unspecific. Another completed unit updated without issue, and several new ones have arrived since this morning. Other projects are up/down loading without issue.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 69255 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 69256 - Posted: 12 Jul 2023, 12:13:11 UTC - in response to Message 69255.  

Thanks. There may be more with http debug enabled. If Andy asks for it I will post again.
ID: 69256 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 69262 - Posted: 12 Jul 2023, 16:13:29 UTC

Fine Dave. I am subscribed, so if anything comes, I will get a mail.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 69262 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 69315 - Posted: 14 Jul 2023, 7:01:07 UTC - in response to Message 69262.  

Something else you might want to mention to Andy, concerns my displeasure with the way tasks have been sent. This is a four core eight thread machine, so it downloaded eight work units. These jobs take sixteen to seventeen days to run, and BOINC has stopped work from all my other projects to run them.This, obviously, includes GPU work. I have suspended six of these new tasks and will release them one at a time as jobs finish, (and presumably get stuck like the first one...). I shouldn't have to micromanage this.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 69315 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1052
Credit: 16,796,403
RAC: 13,394
Message 69316 - Posted: 14 Jul 2023, 7:47:11 UTC - in response to Message 69315.  
Last modified: 14 Jul 2023, 7:47:58 UTC

Something else you might want to mention to Andy, concerns my displeasure with the way tasks have been sent. This is a four core eight thread machine, so it downloaded eight work units. These jobs take sixteen to seventeen days to run, and BOINC has stopped work from all my other projects to run them.This, obviously, includes GPU work. I have suspended six of these new tasks and will release them one at a time as jobs finish, (and presumably get stuck like the first one...). I shouldn't have to micromanage this.
The issue is with the way boinc works, not the way CPDN issue tasks. CPDN can't influence your machine as you describe.

Couple of things. boinc decides which tasks to run by the priority you set for a project. If CPDN has a higher priority, it will run them in preference to other projects. There is a 'bug' in the way boinc works; if a project has not released tasks for a while (Windows tasks in this case from CPDN), boinc tends to assign a much higher priority until it settles down, which is probably why it's running them now. Richard has posted elsewhere about this.

The boinc client should switch from CPDN tasks to other projects depending what you have set in the computing preferences: see under 'switch between tasks every x minutes' - though in my experience boinc doesn't tend to work quite like this, it's more like a 'switch no sooner than...'.

You can also create an 'app_config' file on your machine which allows you to once-and-for-all control how many tasks you want to run at the same time on your machine. If interested, I can post another message with an example I use (there are other posts on the forums explaining how to do this with examples).

I don't understand why your GPU work has stopped. On my 6 core machine with 2 GPUs I still have GPUs tasks running even with other projects comsuming the 6 cores I allow. CPDN won't do this.

This isn't a CPDN issue, complain on the boinc forums about the way the client decides to work on your machine. The only control CPDN has is to limit the number of tasks your client can ask for and we only do that for the large memory OpenIFS tasks.

Hope that helps.
---
CPDN Visiting Scientist
ID: 69316 · Report as offensive     Reply Quote
Harri Liljeroos

Send message
Joined: 9 Dec 05
Posts: 116
Credit: 12,567,835
RAC: 1,448
Message 69317 - Posted: 14 Jul 2023, 8:50:50 UTC - in response to Message 69315.  

Something else you might want to mention to Andy, concerns my displeasure with the way tasks have been sent. This is a four core eight thread machine, so it downloaded eight work units. These jobs take sixteen to seventeen days to run, and BOINC has stopped work from all my other projects to run them.This, obviously, includes GPU work. I have suspended six of these new tasks and will release them one at a time as jobs finish, (and presumably get stuck like the first one...). I shouldn't have to micromanage this.

There is on the web site project preferences possibility to set how many tasks you receive from CPDN. Possible selections are 1...8 and 'no limit' (This may be a new option, I don't remember seeing it before on CPDN), I think that 'no limit' is the default.
ID: 69317 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 69318 - Posted: 14 Jul 2023, 9:14:12 UTC - in response to Message 69317.  

I don't remember seeing it before on CPDN), I think that 'no limit' is the default.
Probably came with the recent update of the BOINC-server version. With CPDN, no limit is the default. With other projects it isn't always. For example with Africa Rain Project on WCG the default is only one or two.
ID: 69318 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,794,510
RAC: 4,268
Message 69319 - Posted: 14 Jul 2023, 9:44:52 UTC

That one passed me by, as well. I find the wording ambiguous - what exactly is being limited? It could be:

  • Number of tasks issued per request
  • Number of tasks held on a volunteer's computer, running or ready to run
  • Number of tasks actively running concurrently

I think the most likely intended interpretation is the middle one, whereas adrianxw's comment relates more closely to the bottom one - so it may not help.

I'm finding the new GitHub search facility less helpful than the old one, so I'll have to try and find it by hand - but possibly not until after the weekend.

ID: 69319 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1052
Credit: 16,796,403
RAC: 13,394
Message 69320 - Posted: 14 Jul 2023, 10:22:04 UTC - in response to Message 69319.  

I had asked Andy to put this option in the project preferences. He added it to the dev site sometime ago for the OpenIFS tasks so my guess he's now put it on the main site.

We (CPDN) do still need to change it so that the models can be controlled separately. As OIFS takes alot more memory, the default should be alot lower.

Richard, I believe this controls the server option that limits the no. of tasks the client can have 'in progerss' (I forget the name of the option).
ID: 69320 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,794,510
RAC: 4,268
Message 69321 - Posted: 14 Jul 2023, 10:52:39 UTC - in response to Message 69320.  

Richard, I believe this controls the server option that limits the no. of tasks the client can have 'in progerss' (I forget the name of the option).
But there's still the question of what exactly "in progress" means - and there are two different points of view.

From the server's point of view: it knows what tasks each particular host is holding, but not what the host is doing with them.
From the client's point of view: it holds a number of tasks, and makes its own decision which ones to actively run - based on resource share, recent history, resource availability, user preferences, and probably much more besides.

The server-side restriction is most useful for restricting the size of users' caches, and hence speeding up the latency in work flow. But for constraints like 'maximum memory per core', it's the client settings that need attention.
ID: 69321 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 69322 - Posted: 14 Jul 2023, 11:10:51 UTC

Pretty sure, "In progress" set on the project website means the number of tasks that show up in a user's task list as, "in progress" so the number downloaded onto the computer. Certainly that is what it means for WCG and the ARP tasks.
ID: 69322 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 69324 - Posted: 14 Jul 2023, 15:40:12 UTC - in response to Message 69322.  
Last modified: 14 Jul 2023, 15:45:32 UTC

I have another of these files "stuck" in upload now. The first one is still there, its Elapsed is now 5:17:12!!! The other has a measly 33 seconds. It is a zip file of comparable size to the first one.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 69324 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 69326 - Posted: 14 Jul 2023, 17:37:48 UTC - in response to Message 69324.  
Last modified: 14 Jul 2023, 19:19:36 UTC

Are these EAS tasks or NZ. Replying with phone so checking not so easy from here.
Edit: If EAS then the server was still down last time I looked.
ID: 69326 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1052
Credit: 16,796,403
RAC: 13,394
Message 69327 - Posted: 15 Jul 2023, 0:50:27 UTC - in response to Message 69322.  
Last modified: 15 Jul 2023, 0:51:56 UTC

Pretty sure, "In progress" set on the project website means the number of tasks that show up in a user's task list as, "in progress" so the number downloaded onto the computer. Certainly that is what it means for WCG and the ARP tasks.
Yes, "in progress" means exactly that. It does not mean number of tasks actually running on the user's host, they could be 'waiting to run', 'suspended' etc. i.e. 'not what the client is doing with them'.

Since this is a server-side control, it's the server's view.
ID: 69327 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 69330 - Posted: 15 Jul 2023, 6:38:08 UTC - in response to Message 69326.  

These stuck transfers are, indeed, eas files. The deadline for the jobs is next summer some time, thus, a curiosity rather than an urgent matter.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 69330 · Report as offensive     Reply Quote
Karen

Send message
Joined: 12 Jul 19
Posts: 9
Credit: 363,587
RAC: 536
Message 69426 - Posted: 28 Jul 2023, 23:50:19 UTC

Hello,
I am also having difficulties with uploading files, for about two weeks now. I now have 12 files of approx 120 MB each waiting to upload. I am running BOINC 7.20.2 for Windows. Here is an extract from the event log:
28/07/2023 18:56:12 | climateprediction.net | Started upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_14.zip
28/07/2023 18:56:12 | climateprediction.net | Started upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_15.zip
28/07/2023 18:56:34 | climateprediction.net | Temporarily failed upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_14.zip: transient HTTP error
28/07/2023 18:56:34 | climateprediction.net | Backing off 04:02:50 on upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_14.zip
28/07/2023 18:56:34 | climateprediction.net | Temporarily failed upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_15.zip: transient HTTP error
28/07/2023 18:56:34 | climateprediction.net | Backing off 05:02:15 on upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_15.zip
28/07/2023 18:56:35 | | Project communication failed: attempting access to reference site
28/07/2023 18:56:37 | | Internet access OK - project servers may be temporarily down.
28/07/2023 19:39:33 | climateprediction.net | Sending scheduler request: To send trickle-up message.
28/07/2023 19:39:33 | climateprediction.net | Not requesting tasks: don't need (CPU: not highest priority project; AMD/ATI GPU: )
28/07/2023 19:39:34 | climateprediction.net | Scheduler request completed
28/07/2023 19:39:34 | climateprediction.net | Project requested delay of 3636 seconds
28/07/2023 19:39:38 | climateprediction.net | Started upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_23.zip
28/07/2023 19:40:00 | | Project communication failed: attempting access to reference site
28/07/2023 19:40:00 | climateprediction.net | Temporarily failed upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_23.zip: transient HTTP error
28/07/2023 19:40:00 | climateprediction.net | Backing off 00:02:56 on upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_23.zip
28/07/2023 19:40:02 | | Internet access OK - project servers may be temporarily down.
28/07/2023 23:31:52 | climateprediction.net | Started upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_13.zip
28/07/2023 23:31:52 | climateprediction.net | Started upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_14.zip
28/07/2023 23:32:14 | climateprediction.net | Temporarily failed upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_13.zip: transient HTTP error
28/07/2023 23:32:14 | climateprediction.net | Backing off 05:52:36 on upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_13.zip
28/07/2023 23:32:14 | climateprediction.net | Temporarily failed upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_14.zip: transient HTTP error
28/07/2023 23:32:14 | climateprediction.net | Backing off 04:27:12 on upload of wah2_eas25_a1om_199911_25_994_012217620_1_r405956311_14.zip
28/07/2023 23:32:15 | | Project communication failed: attempting access to reference site
28/07/2023 23:32:17 | | Internet access OK - project servers may be temporarily down.
ID: 69426 · Report as offensive     Reply Quote
1 · 2 · 3 · 4 . . . 5 · Next

Questions and Answers : Windows : Upload issue.

©2024 cpdn.org