climateprediction.net (CPDN) home page
Thread 'New work Discussion'

Thread 'New work Discussion'

Message boards : Number crunching : New work Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 75 · 76 · 77 · 78 · 79 · 80 · 81 . . . 91 · Next

AuthorMessage
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,722,381
RAC: 7,664
Message 65411 - Posted: 4 May 2022, 16:45:01 UTC - in response to Message 65406.  
Last modified: 4 May 2022, 16:47:17 UTC

Those are very helpful - thanks. There are full download urls in there, which anyone can access - doesn't need to be an Apple machine, or even a machine running BOINC.

I tried the first one in bullschuck's post, using the Chrome browser under Windows 7.

https://download.cpdn.org/download//batch_930/ancils/spec3a_lw_3_asol2c_hadcm3.gz
That looks a bit odd - double // between 'download' and 'batch' - but no dice: connection timed out either way.

But knocking the https:// down to http:// did help. I was able to download the file, and open it using the 7-zip archive manager - it's a real file, present and correct.

So the problem seems to be in the server's SSL security handler. I've noticed at other BOINC projects that servers do take much longer to establish a secure connection, but not several days long!

This sounds like another one for our old friends the University's networking managers. I don't know if the servers they use have a configurable limit for the number of concurrent secure connections they can handle - that might be an easy question to start them off with.
ID: 65411 · Report as offensive
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65412 - Posted: 4 May 2022, 19:53:49 UTC

Well, for now, if you want to get something running, shut down your BOINC client (closing BOINC Manager should do it), head over to /Library/Application Support/BOINC Data, and something like this ought to help you (back up client_state.xml first).

sed -i -e s/'https:\/\/download.cpdn'/'http:\/\/download.cpdn'/ client_state.xml


Be careful... and whatever new tasks you get will be https, so hopefully it's fixed soon. But this unblocked some use of perfectly good photons that weren't otherwise getting used.

Given how... touchy these tasks are about suspending and resuming, don't expect suspend/resume to work. Let everything running drain out before you fiddle with this, though I'm hoping that in a few days, by the time my first batches are done, this will be fixed.
ID: 65412 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65414 - Posted: 5 May 2022, 5:17:25 UTC

Well, for now, if you want to get something running, shut down your BOINC client (closing BOINC Manager should do it), head over to /Library/Application Support/BOINC Data, and something like this ought to help you (back up client_state.xml first).

sed -i -e s/'https:\/\/download.cpdn'/'http:\/\/download.cpdn'/ client_state.xml



Neat. If we get the same issue on Linux tasks I will remember that.

Thank you hopefully the more savvy of our Mack users can use this while waiting for a fix from the project.
ID: 65414 · Report as offensive
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,902,275
RAC: 18,060
Message 65415 - Posted: 5 May 2022, 8:21:04 UTC - in response to Message 65412.  

This worked for me to get a small batch of tasks (on Mojave VM on VBox on Windows10). However it seems to only work once, so to download a second batch one has to go through the process again. client_state.xml file must refresh itself periodically. I had to start a new set of tasks as suspended state prevents new task acquisition so can't try again for about a week. Hopefully this gets fixed by the project by then.
ID: 65415 · Report as offensive
Nigel Garvey

Send message
Joined: 5 May 10
Posts: 69
Credit: 1,169,103
RAC: 2,258
Message 65416 - Posted: 5 May 2022, 8:32:08 UTC - in response to Message 65414.  
Last modified: 5 May 2022, 8:33:05 UTC

Well, for now, if you want to get something running, shut down your BOINC client (closing BOINC Manager should do it), head over to /Library/Application Support/BOINC Data, and something like this ought to help you (back up client_state.xml first).

sed -i -e s/'https:\/\/download.cpdn'/'http:\/\/download.cpdn'/ client_state.xml



Neat. If we get the same issue on Linux tasks I will remember that.

Thank you hopefully the more savvy of our Mack users can use this while waiting for a fix from the project.


The same command in the macOS Terminal would be some variation on:

sed -e 's|https://download.cpdn|http://download.cpdn|' -i '' 'client_state.xml'


But as it's the project's problem, not mine, I'm more inclined to get on with other projects as usual rather than fool around with the actual file in "BOINC Data".
NG
ID: 65416 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65417 - Posted: 5 May 2022, 9:32:13 UTC - in response to Message 65415.  

This worked for me to get a small batch of tasks (on Mojave VM on VBox on Windows10). However it seems to only work once, so to download a second batch one has to go through the process again. client_state.xml file must refresh itself periodically. I had to start a new set of tasks as suspended state prevents new task acquisition so can't try again for about a week. Hopefully this gets fixed by the project by then.
It needs to be done again because the lines being edited are for the files currently stuck. New lines that need editing will be added for the relevant files when more work is downloaded. I would have too check to see if the lines are deleted when the downloads complete or if it happens when the task is reported.
ID: 65417 · Report as offensive
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,722,381
RAC: 7,664
Message 65418 - Posted: 5 May 2022, 9:37:02 UTC - in response to Message 65417.  

it happens when the task is reported.
The files we've seen so far are data files, which change for every task: the server assigns a task, and tells you what files are needed for that particular, individual, task.

The program files which are needed to process the data should be downloaded just once, and kept for future tasks.
ID: 65418 · Report as offensive
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 65420 - Posted: 5 May 2022, 10:05:23 UTC

Hi Folks,
I have 2 Macs that have been trying to download 'Short' tasks since since yesterday. Only just noticed as I hadn't expected any to be available.

Any problems at the site?

Getting the messages :-
Wed 4 May 02:15:46 2022 | climateprediction.net | Temporarily failed download of ozone_cmip6hist+ssp126_2014_2026.gz: connect() failed
Wed 4 May 02:15:46 2022 | climateprediction.net | Backing off 00:03:25 on download of ozone_cmip6hist+ssp126_2014_2026.gz
Wed 4 May 02:15:46 2022 | climateprediction.net | Temporarily failed download of hadcm3s_1mwo_201412_132_930_012138554.zip: connect() failed
Wed 4 May 02:15:46 2022 | climateprediction.net | Backing off 00:03:49 on download of hadcm3s_1mwo_201412_132_930_012138554.zip
Wed 4 May 02:15:46 2022 | climateprediction.net | Started download of 1mwo_2014.ostart.gz
Wed 4 May 02:15:46 2022 | climateprediction.net | Started download of 1mwo_930_atmos.gz
Wed 4 May 02:15:47 2022 | | Internet access OK - project servers may be temporarily down.
Wed 4 May 02:17:03 2022 | | Project communication failed: attempting access to reference site
Wed 4 May 02:17:03 2022 | climateprediction.net | Temporarily failed download of 1mwo_2014.ostart.gz: connect() failed
Wed 4 May 02:17:03 2022 | climateprediction.net | Backing off 00:02:01 on download of 1mwo_2014.ostart.gz
Wed 4 May 02:17:03 2022 | climateprediction.net | Temporarily failed download of 1mwo_930_atmos.gz: connect() failed
Wed 4 May 02:17:03 2022 | climateprediction.net | Backing off 00:02:19 on download of 1mwo_930_atmos.gz
Wed 4 May 02:17:03 2022 | climateprediction.net | Started download of hadcm3s_1bt6_201412_132_930_012138085.zip
Wed 4 May 02:17:03 2022 | climateprediction.net | Started download of 1bt6_2014.ostart.gz
Wed 4 May 02:17:04 2022 | | Internet access OK - project servers may be temporarily down.
ID: 65420 · Report as offensive
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,832,769
RAC: 5,024
Message 65421 - Posted: 5 May 2022, 11:27:33 UTC

The Batch #930 download problems should now be fixed.

(My Mac is now running a model, at least.)
ID: 65421 · Report as offensive
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,722,381
RAC: 7,664
Message 65422 - Posted: 5 May 2022, 11:33:01 UTC
Last modified: 5 May 2022, 11:33:22 UTC

Andy Bowery has emailed back

Thank you all for your messages. This should be fixed now.
and indeed my manual test seems to work now.

A simple 'Retry now' on individual transfers, or ' Retry pending transfers' (tools menu) if you have a lot waiting, should start to get things moving again.

Please don't all click at once!

Perhaps a friendly mod could quietly ask "who dropped what"? It would be nice to add it to the list of "known common errors", for the future.
ID: 65422 · Report as offensive
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 65423 - Posted: 5 May 2022, 11:44:57 UTC - in response to Message 65422.  

Yes, all OK with downloads -= thanks.
ID: 65423 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65427 - Posted: 9 May 2022, 21:21:49 UTC
Last modified: 10 May 2022, 10:12:09 UTC

Doubtless some way off assuming it does happen eventually but there are tasks on the testing site for ARM processors. - I shall have to get myself a PI. I see any broadening out of the potential user base as a good thing. Whether anything will happen to get more native work for windows is anyone's guess though much change there in the short to medium term looks unlikely.

Edit: In fact it is for a particular embedded chip and so won't run on a Pi without virtualisation so probably not going to be relevant to the main site.
ID: 65427 · Report as offensive
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65428 - Posted: 10 May 2022, 15:17:54 UTC

Embedded chip? Is it just some sort of distributed sensor network? An embedded ARM chip doesn't strike me as a good way to get a lot of raw compute work done.

I've got a pile of ARMv7/ARMv8 chips laying around, though, if there's work for them! Quite efficient per unit compute, too.
ID: 65428 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 65429 - Posted: 10 May 2022, 18:50:52 UTC - in response to Message 65428.  

We'll know more "in the future".
Maybe next year. :)
ID: 65429 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65430 - Posted: 10 May 2022, 18:52:53 UTC - in response to Message 65428.  

I have learned a bit more. At this stage, it is a proof of concept thing only so quite a long time before or if anything even makes it to the testing crew outside of people at Oxford. I probably got over-excited at seeing an entry on the server status page for the testing site.
ID: 65430 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65431 - Posted: 13 May 2022, 12:00:19 UTC

Last batch of N216 testing tasks were successful and thetesting of Mac HADCM3s tasks seems to be good so hoping that there will be some main site work soon as nothing else is appearing on the test site, WCG is still down so I am restricted to third and fourth choice projects at the moment.
ID: 65431 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65432 - Posted: 13 May 2022, 13:33:33 UTC - in response to Message 65431.  

The only project that works for my Linux machine is universe@home and even that one had about 3 days of uploads that were not going up until now. Even now, that is pretty spotty.

So ClimatePrediction, Rosetta, WCG are down for all practical purposes.

I am not paranoid enough to believe there is a conspiracy to shut down distributed computing, but I would not be surprised if some though so.
ID: 65432 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65433 - Posted: 13 May 2022, 13:52:20 UTC

The only project that works for my Linux machine is universe@home and even that one had about 3 days of uploads that were not going up until now. Even now, that is pretty spotty.


TN-Grid is working for me. Prime Grid also works but is of little interest to me. Prior to WCG going off air to move everything, Africa Rainfall Project was my go to when nothing available here. Despite the fact that I have run Ubuntu in a VM using virtual box on here, the projects that use a vm don't seem to be working for me.
ID: 65433 · Report as offensive
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,902,275
RAC: 18,060
Message 65436 - Posted: 13 May 2022, 19:47:06 UTC

Yes, non-VBox Rosetta task releases have reduced in frequency recently (they release them in batches like CPDN). The latest update for WCG return is 5/24. Universe has been having issues because of a BOINC pentathlon that's going on and the demand on the server is very high.

One project that could really use more CPU time is MilkyWay, specifically the N-Body Simulation sub-project. They had a disk crash some weeks ago and after a rebuild over 14 million tasks were created unintentionally (usually it's kept at 1000). It's now down to 10+ million tasks that will take weeks to clear. As a result many users have hundreds to thousands of tasks stuck in "validation inconclusive" state as they wait for "wingman" tasks to be crunched for validation. N-Body is a multi-thread app (1 to 16) so app_config.xml is needed to control the number of threads (3-5 should give highest throughput).
ID: 65436 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,053,019
RAC: 14,719
Message 65437 - Posted: 13 May 2022, 22:32:21 UTC - in response to Message 65432.  

The only project that works for my Linux machine is universe@home and even that one had about 3 days of uploads that were not going up until now. Even now, that is pretty spotty.



I was gettng Einstein and Milky Way to work OK on my Linux box. Temporarily shut down to see effect on electrcity consumption. Also waiting for WCG to come back online.
ID: 65437 · Report as offensive
Previous · 1 . . . 75 · 76 · 77 · 78 · 79 · 80 · 81 . . . 91 · Next

Message boards : Number crunching : New work Discussion

©2024 cpdn.org