climateprediction.net (CPDN) home page
Thread 'Lots of pending WUs'

Thread 'Lots of pending WUs'

Message boards : Number crunching : Lots of pending WUs
Message board moderation

To post messages, you must log in.

AuthorMessage
ProfileSteve Dodd

Send message
Joined: 28 Oct 11
Posts: 15
Credit: 9,897,850
RAC: 9,723
Message 58665 - Posted: 30 Aug 2018, 15:45:40 UTC

Hi, didn't see this addressed anywhere else. I have 858 WUs pending validation. Seems like a lot. Am I being too impatient?
ID: 58665 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 58666 - Posted: 30 Aug 2018, 15:51:42 UTC - in response to Message 58665.  

They don't compare work units from different users on CPDN, so they never get "validated".
ID: 58666 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,002,360
RAC: 21,497
Message 58667 - Posted: 30 Aug 2018, 17:08:32 UTC - in response to Message 58666.  

They don't compare work units from different users on CPDN, so they never get "validated".


Or in other words, completed tasks are always labelled, "pending" with CPDN because the BOINC server software doesn't have an option to say that they don't need validating.There are two or three other projects like this but I can't remember which they are off the top of my head.
ID: 58667 · Report as offensive     Reply Quote
CJ Xuereb

Send message
Joined: 24 Oct 16
Posts: 6
Credit: 1,866,525
RAC: 1,022
Message 58668 - Posted: 31 Aug 2018, 23:30:53 UTC

Collatz
GPUGRID
Moo! Wrapper
World Community Grid (compares if you return errors)
Rosetta

None of the above compare work units from different users and yet none of the tasks are listed as 'pending validation'

So I think the BOINC software does have such an option.

CPDN must be doing something wrong.
ID: 58668 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 58669 - Posted: 1 Sep 2018, 2:45:44 UTC - in response to Message 58668.  

There's several BOINC options that cpdn just ignores.

Validation and Pending credit are two of them.
ID: 58669 · Report as offensive     Reply Quote
ProfileastroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 58674 - Posted: 1 Sep 2018, 19:02:56 UTC

... however, there is quality control -- it is done mathematically.
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 58674 · Report as offensive     Reply Quote
Nuadormrac
Avatar

Send message
Joined: 14 Oct 05
Posts: 44
Credit: 4,011,789
RAC: 8,782
Message 58680 - Posted: 2 Sep 2018, 11:40:19 UTC - in response to Message 58666.  

Primaboinca has a quorum of 1, hence no comparison, but doesn't show pending tasks. I'm not sure how their project admins do it differently from the project admins here. Perhaps they could chat, if they are uncertain, or if we got credits it wouldn't really be that big a deal...
ID: 58680 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,002,360
RAC: 21,497
Message 58681 - Posted: 2 Sep 2018, 13:19:14 UTC - in response to Message 58680.  

or if we got credits it wouldn't really be that big a deal...


Once things are fully sorted, credit will be granted again. There was an attempt to run the credit script last week but it didn't complete. Andy has started to go through the logs looking for the problem. I think he said it got to about the 7 hour point before it failed.
ID: 58681 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,002,360
RAC: 21,497
Message 58682 - Posted: 2 Sep 2018, 13:22:09 UTC
Last modified: 2 Sep 2018, 14:04:17 UTC

Primaboinca has a quorum of 1, hence no comparison, but doesn't show pending tasks. I'm not sure how their project admins do it differently from the project admins here. Perhaps they could chat,


Based on what others have said, it may be that CPDN are just leaving everything they don't have to fiddle with on the default settings if they don't interfere with the business. Changing this would produce a significant decrease in traffic on the forums. :)
ID: 58682 · Report as offensive     Reply Quote
William Timbrook

Send message
Joined: 25 Sep 05
Posts: 9
Credit: 6,955,234
RAC: 0
Message 58683 - Posted: 2 Sep 2018, 15:50:47 UTC

From what I'm seeing, CPDN is not waiting for the same result from multiple machines to compare the results before issuing the boinc credit.

I have 3 jobs that are listed as successfully completed on Apr 22, 2018 and another 16 jobs that have successfully completed since Jul 20, 2018 that are still showing 0 credit.

I did see a reference before on the message boards that someone at cpdn runs a script that calculates the credit - I don't know what the process is.

Given the current state of the CPDN credit issuing, I lowered my boinc project priority setting. I hope that CPDN does NOT go the way of QMC or some of the other projects that went extinct.

William
ID: 58683 · Report as offensive     Reply Quote
mmonnin

Send message
Joined: 28 May 17
Posts: 49
Credit: 17,296,473
RAC: 6,135
Message 58684 - Posted: 3 Sep 2018, 0:55:19 UTC - in response to Message 58681.  

or if we got credits it wouldn't really be that big a deal...


Once things are fully sorted, credit will be granted again. There was an attempt to run the credit script last week but it didn't complete. Andy has started to go through the logs looking for the problem. I think he said it got to about the 7 hour point before it failed.


Maybe it should be run more often so it doesn't take so long. *GASP*
ID: 58684 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,002,360
RAC: 21,497
Message 58685 - Posted: 3 Sep 2018, 6:03:14 UTC - in response to Message 58684.  

Maybe it should be run more often so it doesn't take so long. *GASP*


Don't know if this was the old script which took over 24 hours to run or the new one which is much quicker.Before the database corruption problem scuppered everything, Andy was working on a script that would work on changes since last run rather than adding everything up since the year dot. If the new script, the plan is it should run daily once fully debugged.
ID: 58685 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 58686 - Posted: 3 Sep 2018, 6:44:36 UTC

There are so many people posting about this that haven't a clue, that it must be time to explain yet again.


cpdn DOES NOT USE the standard BOINC credit system.

Credit here is based on the trickle_up files that get returned several times during the creation of a climate model.

Each different program that is used, uses different amounts of computer resources/time.
Models also take different amounts of time because they cover different amounts of planet area, with different resolutions for the high-res part.

A value for each of these variables is stored in a table.

ALL of the trickle_up files are stored in a data base.

******************

Each time the credits are calculated, it is done from the beginning, for each person, each person's models, each person's computers, each person's team.

So the time taken to do this gets longer each time.
And longer ...
And longer ...
And longer ...
And longer ...

And probably a lot longer than that by now.

And for those that haven't been paying attention, the script has run each day for the past week, and each time it has, BOINCstats has imported it.

Unfortunately, the SQL script isn't working correctly for some reason, which is what Andy is currently trying to work out.
ID: 58686 · Report as offensive     Reply Quote
Brummig

Send message
Joined: 3 Nov 05
Posts: 26
Credit: 687,388
RAC: 529
Message 58687 - Posted: 3 Sep 2018, 12:49:18 UTC - in response to Message 58686.  
Last modified: 3 Sep 2018, 12:49:52 UTC

If the script is running, why have I received no credit for the trickles my host uploaded on the 15th and 28th August? And if the script always starts from the beginning, why hasn't the credit I lost after the crash been restored? This flat-line is shown both on the CPDN website and on BOINCstats. Are you saying the script is running, but not doing anything useful?
ID: 58687 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 58688 - Posted: 3 Sep 2018, 15:03:51 UTC - in response to Message 58687.  

Are you saying the script is running, but not doing anything useful?


Yes.
ID: 58688 · Report as offensive     Reply Quote
ProfileRandi
Avatar

Send message
Joined: 28 Jun 07
Posts: 31
Credit: 4,337,654
RAC: 522
Message 58689 - Posted: 3 Sep 2018, 16:52:05 UTC

Thanks for the responses to Les and Dave.
Also, thanks to Andy working behind the scene.
Zooniverse Old Weather transcriber
and
Old Weather BOINC team member.
ID: 58689 · Report as offensive     Reply Quote

Message boards : Number crunching : Lots of pending WUs

©2024 cpdn.org