climateprediction.net (CPDN) home page
Thread 'Trickle and credit delays'

Thread 'Trickle and credit delays'

Message boards : Number crunching : Trickle and credit delays
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Virtual Boss*
Avatar

Send message
Joined: 14 May 08
Posts: 29
Credit: 776,852
RAC: 0
Message 37415 - Posted: 1 Jul 2009, 6:59:35 UTC - in response to Message 37394.  

For God’s sakes, is everyone obsessed with credits? It’s the work that’s important, not the meaningless cobblestones. If we don't stop global warming we are all in serious trouble.


Have you ever considered this:

The obsession for more credits drives some people to crunch more WU's, which gets more work done.

You should be praising those that are obsessed with credits for their beneficial byproduct!
ID: 37415 · Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 13 Jul 05
Posts: 125
Credit: 11,778,421
RAC: 0
Message 37582 - Posted: 30 Jul 2009, 2:38:56 UTC

I see that trickles are not posting again -- past two days anyway. I know they will eventually find their way home. But I've not seen any other comment on this or news information regarding server scripts not running, so I figured to melt the ice as it were.

ID: 37582 · Report as offensive     Reply Quote
ProfileastroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 37583 - Posted: 30 Jul 2009, 3:52:11 UTC

As you note, it will all come out in the wash. It is said that patience is a virtue.
ID: 37583 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 37584 - Posted: 30 Jul 2009, 4:10:26 UTC

Something to do with installing new servers, and some rather serious changes to trickles/credits. Including shutting down the BBC CCE project.
As this is being done by the project people, they already know about it.

I'll post something though.

ID: 37584 · Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 13 Jul 05
Posts: 125
Credit: 11,778,421
RAC: 0
Message 37587 - Posted: 30 Jul 2009, 6:43:49 UTC - in response to Message 37584.  

Thanks -- information exchange is a good thing.

Something to do with installing new servers, and some rather serious changes to trickles/credits. Including shutting down the BBC CCE project.
As this is being done by the project people, they already know about it.

I'll post something though.



ID: 37587 · Report as offensive     Reply Quote
old_user573726
Avatar

Send message
Joined: 26 Jun 09
Posts: 1
Credit: 455,242
RAC: 0
Message 37593 - Posted: 31 Jul 2009, 14:12:32 UTC

Four days since an update of user statistics on CPDN. I'm tuning a new workstation/cruncher. The trickles (particularly after a new workunit begins) are a handy quantifier of a new system's performance. So I miss the updates.

I have been comparing XP 64-bit vs. XP 32-bit on a new machine, as well as Hyperthreading on vs. off. So far, some surprising results.

As soon as CPDN starts updating user statistics again, I can see some more surprises. :>

See, it is not always about credits or RAC. Quite often, it is about determining the proper setup for a new or modified computer. The consistency of the workunits on CPDN makes it an ideal testing environment. Side effect of a lot of science getting done? Bonus for everyone.

Bob Mahoney
ID: 37593 · Report as offensive     Reply Quote
ProfileIain Inglis

Send message
Joined: 9 Jan 07
Posts: 467
Credit: 14,549,176
RAC: 317
Message 37595 - Posted: 31 Jul 2009, 15:40:45 UTC - in response to Message 37593.  

Quite often, it is about determining the proper setup for a new or modified computer.

Yes indeed: a lot of people do this from time to time.

There's a news announcement about trickle delays (here) while a new server is commissioned and the credit processing is re-worked. They'll be back eventually.
ID: 37595 · Report as offensive     Reply Quote
old_user294426

Send message
Joined: 20 Feb 06
Posts: 158
Credit: 1,251,176
RAC: 0
Message 37606 - Posted: 1 Aug 2009, 17:53:52 UTC - in response to Message 37593.  
Last modified: 1 Aug 2009, 18:00:15 UTC

Four days since an update of user statistics on CPDN. I'm tuning a new workstation/cruncher. The trickles (particularly after a new workunit begins) are a handy quantifier of a new system's performance. So I miss the updates.

I have been comparing XP 64-bit vs. XP 32-bit on a new machine, as well as Hyperthreading on vs. off. So far, some surprising results.

As soon as CPDN starts updating user statistics again, I can see some more surprises. :>

See, it is not always about credits or RAC. Quite often, it is about determining the proper setup for a new or modified computer. The consistency of the workunits on CPDN makes it an ideal testing environment. Side effect of a lot of science getting done? Bonus for everyone.

Bob Mahoney


Bob.

Although the trickles have not been coming through with the relevant credits (whatever they may mean) to update the statistics page, the trickles are still showing on Your Account/Computer ID/tasks, so you can at least monitor the rate at which the time steps are completing.

[The problem whatever it was, happened on the morning of Monday 27th July. I reset at that time to have a clean start to test a couple of HADAM3P tasks, and my statistics page has remained a horizontal straight line ever since then!!!]

Keith
ID: 37606 · Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 13 Jul 05
Posts: 125
Credit: 11,778,421
RAC: 0
Message 37607 - Posted: 1 Aug 2009, 18:20:09 UTC - in response to Message 37595.  
Last modified: 1 Aug 2009, 18:50:07 UTC

I see the updated news announcement which provides more detailed information and truly appreciate it.

Thanks for the update. Information is a good thing.
ID: 37607 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 37609 - Posted: 1 Aug 2009, 19:51:16 UTC
Last modified: 1 Aug 2009, 19:52:23 UTC

It will be worth waiting for our credits on the stats sites for a week to avoid the terrible slowdowns on this forum and the CPDN database.

I think the old credit scripts may have scanned all the trickles we've ever sent to CPDN-Boinc, every day. This method was fine when Carl originally wrote these scripts, but as the database grew the method became unrealistic. If I've understood things correctly the new scripts will usually only scan the last month's data.
Cpdn news
ID: 37609 · Report as offensive     Reply Quote
old_user294426

Send message
Joined: 20 Feb 06
Posts: 158
Credit: 1,251,176
RAC: 0
Message 37610 - Posted: 2 Aug 2009, 5:29:27 UTC - in response to Message 37609.  

It will be worth waiting for our credits on the stats sites for a week to avoid the terrible slowdowns on this forum and the CPDN database.

I think the old credit scripts may have scanned all the trickles we've ever sent to CPDN-Boinc, every day. This method was fine when Carl originally wrote these scripts, but as the database grew the method became unrealistic. If I've understood things correctly the new scripts will usually only scan the last month's data.


Mo
It will certainly be worth waiting for if the averages make more sense after the revised script.
It is a great shame that there was not more foresight in realising the script would take so long (unless possibly, it was never expected that projects would last as long as they have done).
Keith
ID: 37610 · Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 13 Jul 05
Posts: 125
Credit: 11,778,421
RAC: 0
Message 37611 - Posted: 2 Aug 2009, 7:04:56 UTC - in response to Message 37609.  

Definitely -- no problem at all with what is getting done. I was just getting a bit antsy with the lack of information but that has been resolved now.


It will be worth waiting for our credits on the stats sites for a week to avoid the terrible slowdowns on this forum and the CPDN database.



ID: 37611 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 37613 - Posted: 2 Aug 2009, 10:40:38 UTC
Last modified: 2 Aug 2009, 10:41:04 UTC

Keith said:

'It will certainly be worth waiting for if the averages make more sense after the revised script.
It is a great shame that there was not more foresight in realising the script would take so long (unless possibly, it was never expected that projects would last as long as they have done).'

I'll ask Carl whether he's found anything in the old scripts that would cause the HadAM3P models to produce higher RAC than they should. Milo did find a typo in the old script but correcting it didn't eliminate the problem.

Milo's known for a while that the old scripts were taking far too long and took some palliative measures. But he knew that rewriting them would be a big job and it's only now when Carl is providing extra man hours that time has been available to get this done.
Cpdn news
ID: 37613 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 37616 - Posted: 2 Aug 2009, 11:36:36 UTC

I've moved a post by cwhyl and my reply to a separate thread here.
Cpdn news
ID: 37616 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 37619 - Posted: 2 Aug 2009, 21:39:24 UTC


BOINCstats shows that I got 18,008 credits overnight, so we've had an update.

ID: 37619 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 37620 - Posted: 2 Aug 2009, 22:14:35 UTC

I think Carl must have been working on the scripts on Saturday. We'll have to remind him that there's a quaint European custom called das Wochenende, le weekend, el fin de semana etc.

I think the scripts ran about 23 hours ago because this forum slowed down though not for long compared with what happened previously.


Cpdn news
ID: 37620 · Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 13 Jul 05
Posts: 125
Credit: 11,778,421
RAC: 0
Message 37632 - Posted: 4 Aug 2009, 21:05:18 UTC - in response to Message 37620.  

And then went offline overnight and now is seriously slow. Seems the urge for information exchange is something of a rare thing. There are times it seems that the 'under the hood' activity that goes on with climateprediction is an MI-5 operation -- but one with good leak control.



I think the scripts ran about 23 hours ago because this forum slowed down though not for long compared with what happened previously.



ID: 37632 · Report as offensive     Reply Quote
old_user294426

Send message
Joined: 20 Feb 06
Posts: 158
Credit: 1,251,176
RAC: 0
Message 37633 - Posted: 5 Aug 2009, 7:19:28 UTC
Last modified: 5 Aug 2009, 7:24:42 UTC

The description of this thread is at present somewhat misleading.
It should possibly read "Trickle credits not displayed on Statistics page".

All trickles normally seem to be sent and ARE DISPLAYED on the TASKS ID Details page.
And also in TRICKLES INFORMATION FOR RESULT xxxx page (slightly more up-to-date).

So, it is only the credits on trickles that are not being updated yet.

If the "average" side of the missing credits is causing all the problem and taking up so much computing time to calculate, it might be worth a serious consideration whether it is worth keeping.
Or maybe it could be considerably simplified to TOTAL DURING LAST WEEK (or MONTH) instead.

Keith
ID: 37633 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 37640 - Posted: 5 Aug 2009, 17:30:36 UTC

When the thread was started it may have been about delays in trickle displays. Now everyone's talking about credits. If nobody objects in the next few hours I'll rename the thread to 'Trickle and credit delays'.

The old credit scripts (still being used twice weekly and very slow) must be kept until the new server is installed. The new fast scripts (30 minutes per run I think) can only run on the new server. The new server can't be activated this week because this will cause be an outage. The programmers don't want an outage while so many new Progress thru Processors members, who are mostly total newbies, are joining.

The calculation of RAC isn't the reason why the old credit scripts are so slow. It's because the scripts had/have to scan so much old data. The new scripts will only scan changes in our trickle records during the past month.

We're trying to keep the News thread updated about all these things.
Cpdn news
ID: 37640 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Trickle and credit delays

©2024 cpdn.org