For the first time in a very long time, it looks like the daily trickle credits were missed. Didn\'t see any announcement (though typically in Climate that sort of thing, if it does appear shows up buried in a thread somewhere it seems).

Does anyone from the project side know what happened -- and if it is a harbinger of multiple days of missed trickle credits?

---

If an event is known about it\'ll be added to the news thread here. It\'s a good idea to subscribe to that thread.

However, the failure of a trickle run is not normally a planned event and news of it will emerge as people spot something awry, as you have ...

Yesterday 19 Sep the server web pages for cpdn were down again for a few hours. That\'s the \'data-driven web pages\' server. The boinc software was upgraded last Friday for all 4 cpdn projects and the job still isn\'t finished on the BBC project. However, uploads, downloads and trickles have not been affected by the upgrade.

I imagine that when the cpdn web pages server catches up on the backlog, you\'ll get the credits. In any case, every time a trickle is received, the server recalculates the credits for that model from when it began. This method should ensure that every trickle received gets its credits.

So I don\'t think your credits will have gone missing for ever.

---

Cpdn news

OK -- I\'ve subscribed to that thread. I figured to report what I observed in the message boards -- looks like that worked...

If an event is known about it\'ll be added to the news thread here. It\'s a good idea to subscribe to that thread.

However, the failure of a trickle run is not normally a planned event and news of it will emerge as people spot something awry, as you have ...

---

Thanks for the update. I figured the credits hadn\'t got lost, just that the updates were not reflected in the daily update to the statistics pages. I will be on the lookout this evening for the updates.... and will whine here if they don\'t show the updated trickle information (smile).

Yesterday 19 Sep the server web pages for cpdn were down again for a few hours. That\'s the \'data-driven web pages\' server. The boinc software was upgraded last Friday for all 4 cpdn projects and the job still isn\'t finished on the BBC project. However, uploads, downloads and trickles have not been affected by the upgrade.

I imagine that when the cpdn web pages server catches up on the backlog, you\'ll get the credits. In any case, every time a trickle is received, the server recalculates the credits for that model from when it began. This method should ensure that every trickle received gets its credits.

So I don\'t think your credits will have gone missing for ever.

---

Looks like, notwithstanding that the outage was only a couple of hours yesterday, data the trickle update statistics outage is still in effect for a second day.

See, I told you I\'d be back this evening to whine. (smile).

---

Looks like, notwithstanding that the outage was only a couple of hours yesterday, data the trickle update statistics outage is still in effect for a second day.

See, I told you I\'d be back this evening to whine. (smile).

And hey I have 3 kinds of cheese too!.. (VBG)

Anybody have decent crackers?

Seriously though, having been through this before I don\'t mind getting a bunch of credits in a lump sum. Just so long as the data doesn\'t get lost.

Arion

---

i confirm my credits have not been updated for 2 days either.

Brazzmonkey, your HADCM model hasn\'t produced enough trickles to check that all of them are there on your webpage, but your slab HADSM model

http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=6825916
shows all its trickles correctly recorded.

On Wednesday 19 Sep, the only server down was the one for data-driven web pages. This is the server that provides this forum and lets us see our account and results pages. Cpdn uploads, downloads and trickles were not affected at any time as far as I know.

Milo is still working on the servers. The BBC data-driven web pages server has been down for a whole week.

There is usually a delay of one day and sometimes two days between results (trickles and file uploads) being received by the cpdn server, the credits being awarded, and the stats sites seeing these credits and displaying them.

Nobody\'s trickles should be missing on their results pages. And as I said before, every time the server receives a trickle, it recalculates ALL the credits for ALL the trickles received for that model from when the model started. So if all your trickles are recorded, you will get all your credits.

We have (very very rarely) seen cases where trickles or file uploads have apparently been lost. But I don\'t think we\'ve ever seen a case where trickles received by the server have not generated the correct number of credits.

So we will need to be patient.

Barry, I couldn\'t check your current model trickles because there are so many computers and so many models..... And I couldn\'t whether your recent trickles are all present and correct, Arion, because your computers are hidden.

I\'ve been copying my News thread posts about server outages to Neil Munday\'s stats site forum and to the project news section of Willy\'s BoincStats forum.

---

Cpdn news

Mo,

I monitor and log activity on the single host I have connected to CPDN (host 518112).

I can confirm that all appropriate trickles have been uploaded and are correctly shown on the results pages - no problem there.

However, the BOINC credits are now two days out of date - the translation process from trickles to credits (which normally runs once every 24 hours, at around 4am BST) has failed to run on each of the last two nights. I\'m not worried - the science is there, and the credits will catch up eventually - but it is evidence that at least one CPDN server process has stalled, and perhaps needs attention alongside Milo\'s work on the BBC servers.

@mo.v
thanks for making things clear. i don\'t care about credits anyhow...

Mo,

I monitor and log activity on the single host I have connected to CPDN (host 518112).

I can confirm that all appropriate trickles have been uploaded and are correctly shown on the results pages - no problem there.

However, the BOINC credits are now two days out of date - the translation process from trickles to credits (which normally runs once every 24 hours, at around 4am BST) has failed to run on each of the last two nights. I\'m not worried - the science is there, and the credits will catch up eventually - but it is evidence that at least one CPDN server process has stalled, and perhaps needs attention alongside Milo\'s work on the BBC servers.

Hi,

I have the same effects on my host (host 506953). Five trickles and no credits ???

jan

I\'ve just sent Milo a message. He\'s just now got the BBC data server and forum up and running again, so if something else is holding cpdn credits up, it has probably had to wait until the BBC crunchers got their results pages and forum back.

I\'ll post back as soon as possible.

---

Cpdn news

I\'ve just posted this in the News thread (top of the Number Crunching section):

All ClimatePrediction servers for all projects are now running again.

However, Milo says there was a problem with the database yesterday 20 Sep - a mysterious crash the cause of which could not be determined. This was entirely separate from the outage to upgrade the web pages, forums &c. It is likely that the credit generation script didn\'t run for that reason.

Credits should be up again by tomorrow; the script will run overnight. It may be a bit slow to catch up, as usual after an outage. So members may not receive their outstanding credits until Sunday or Monday.

A new file server will be installed in Oxford soon, probably within a week or two, easing the pressure on disk space and of course on Milo and Tolu.

---

Cpdn news

OK -- for me, the \'non-posting\' was pretty obvious -- I\'d been running between 2K and 3K daily and then when I checked on Wednesday -- 0. I temporarily suspended climate on *some* of my farm of workstations, and when I checked on Thursday, there was still no change from Tuesday. What I have done since then as sort of a \'test case\' is \'unsuspend\' a few workstations (did that yesterday afternoon). I figure to wait to see a change in the credits (typically the daily update shows up in my time zone around 10PM). Once I see it is \'moving\' again, I\'ll bring my other Climate PC\'s back into the fray. During the interim, SETI, Spinhenge, Rosetta, World Grid, and Einstein get the extra cycles.

I\'ve just sent Milo a message. He\'s just now got the BBC data server and forum up and running again, so if something else is holding cpdn credits up, it has probably had to wait until the BBC crunchers got their results pages and forum back.

I\'ll post back as soon as possible.

---

I\'ve just posted this in the News thread (top of the Number Crunching section):

All ClimatePrediction servers for all projects are now running again.

However, Milo says there was a problem with the database yesterday 20 Sep - a mysterious crash the cause of which could not be determined. This was entirely separate from the outage to upgrade the web pages, forums &c. It is likely that the credit generation script didn\'t run for that reason.

Credits should be up again by tomorrow; the script will run overnight. It may be a bit slow to catch up, as usual after an outage. So members may not receive their outstanding credits until Sunday or Monday.

A new file server will be installed in Oxford soon, probably within a week or two, easing the pressure on disk space and of course on Milo and Tolu.

Thanks mo and milo for your engagement.

jan

Barry, I couldn\'t check your current model trickles because there are so many computers and so many models..... And I couldn\'t whether your recent trickles are all present and correct, Arion, because your computers are hidden.

I\'m not worried about my credits and I\'ve seen my trickles going up. As for my computers being hidden I have to do that right now because of some testing I\'m doing. I\'ll be releasing them as soon as I am able to. We went through this not long ago for about a week or so and got all our credits over a 2 day period. I\'m not concerned as long as the data is okay.

Arion

---

Trickle-to-credit script seems to have run correctly this morning. Thanks, Milo.

It was fortunate that you guys all reported your lack of recent credits because it was only when I told Milo about this that he realised the credits script must have been affected by the server crash. That\'s why none of this was explained in the news thread earlier - nobody had realised what had happened.

---

Cpdn news

It was fortunate that you guys all reported your lack of recent credits because it was only when I told Milo about this that he realised the credits script must have been affected by the server crash. That\'s why none of this was explained in the news thread earlier - nobody had realised what had happened.

In a discussion about a similar but unrelated problem in another project, I coined the phrase \"distributed wisdom\", as being something that project administrators get for free when they set up a \'distributed computing\' project. Sometimes we users can spot a problem quite early in its formation, and even analyse quite closely which part of the system needs attention. In the best of all possible worlds, that analysis happens by discussion among users in a different time-zone, and the solution is available on a plate before the project administrators even wake up in their morning!

It\'s up to project administrators how much advantage they want to take of the available \'distributed wisdom\'. Sometimes the signal-to-noise ratio is rather low: as Milo already knows, reading every thread in every bulletin board would waste more time than it saves.

CPDN is fortunate in that the forum moderators take an active interest in the technical aspects of the project, and have a direct line of communication with the administrative staff. Other project moderators are sometimes more involved with the \'social\' aspects of forum management, and don\'t always have such a close working relationship with the technical staff. Then, technical glitches don\'t always get attended to in as timely a fashion as this one has been.