Hi Les,

Thanks for taking the time and effort into putting that behind-the-scenes exposition together, it certainly makes clear what is (and is not) going on at CPDN. My sympathies go out to you in more ways than you know...please know that my comments below are not directed at you personally.

Time for a story...

When your IT department makes a call to my company because they are looking into upgrading their server racks, or your service has gone down, my face would be one of the first you would encounter. I know what it is to be the guy behind whom an entire fumbling army was hiding.

...end of story.

In the context of CPDN/BOINC your in-depth explanation is certainly appreciated. However, back in the real world, burdening my customers with the minutia, politics, workloads, incompetence, nepotism, in-fighting, and general clumsiness that lead them to their frustrations at my company's inability to meet their needs or deadlines was never an option. I got paid to solve problems for everything from making sure there were the right screws on the job site to attach your equipment to the racks with to making sure everything was up and running when we signed-off on your fiber loop. I got paid to make things happen, I got paid to make things right. So you'll have to forgive my lack of sympathy with the powers-that-be at CPDN when it comes to resolving user issues! :)

("Jimbo, you're not gonna bottom-line them, are you?" "Yeah, I'm gonna bottom-line them.") So, here's the bottom-line...it appears to me that CPDN management has not quite embraced wholly the idea that the key players on their project are their volunteer crunchers...not their programmers, not their scientists, not their professors, not even the folks who are funding the project...crazy, huh?! No crunchers = no data = no project. Kind of like my job, no customers = no business = no job for Jimbo. Yeah, that's the bottom line rule. I didn't come up with it, it's just the way things are...

So, as I see it, CPDN is in the hemorrhage-control-mode slope of an equalization cycle, whether they're aware of it or not. Too many data cycles to process leading to both a backlog in processing and a drop-off of user participation. When the load gets down to manageable levels they'll (possibly) be able (or willing) to free resources to solve user issues (if the project is still around). The project will find its own equilibrium.

Well, I feel better now, my vacation from crunching for them is, in fact, helping to alleviate stress on their system.

Best!
:)

P.S. I ran an all-volunteer non-profit some years ago, I know what it's like not to be "paid" for one's efforts! We've walked the same mile but in different shoes, Les. ;)

This project is something that one of the professors at the University of Oxford is trying to run.
As can be seen from a link here on our front page, he is only a small part of one department of the Uni, with no doubt, limited resources, and possibly limited interest from the board of governors.

The university's bottom line may well be: "We're here to teach. Where can we make budget cuts, as asked for by the government?"

But the credits matter has been raised yet again with the project, and will be discussed next week, taking time away from other matters. Such as the servers which have been shutting down a lot lately, and are currently not accepting data. No working servers, no data for the paying customers.

---

Backups: Here

In my BOINC manager Statistics window my CPDN credits lie on a straight line with a constant slope, contrarily to other projects (6). So I am regularly getting credits by the trickle mechanism.
Tullio

---

The hours my computer runs are erratic but the credits don't show anything untoward, at least not to a level that I could spot without doing a lot of work searching for problems and I am not rich enough to have time to do that.

Dave

This project is something that one of the professors at the University of Oxford is trying to run.
As can be seen from a link here on our front page, he is only a small part of one department of the Uni, with no doubt, limited resources, and possibly limited interest from the board of governors.

The university's bottom line may well be: "We're here to teach. Where can we make budget cuts, as asked for by the government?"

But the credits matter has been raised yet again with the project, and will be discussed next week, taking time away from other matters. Such as the servers which have been shutting down a lot lately, and are currently not accepting data. No working servers, no data for the paying customers.

Hi Les,

"Do, or do not. There is no try." -Yoda

Looks like they've built and paid for their own in-house solution...

Grid computing | University of Oxford Department of Physics:
http://www2.physics.ox.ac.uk/it-services/grid-computing

..., maybe CPDN can invite a few folks from Grid over to that meeting next week so they can add BOINC-fluency to their accreditation's list! Grid's "client-services" can then lobby for more resources to resolve university-project inefficiencies (it's that whole, "nominal upfront investments will yield huge data-and-results dividends" spiel) while CPDN makes the case for the "critical importance" of their data and results and how they're being "under-served" by current university resource priorities.

Seriously, somebody needs to sell the heck out of this to management, though. You folks have any "cheerleaders" on your team or any "allies" over at "Budget" whom you can bring into that room?! Sharks don't get to eat by waiting for some fish to swim into their mouths.

Given the choice between "waiting for the "Server Fairy" to come and leave me a present under my pillow" or "hunting down the right person on staff to come over and show me how to get those money gears turning properly", well, you know what I'd be doing..."Oil can, Nurse!"... :)

Come on! This is the fun stuff!
:)

Well Jimmy, I can just about see where you coming from but I do feel you have your ontological knickers in a twist.

The bottom line - if one wishes to call it that - is that this is a RESEARCH project with a fixed and limited budget. Any 'management' responsibility is to use the funds to maximise 'returns', and in this case, returns are climate modelling data.

The resources we are offering as volunteers ARE being used in the most efficient way possible - acquiring climate modelling data for the real users to analyse

We volunteers offer our spare computing capacity to speed up the acquisition of this data, primarily I feel, because the 'face' that CPDN presents to the world is that of a highly motivated, under resourced, world leading team that is trying to determine facts that could well alter international decision making if acquired in time. (Also one which has a very user friendly and informative interface on its website.)

I'm sure that the world doesn't see CPDN as 'struggling with data issues'. Data are getting back to the researchers and being processed in a timely fashion. (And yes, I know there is a current problem with some uploads, although I haven't had any problems myself - probably that's where all current resources are being used!)

All projects have activities peripheral to the main core. If the maximising activities need attention, good management does not use limited resources on a peripheral activity (in this case, the credit system) that contributes least to maximising returns.

We are simply volunteers, not users or even 'customers', freely giving our resources to the project. One wouldn't call someone giving money to a charity a 'user'.

As I said before, we offer facilities to enable researchers to reach conclusions more quickly. That, I would argue, is the main objective of our efforts which, incidentally, is not hindered by a lack of 'progress' on the credits front.

We obviously want an efficient use of the system we are supporting and I think the reasoning I use above shows that we are getting just that, as well as an enjoyable interface with other users and the project in general.

Unfortunately, more volunteers do not equal more income, so when the project programmers have a little spare time to look at the credit system, then fine. Until then I'm more than happy to wait. (And crunch)

David Roberts

Other projects give far more credits/hour than CPDN, others even less. In my opinion, credits are like money. The more there is the less its value.
Tullio

---

Hi David,

ontological knickers

You got me chuckling with that one! Ha! Thanks, I need a good hosing down once and a while! :)

My apologies for not being clearer in my previous post as to whom I was speaking about when I said "management", I was referring to the University's management and not CPDN's.

To further clarify my analogy, and hopefully simplify discussion here...

Corporate Management = University Management
Corporate Project = University Project (CPDN)
Primary Corporate Project Clients = CPDN Clients (Crunchers)
Generated Primary Corporate Project Client Data = CPDN Cruncher Data Results
Secondary Corporate Project Clients (i.e. say, Ad Services, etc.) = Secondary CPDN Clients (i.e. Climate Researchers, etc.)

Now let me try this from a couple of different angles...

To say that the CPDN Researchers are CPDN's Primary Clients and the Crunchers are its Secondary Clients is to argue that the Restaurant's Primary Clients are it's Food Suppliers and it's Secondary Clients are it's Walk-in Customers. What I'm saying is that this is upside-down thinking, and it's the kind of thinking that sinks projects. I stand by my "Bottom Line" comments. :)

Methinks the horse has got one final cautionary tale to tell before it passes on...

"Heaven goes by favor. If it went by merit, you would stay out and your dog would go in." - Mark Twain

Over the 32-years at my own "nickel-and-dime-you-to-death" factory I've seen plenty of worthy projects fall by the wayside and an equal number of worthless projects get the green light. When it comes to getting resources, "worthiness" is not the primary or final deciding factor in loosening the purse strings.

The horse goes on...

So you and I want to see CPDN discover that indisputable truth about Climate Change that will finally wake up the world and get humanity to finally start moving towards a sustainable and clean future. And the Researchers are frantically looking for a steady stream of data and person-power and equipment to make those vital discoveries possible. And a lone professors sits by himself, cloistered in a lonely corner of a vast university campus fretting over how to make his project thrive and, thus, save the world from imminent disaster. And all of this, all of this, falls on the deaf ears of the elder and jaded University Lord Master who is looking out across his well-trimmed lawns on this perfect, sunny day, whose mind is only concerned with the latest cost-cutting measures for the upcoming semester. And no one's attention is ever turned to the marketing folk who have been showing up everyday banging their drum trying to get someone, anyone, assembled in this vast army and effort to listen to their plan on how to make it all work. All the while, many miles away, the ocean floor temperatures continue their incessant rise, degree-by-perilous-degree, and the trillions of tons of trapped methane lying there slowly creep to their liberating boiling point, threatening the future existence of every living thing on the planet save for a few microbes...

Tapping on the microphone. "Ahem, check, check." The room comes to order. "Can I have everyone's attention please. The Project's Marketing folks have something they'd like to share..."

"Yes, it all reads like a soft disaster thriller." gasped the horse, And with a final,"Gaaaah." it expired.

Ta-ta for now!
:)

"To say that the CPDN Researchers are CPDN's Primary Clients and the Crunchers are its Secondary Clients is to argue that the Restaurant's Primary Clients are it's Food Suppliers and it's Secondary Clients are it's Walk-in Customers."

Surely the Researchers who pay the bills are the ones who map onto the walk in customers in this analogy? The crunchers are the food suppliers, even if they are not charging.

I don't know in the current climate whether or not a more enlightened approach by some of those at the university might result in a lot more funding. That may well be true but those who commission the research pay the bills at the moment and for any project, that will always be the bottom line.

[Jimmy Gondek wrote:]So you and I want to see CPDN discover that indisputable truth about Climate Change that will finally wake up the world and get humanity to finally start moving towards a sustainable and clean future. And the Researchers are frantically looking for a steady stream of data and person-power and equipment to make those vital discoveries possible.

... it might be a mistake to assume that the academics have any view about climate change at all: for them it's just another interesting problem to solve, albeit a topical one: atmospheric science, oceanic science and who-knows-what other disciplines would be worth studying absent of climate change. Indeed, I suspect that one of the disjunctions between the volunteers and the academics is precisely the question of motive. Your first sentence is true for only some of us, and your second for only some of them. Maybe our teleological knickers are in a twist as well. Ouch!

...snip...

That may well be true but those who commission the research pay the bills at the moment and for any project, that will always be the bottom line.

And here I've been thinking that CPDN, and any other Grid programs for that matter, were sitting in a quiet corner of the freeware section at the software trade show hoping that someone, anyone, just might pick up a copy of their floppy and give their wares a spin. And to get themselves a little more visibility they decided to huddle under the BOINC tent with the other Grid projects hoping to find a computer to play on. And it's not like this freeware is gonna balance the user's checkbook or straighten out their hard drive or even act as an egg timer, no, it doesn't perform any useful function for the user. It asks potential installers for a free lift down the road with the promise of a handshake for their troubles when the ride is over...hitchhikerware, if you will. And not a cent to help you fill your tank. Paraphrasing Wimpy for their motto, "I'll gladly pay you imaginary credits Tuesday for the coal I burn today." And CPDN is welching on their handshake.

Where you might be seeing fat cats in top hats with 50-dollar bills falling out of their pockets commandeering research-software yachts, I'm seeing earnest researchers on miniscule budgets who, in a million years, could never afford the upfront costs of the tens-of-thousands of computers they need to run their project's software, never-minding the person-hours and energy costs that go along with running something on those scales, standing on a wooden dock next to their rubber dinghy's with a tin cup plaintively sitting by their shoe whilst wearing a cardboard sign offering free handshakes for a lift.

And it's only gonna get tougher for Grid projects to find a ride. Laptop and desktop sales have been flat and are now falling with the advent of smart phones and tablets. The days of using a tethered contraption-in-a-box to check emails is quickly going the way of grandmama's rotary phone as folks migrate to mobile devices and texting. What's a researcher to do? I'm thinking a port to pocket devices running iOS and Android, that hibernate after 30-seconds of non-use, is not a viable future-proofing strategy for saving the planet, finding aliens and cracking encryption codes.

So, I should sleep well at night in comfort and with the confidence that the "fat cats" shouldering a few small percentages of the costs for running their projects will figure this out? I stand by my "upside down thinking" comments. :)


"I don't think these knickers are tight enough, give 'em a yank for me, will you? (hard pull) Oooh, that's about right, now. Thanks, mate!"

(Kindly read above paragraph with Eric Idle intonations)

:)

[Jimmy Gondek wrote:]So you and I want to see CPDN discover that indisputable truth about Climate Change that will finally wake up the world and get humanity to finally start moving towards a sustainable and clean future. And the Researchers are frantically looking for a steady stream of data and person-power and equipment to make those vital discoveries possible.

... it might be a mistake to assume that the academics have any view about climate change at all: for them it's just another interesting problem to solve, albeit a topical one: atmospheric science, oceanic science and who-knows-what other disciplines would be worth studying absent of climate change. Indeed, I suspect that one of the disjunctions between the volunteers and the academics is precisely the question of motive. Your first sentence is true for only some of us, and your second for only some of them. Maybe our teleological knickers are in a twist as well. Ouch!

Idle: Teleological knickers? I thought he was being farcical.
Cleese: No, he was being serious.
Gilliam: I thought he was practicing irony.
Chapman: That was never irony (chomping on pipe)...poofery, perhaps.
Palin: Dont' you mean, puffery?
Chapman: No, poofery. Clearly.
Jones: No such thing.
Chapman: Well there is now.

:)

Priorities.
When the servers are all back and running after the recurring failures of the past week, there'll be time to look at other matters.

Hi Jimmy,

I guess one way in which our analyses differ is your considering we �crunchers� as primary corporate project clients. I can only view us as a resource, as are the modelling tools (models + computers + support staff etc) and additionally, I can�t really view the researchers as secondary from any point of view.

Now I don�t feel that corporate terminology maps at all well onto descriptions of research activities but for the sake of discussion I�ll use this nomenclature.

From first principles, a research project (Start up idea) begins in the mind of a person or persons � eg climate researchers (corporation) � leading to a research proposal with well defined aims and expectations of investigations, analysis and conclusions. (potential corporate product)

Funding sources (corporate sponsor ) supply money if the research proposal is considered sound enough. (Ultimately, all of us in the case of public funding � but not just crunchers)

The data returned from modelling and it�s analysis etc., is the raison d�etre of the project. (corporate product)

Thus, in this case, as users of the generated data, the researchers themselves are primary clients of the project together with the human race that benefits from research (The sponsors?). (Almost an automorphic relationship not unlike a group of friends taking turns to cook meals� all producers & consumers)

When it come to the administration and management of the project, there are as many ways of doing this as there are management theories. In large organizations there will be a network of facilities available for the basic donkey work. (Payroll, billing etc..) But, it will probably be one or more of the research team who decides on priorities for who does what and when in using project resources.
Thus, in a very real sense, the research team is the corporate management and prioritisation of resources is often the name of the game.

In CPDN�s case, an inevitable restriction with the scope of the research proposal was always going to be computing resources. Climate modeling is computing intensive (resource) - even with super computers, timescales are intimidatingly distant.

So � Question - Can we get more resources for the project to shorten timescales?
Answer - Yes, if people in the big wide world will donate their spare computer resources.
Conclusion - Crunchers (in reality their computers) are a resource. (Happily, free of charge) QED

It is the use of product that defines a client relationship between producer and consumer, not the use of resources. For crunchers to view themselves as primary clients of CPDN is rather like pre-Galileo Christians viewing the Earth as the centre of the Universe. (Or even inhouse computers as primary clients.)

Well, that�s it from me. Far from completely definitive but perhaps there should be a separate thread for these terminological musings. As I noted above, my own view is that corporate terminology does not map well onto research project activities.

PS Crunchers are primary clients of the computing credits system but that really must be at the bottom of any priority list.

And currently, with more computers awaiting work than there are work units available, (the odd automatically regenerated or re-issued work unit), keeping people happy with credits is not really needed.

Dave

This week on the Mystery Twilight Zone Theater 4000 radio broadcast....

"Illusions Of Grandeur"
(A "morning-at-the-computer-with-a-cup-of-coffee" tale, narrated by its author.)

Prologue:

The fish tank and accessories found their way out to the curb on Tuesday for trash pickup. It's algae-stained glass and remnant few gravel bits and musky smell barely hinted at it's barely note-worthy past. Gone, too, a season prior, the wrapping paper and shiny bows in which it all arrived.

Segue:

No one to tell the tale of the rise and fall of little Billy's big birthday wish except for Billy himself, and then, only in passing, "Oh yeah, I had a fish tank once. But then all the fish died and we got rid of it."

And what of the mighty Beta who once and briefly ruled over this small and glorious kingdom? And its minions of lesser fish? And what of the tales of their glorious lives lived out amongst a few plastic sea plants and bubbling Pirate's treasure chest? Who would tell their tales? Lives once blessed with abundant resources of food, fresh air and tank cleanings? Lives which quickly fell into deprivation, starvation and, ultimately, suffocation?

Epilog:

The reflections of the young and mesmerized Billy would no longer be found upon the surface of the tank's glass, his now slightly-older face finding a new home in the glass of a shiny new computer monitor. And, Billy's parents, noticing his fleeting interest, abruptly and quietly stopped financing the upkeep of his 12th birthday gift. Unrecorded in the annals of fish history were the self-important and omnipotent bleatings of an anonymous Beta and its tank-mates of lesser fish.

The end

Music and credits


And now some scenes from next week's radio drama, "Grandeur II: Billy's Face Finds A New Home"...

Preview: Rated PG-13

The computer found it's way out to the curb on Tuesday...Billy's face found a new home on the shiny new screen of a smart phone...Recorded in the annals of research history...

Music

Be sure to join us, we'll "see" you all next week on the radio!

:)

A fair point, though lacing it with acid won't help your cause. Which is what, precisely? That CPDN should be improved? That CPDN is doomed?

Self-important and omnipotent bleatings? Physician, heal thyself.

June 16 was Bloomsday.
Probably surreal stream-of-consciousness stuff could go on the "Cafe CPDN" page. If anywhere.
None of us here are likely to be able to beat James Joyce in the genre.
Nor have time or inclination to respond in kind here.

---

Hi Chaps,

I am the sysadmin for CPDN. I am not a BOINC expert, and I do not run BOINC on any of my systems.

CPDN uses a custom credit system that calculates credit based upon your total participation in the project, rather than simply appending your recent activity to a running total.

This means that if I try to fix the credit system and get it wrong, I can drastically affect the reported credit of every user who has ever used the project.
Indeed, my first task when I joined the project was to run an existing credit rationalisation script that brought the project down. That had the result of losing credit for some users. I spent a long time determining who had lost data, and stored the information for when I get the chance to fix the problem.

In investigating this problem, it would help me greatly if you could take me through the issues as you see them.

Please appreciate though, that we have 10 years of user credit data in our database (not a very good design, I agree, but we have to work from where we find ourselves).

I cannot 'just run it through Excel' because we have billions of records, and the exported user data is many 10s of GB in size. Excel is not up to the job.

As Iain said, I need your help to identify what is going wrong. I need you to point me to clear problems, and I will then see if I can understand what is going wrong.

Bear in mind that I am not a BOINC user, so I need this spelt out in words of one syllable (I am a bear of very little brain, who has a piece of fluff in his ear)



Jonathan

---

I suspect that it is a small but significant number of users affected. The other option is that I and many others are not paying enough attention to realise we are also affected but, I do not have the time nor inclination to go through my own records of tasks completed and part completed to work out whether there is a problem or not. If there is one, it certainly isn't obvious. Sometimes there is a discrepancy between the score on the "my account" page and those on my team page but these always catch up eventually so I don't worry about it. I share your' wanting to know exactly in words of one syllable what the problem is as even having read back through some of Les's posts and others' I don't get it all. It seems that some users have lost credit from before the system was changed to run on the mirror server but I assume all more recent data is being logged, so it is just those who have lost data that have a problem. Having not programmed for about 30 years, I don't know how much work is involved in getting something to troll through the records to find out who has lost credits and how much.

My main point however is that I and I think a lot of other users do appreciate the work you are doing on this and the server issues etc.

Dave