Message boards : Number crunching : Database Sluggish -- Moved Trickle Info \'Down\'
Message board moderation
Author | Message |
---|---|
Send message Joined: 5 Aug 04 Posts: 907 Credit: 299,864 RAC: 0 |
Hi, it looks like with our new user numbers, it\'s not efficient to keep reporting trickle details on the user & host pages. You can still see the trickles for a result \"drill-down.\" |
Send message Joined: 7 Aug 04 Posts: 2187 Credit: 64,822,615 RAC: 5,275 |
So, in the past some trickles were listed on the User, Computer Summary, Result, and Trickles Information for Result. This change makes them just listed on the Trickles Information for Result? I\'m thinking that lots of people used the trickles under the User page to kind of do a quick check on how their computers were doing, any problems, whether they were home or not. Now one either needs to do Click on Your Account Click on View Results Click on the Result ID (unsure of associated host at that time) Click on Trickle # or, to see how a particular PC is doing: Click on Your Account Click on View Computers Click on Computer ID Click on Results Click on Result ID Click on Trickle # I think the trickles on the User page would be pretty handy to have. Perhaps having the trickles removed from just the Computer Summary and Result pages will improve performance enough without the removal from the User Page? |
Send message Joined: 5 Aug 04 Posts: 907 Credit: 299,864 RAC: 0 |
well actually the user page was the worst one, although I just show the latest 10 or 20 to the database it still has to get maybe 10K. perhaps they\'ll come back someday when I get a chance to optimize the tables etc, but for now they were taking 1-2 minutes per user request! |
Send message Joined: 7 Aug 04 Posts: 2187 Credit: 64,822,615 RAC: 5,275 |
I\'ll make this a sticky then since people will start noticing soon. |
Send message Joined: 19 Aug 05 Posts: 104 Credit: 1,866,495 RAC: 0 |
Already noticed, thought it was server problems at first. I miss them there but understand the server load. Can get at them easy the way that they are. When the database is optomized even having the newest 5 rather than 20 would be handy, as that would let us know which computers reported in without having to chack all units. Feels good to have a Sulpher model almost finished. Will be done in plunty of time for use in the next round of couppled models. Cheers Ray Keep on crunching Pizza@Home |
Send message Joined: 23 Feb 05 Posts: 55 Credit: 240,119 RAC: 0 |
Thanks for explaining Carl. Indeed saving a user page has been taking much longer than saving a result or trickle page! Nevertheless a loss of overview for the users, but it might be more direct to access the trickle page directly. In Internet Explorer the user can add and name the different trickle pages for models that have to be checked on trickle update. (Other browser would have possiblities alike) It takes a bit of reorganising but it might be more direct and with less paging around, once you are used to it. |
Send message Joined: 27 Jun 05 Posts: 74 Credit: 199,198 RAC: 0 |
Hi Carl, Thanks for the explanation Obviously that kind of load is unacceptable and you had to do something. It does not make sense to include expensive information on a page which may be loaded for other reasons. However with multiple computers, not all in the same place, it was very useful to have all recent trickles on the same page. Three useful alternatives would be 1) Have a separate \'recent trickles\' page that users could click on if they wanted it. The advantage to the project is that the heavy load is only produced when the user specifically wanted it, no for example when the user only went to the page to see their latest stats or whatever. 2) Introduce a \'last successful trickle\' date column to the page that lists computers (not quite the same as the existing \'last contact\' which always might be for some kind of err, manual update, even machine reboot sometimes) 3) Introduce a \'last successful trickle\' column on the page that lists a user\'s results. The advantage of any of these, from the viewpoint of a user with many machines, is that one page will give a snap overview of whether any of the machines / results needs some attention. Perhaps with option 1 you could extract a derived table once an hour with trickle info and userid in it, with an index on userid. The query would be an hour old but that is fine for a quick-look facility. With option 1, you could also make the script run at a lower priority than the rest of the web pages. This would mean two things, firstly, users asking that question would not slug the rest of the website, and secondly if those pages got overloaded the slowness of their response would deter their use for frivolous queries. The page would need to contain an explanation that \"at busy times this page may be slow\", and \"please do not repeatedly reload - this will only make it worse\". (One way I\'\'ve done this is for the page to call a separate script via a \'nice\' command -- it got round a similar problem we had of one particular page slugging a whole website) Any one of these options, or any other way of doing a quick \"is everyone happy\" check one to three times a day would be a welcome replacement. With 8 active machines I won\'t have the time to check on every machine every day without some way of doing it from a single bookmarked page. And to be clear, this is not a complaint, it\'s a request. You clearly did the right thing in the short term to de-slug the db. It is much better to have most of the old functionality running at a workable speed that have it all running as slow as it has been lately. The performance boost since your change is both obvious and welcome. |
Send message Joined: 27 Jun 05 Posts: 74 Credit: 199,198 RAC: 0 |
In Internet Explorer the user can add and name the different trickle pages for models that have to be checked on trickle update. hi again Kilcock yes, in IE you set the trickle pages as \'favorites\'; in Firefox you bookmark them - same thing really. It still means one bookmark/favorite for each model - with 8 boxes and 11 active model that still leaves 11 bookmarks to check. An even better way in the firefox browser is to open each of these pages in separate tabs, then use the firefox facility to save the entire set of tabs as a single bookmark. This means that one click gets all the pages, and you simply scan from one tab to the next to look at them all. Firefox is free and open source and available for all the platforms running CPDN, more info & downloads from www.firefox.com It is still 12 clicks to see all the info, so I\'d still prefer some way of getting it all onto one page. Also it means that when I click on the one bookmark it sends 11 queries to the database all together - a well designed single page might be less of a load on the database River~~ edits: speeling & typos ;-) |
Send message Joined: 5 Feb 05 Posts: 28 Credit: 1,846,059 RAC: 0 |
In Internet Explorer the user can add and name the different trickle pages for models that have to be checked on trickle update. I do agree with this fullheartedly! It gets especially annoying when you use computers with multiple CPUs, each CPU running its own workunit. For each CPU you need at least 4 clicks to find out if it reported in or not: 1. Your Account 2. Results 3. Result ID 4. Trickle # All of this times the number of CPUs you use. And this is the best-case scenario! |
Send message Joined: 23 Oct 05 Posts: 22 Credit: 526,746 RAC: 0 |
A \"recent trickles\" page would indeed be very welcome, if possible to do without excess db load. The way it is now is a bit annoying in the long run. |
Send message Joined: 5 Feb 05 Posts: 465 Credit: 1,914,189 RAC: 0 |
Suggestion... The computer list shows the last time the computer talked to the server, how about each WU also show the last time it was trickled on that page? At least we would quickly know if trickles are coming in. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
All these suggestions require a commodity that the two programers don\'t have: time. And the support staff have all moved on to other things. |
Send message Joined: 31 Oct 04 Posts: 336 Credit: 3,316,482 RAC: 0 |
Question is, if the database even needs all those trickles. If I understood the concept of the 6-hourly fixup run right, only the latest / highest trickle for each active model is relevant. Models in over/success state probably wouldn\'t need even that. The information of this latest trickle could be in the table of results, so it wouldn\'t even be an extra record in an extra table. This would be a temporary redundancy - but only until the next fixup collected the trickles, so it would save a lot of queries and db space on the long run. |
Send message Joined: 14 Aug 04 Posts: 37 Credit: 276,676 RAC: 0 |
Hi, Perhaps a script of some kind can be added so as to update credit displays (from Trickles) every, say, 8 hours, so as to better handle the server load. As it is , my trickles RAC is riseing but on the stats display pages it is falling. K. |
Send message Joined: 5 Aug 04 Posts: 390 Credit: 2,475,242 RAC: 0 |
Perhaps a script of some kind can be added so as to update credit displays (from Trickles) every, say, 8 hours, so as to better handle the server load.As I understand, millions of trickles are to \'blame\' databse sluggish so recounting them several times a day is not an option (for the moment). RAC never did good job for CPDN... <i>phpBB forum for CPDN, all are </i><a href="http://www.climateprediction.net/board">invited</a> |
©2024 cpdn.org