Message boards : News : New study going out to volunteer's machines
Message board moderation
Previous · 1 · 2 · 3 · Next
Author | Message |
---|---|
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
I would love to run some of these, but I haven't received any tasks from CP. in quite a while,If you've never used Linux, don't bother trying, it's a farce. If you know how to run Linux, VB isn't that hard to set up. Basically just create a new machine, tell it to use a linux image you can get off the net somewhere, and set a few parameters to your liking (as in how many cores and how much disk space it gets). I set it to unlimited variable disk space. Set the network mode to bridged as per this post: https://www.cpdn.org/forum_thread.php?id=9187&postid=68371#68371 Then you can control it with Boinctasks or a Boinc Manager on one of your windows machines. The Linux OS will have it's own IP address. |
Send message Joined: 6 Oct 06 Posts: 204 Credit: 7,608,986 RAC: 0 |
Most people like the Mint(Ubuntu) flavour of Linux. It is thought to be easier for Windows users to use. Popular on Climate. Les, is ever willing to help. As to instructions you will find VB concerned on the VB site. Mint forums and the site have their own instructions. You will need the 32-bit lib files. Instructions are on the Linux-concerned Boinc site. Someone will be along to tell if they are still required for these WUs. I have them from the last batch of WUs, so not too sure. |
Send message Joined: 1 Jan 07 Posts: 1061 Credit: 36,743,089 RAC: 6,177 |
32-bit libs are not needed for the latest IFS range of tasks. But they are needed for the older 'Hadley' tasks and WaH tasks, still sometimes released. Ignore then at your own risk. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
Ubuntu is most like Windows, ROFL! In Windows I don't have to use the command line every 5 minutes! I'd forgotten about this library nonsense in Linux. I find it insane Boinc can't install any required libraries itself. Seems to manage in Windows.... Knowing nothing about what I'm doing, I found these instructions on part of the Boinc website: sudo apt-get install ia32-libs libstdc++6 freeglut3 I did that, Linux told me ia32-libs wasn't available anymore and to use lib32z1 So I entered: sudo apt-get install lib32z1 libstdc++6 freeglut3 This seemed to do a lot of stuff. I then restarted Boinc. Will that do? |
Send message Joined: 15 May 09 Posts: 4540 Credit: 19,039,635 RAC: 18,944 |
Yes, it should suffice. lib32z1 will only be needed if the hadcm3s type tasks come back to Linux. It is used in zipping up the final results. I don't think it was used for the intermediate files but I can't be certain without checking. The most up to date information on this is in a sticky post in the Linux section. And as Richard has said, not needed at all fro the OIFS tasks. As to needing the command line every five minutes I see that as rather an exaggeration. I don't need to use it more than once a month at the most. I choose to use it a bit more often than that because some things are quicker when you know your way around it than going the graphical route. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
Yes, it should suffice. lib32z1 will only be needed if the hadcm3s type tasks come back to Linux. It is used in zipping up the final results. I don't think it was used for the intermediate files but I can't be certain without checking. The most up to date information on this is in a sticky post in the Linux section. And as Richard has said, not needed at all fro the OIFS tasks.The page I looked at https://boinc.berkeley.edu/wiki/Installing_on_Linux said there was something else that might be needed for some projects. I assume this isn't one of them. It says "libstdc++5 was deprecated, some projects will not work without it, like QMC." As to needing the command line every five minutes I see that as rather an exaggeration. I don't need to use it more than once a month at the most. I choose to use it a bit more often than that because some things are quicker when you know your way around it than going the graphical route.You need it every time you want to download libraries or change anything even remotely technical. What annoyed me the most was the simple act of editing Boinc's cc_config.xml and app_config.xml, something I do a lot. Apparently I don't have permission to make a change to a program I installed on my own machine. Well I got very angry and am now logged in as root, just like I've done for 30 years on Windows. Now I'm actually allowed to edit a text file in a proper GUI, just by double clicking it and opening it in Mousepad. When finding out how to do that, it was quite funny all the folk answering a similar question with something to the effect of no you mustn't you'll cause a rip in the space time continuum. And I get a really angry banner in some windows telling me I'm going to die. |
Send message Joined: 1 Jan 07 Posts: 1061 Credit: 36,743,089 RAC: 6,177 |
The page I looked at https://boinc.berkeley.edu/wiki/Installing_on_LinuxOther readers please note that "page was last edited on 24 October 2012". For this project, more recent information (including information for a wider range of more recent distros), is available in the Linux area of this message board, at: *** Running 32bit CPDN from 64bit Linux - Important *** |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
Then it should be updated, since that's what you find when you Google for it.The page I looked at https://boinc.berkeley.edu/wiki/Installing_on_LinuxOther readers please note that "page was last edited on 24 October 2012". For this project, more recent information (including information for a wider range of more recent distros), is available in the Linux area of this message board, at:Ah, that explains the 8 computation errors I got for OpenIFS43r3. I shall try the above. Anyone know how you get the clipboard to share into a VB? I turned it on (bidirectional) and it doesn't work. And does it pass cores or threads? I have 12 core 24 thread CPU, and passed it 19. It says it's wrong because 19 is higher than 12. Maybe I'll just pass it all of them and tell Boinc not to use them all. Also, does anyone know how to stop and start the Boinc client in Linux? In Windows I use the system tray icon, since there is no full exit in the manager window. But Linux has no system tray, I cannot see what's running in the background. To close it I have to go into task manager and terminate (not kill) the process and it shuts down. But I have no idea how to start it again, at the moment I'm rebooting the (virtual) machine! |
Send message Joined: 12 Apr 21 Posts: 317 Credit: 14,925,468 RAC: 12,903 |
Mr. P Hucker, That link is from a BOINC website so they'd have to update it, not any specific project. The 8 OpenIFS failures you had are unrelated to the 32-bit libraries as OIFS is a 64-bit app and doesn't require them. I think so far everyone's attempts with this latest batch has failed. The reason(s) for failures are being investigated but it's the first time that specific app is being used in production and it seems like some issues haven't been discovered in testing. I don't think you can share a clipboard between a VM and host, especially with different OSs but I'm not certain of this. Threads get passed to the VM (assuming HT is on in BIOS). VBox warns if more than 50% of threads and RAM are being assigned to it (not 100% on those numbers) but you can still do it, probably without issues most of the time. I haven't used a regular Linux VM in a while but to start BOINC I believe there should be a BOINC icon on the desktop or in the Start menu like in Windows. To shut it down I think you have to use the command line, unless there's a way I'm not aware of. In Windows I wouldn't use VBox as Windows has its own built in hypervisor, Hyper-V, which is a Type 1 as opposed to Type 2 (VBox) so resource usage is better. Better yet, I'd use WSL2 (Windows Subsystem for Linux) which is also part of Windows and uses a lot less resources than a full-bore Virtual Machine (VBox, Hyper-V, etc.). I used to use Hyper-V but once I learned of WSL2 I switched to it for Linux BOINC work. Several different Linux distributions are available on it but the default one that gets installed is Ubuntu. WSL2 also already comes with all of the necessary 32-bit libraries preinstalled for the older Hadley models. The setup is a bit different than a regular VM, I think it's a bit simpler and quicker. As for BOINC, you'd just install the BOINC client (not manager) and set things up to control it from Windows BOINC manager (as I do), and probably BoincTasks. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
That link is from a BOINC website so they'd have to update it, not any specific project.I know, but Richard has connections :-) The 8 OpenIFS failures you had are unrelated to the 32-bit libraries as OIFS is a 64-bit app and doesn't require them. I think so far everyone's attempts with this latest batch has failed. The reason(s) for failures are being investigated but it's the first time that specific app is being used in production and it seems like some issues haven't been discovered in testing.Drat, it's put my new host in the naughty corner for 24 hours. I see it managed a Baroclinic Lifecycle ok. I checked another host who also failed the same task, and he has the same problem, he's got failing unlabelled tasks, but Perturbed is ok. I assumed it was a lack of libraries on his machine too, but I guess it's the tasks's fault. I don't think you can share a clipboard between a VM and host, especially with different OSs but I'm not certain of this.The setting is there: Threads get passed to the VM (assuming HT is on in BIOS). VBox warns if more than 50% of threads and RAM are being assigned to it (not 100% on those numbers) but you can still do it, probably without issues most of the time.I do CPU and GPU work in Boinc, Sheepit, and Folding@Home. I check the 8 computers daily to see if everything is running decently. If I find a slowed down GPU, I knock a thread off the CPU program. At the moment this machine runs 19 threads in Boinc for Rosetta, while allowing me to use the computer and for it to run two GPUs on Folding@Home. So I've allowed all the threads to go to the VM, but told Boinc in the VM to use only 19 of them. As it gets CPDN tasks, I'll turn Rosetta down in Windows by the same amount. I guess I could move Rosetta and Sheepit (Boinc automatically pauses when Sheepit has work) across.... but that would introduce more problems - learning how to make Sheepit run on Linux, and also CPDN would get paused very often by Sheepit, which is probably going to break things. I think I'll just let Linux do CPDN only and adjust the Windows stuff as I get work. I haven't used a regular Linux VM in a while but to start BOINC I believe there should be a BOINC icon on the desktop or in the Start menu like in Windows. To shut it down I think you have to use the command line, unless there's a way I'm not aware of.Thanks, that's easier. I made 2 "launchers" which I assume is the Linux word for a Windows shortcut (or a Mac alias). One runs /etc/init.d/boinc-client start and the other with stop at the end instead. In Windows I wouldn't use VBox as Windows has its own built in hypervisor, Hyper-V, which is a Type 1 as opposed to Type 2 (VBox) so resource usage is better. Better yet, I'd use WSL2 (Windows Subsystem for Linux) which is also part of Windows and uses a lot less resources than a full-bore Virtual Machine (VBox, Hyper-V, etc.). I used to use Hyper-V but once I learned of WSL2 I switched to it for Linux BOINC work. Several different Linux distributions are available on it but the default one that gets installed is Ubuntu. WSL2 also already comes with all of the necessary 32-bit libraries preinstalled for the older Hadley models. The setup is a bit different than a regular VM, I think it's a bit simpler and quicker. As for BOINC, you'd just install the BOINC client (not manager) and set things up to control it from Windows BOINC manager (as I do), and probably BoincTasks.Since I'm familiar with virtualbox I'll stick with it. |
Send message Joined: 12 Apr 21 Posts: 317 Credit: 14,925,468 RAC: 12,903 |
Mr. P Hucker, So it looks like the Shared Clipboard feature requires installation of something called Guest Additions. It's installed from within the guest OS and is available for most common OSs (including Ubuntu 20.04). It adds drivers and system apps to make the guest OS work better. I believe the process to install it is similar to installing a guest OS but it's done from within the guest OS and it's much quicker, it's been awhile since I've done it. OIFS tasks are most prone to failure if you push the RAM too much. With the amount of RAM you have on your VM, I'd probably say no more than 9 concurrent currently available OIFS tasks. The desired failure rate is <5% so if you see a higher rate, try reducing the number of concurrent tasks. For long term usage I'd still suggest taking the time at some point and getting BOINC on WSL2 set up. Resource usage of WSL2 is much less compared to regular VMs. Once it's running, you can close the PowerShell window and it'll run in the background, you won't even see it. It recently became Generally Available and Microsoft simplified the installation significantly. |
Send message Joined: 29 Oct 17 Posts: 1049 Credit: 16,578,380 RAC: 15,009 |
WSL2 is considerably easier to set up, use and manage. WSL can see the Windows filesystems without any extra installation of 'guest additions'. Programs like File Explorer can see both Windows & WSL files without any extra mounting of filesystems. Other programs like browsers, VS Code etc can all load WSL files from Windows. Unless you really do need a VM environment, I'd suggest trying WSL first. There's less linux to learn with WSL too; getting boinc running is just a couple of linux commands. I hate to say it, but MS have done a nice integration job with WSL (and I have never liked Windows). The only WSL gotcha as far as boinc is concerned is make sure the WSL2 install of boinc uses a different port number (e.g. 31414 instead of the default 31416) if you also have boinc under Windows. Otherwise boincmgr etc can get confused. And be careful not to overcommit system resources if running boinc under WSL & Windows. Mr. P Hucker, --- CPDN Visiting Scientist |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
So it looks like the Shared Clipboard feature requires installation of something called Guest Additions. It's installed from within the guest OS and is available for most common OSs (including Ubuntu 20.04). It adds drivers and system apps to make the guest OS work better. I believe the process to install it is similar to installing a guest OS but it's done from within the guest OS and it's much quicker, it's been awhile since I've done it.Wow, to install that I needed to click on a VB menu, which wasn't there! I had to find a Youtube video explaining how to get out of "stretch mode" to get the menus back, as for some insane reason stretch mode (which I need to get a decent resolution) makes the menus go away. Typical buggy Oracle crap. Their server grade database software is rubbish too, the NHS tried to use it once. Then I find the instructions to install it assumes Ubuntu autoruns CDs. Well mine doesn't, and I have no idea what to run, all I can see are Windows executables. There's something called autorun.sh which I assume is Ubuntu's equivalent of autorun.inf, but it just blindly opens in a text editor! Then I remember about the silly ./ nonsense to tell Linux to look right in front of it in the current folder. Oh dear, download a library to make it run, find that on Google since it doesn't get it itself, now it works! Ha! Finally. Also it fixed the bug with the menus disappearing. OIFS tasks are most prone to failure if you push the RAM too much. With the amount of RAM you have on your VM, I'd probably say no more than 9 concurrent currently available OIFS tasks. The desired failure rate is <5% so if you see a higher rate, try reducing the number of concurrent tasks.Boinc is set to use 80% of the RAM, so I fail to see how it can be "pushed". For long term usage I'd still suggest taking the time at some point and getting BOINC on WSL2 set up. Resource usage of WSL2 is much less compared to regular VMs. Once it's running, you can close the PowerShell window and it'll run in the background, you won't even see it. It recently became Generally Available and Microsoft simplified the installation significantly.Since I've started with VB I might aswell continue instead of starting from scratch and learning new stuff. But yes, as per my monologue above, don't use VB! It's rubbish! It came with Boinc so I was vaguely familiar with it, and didn't think anything written by MS would be any good. |
Send message Joined: 12 Apr 21 Posts: 317 Credit: 14,925,468 RAC: 12,903 |
Could we please have the client notifications removed? The last 2 have been pretty big comparatively, and both are still showing up despite the fact that all tasks have been snapped up and majority have already been competed. In the future, I'd propose that they should be removed no later than when all of the tasks have been snapped up. I'd also propose that they be made shorter: Name of study, RAM requirement, # of CPUs, and Run Time. The rest of the info, like study description and more technical data be provided in the forum post with a link in the client notification. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
Could we please have the client notifications removed? The last 2 have been pretty big comparatively, and both are still showing up despite the fact that all tasks have been snapped up and majority have already been competed.What on earth do you mean? This is analogous to you asking if I can delete the email I sent you. Once you have a notification and have read it, it's not there anymore. You're in control of the notifications you've already received. How could CPDN possibly delete it from your computer? I also completely disagree in the strongest possible terms about removing details. I want to see them in front of me, not have to follow a link. There's no harm in having them there in front of you. |
Send message Joined: 29 Oct 17 Posts: 1049 Credit: 16,578,380 RAC: 15,009 |
The boinc client does not manage memory very well for large memory tasks like OpenIFS. It also does not check the memory_bound value set in the task in deciding whether to start a task or not, only the number of available CPUs. So what can happen is that the client will start more tasks than the machine has memory for, regardless of what %age you have set. Memory is only checked once the task is running, which for large memory tasks is not a good idea. LHC has the same problem with their large memory tasks and had to work around it by setting server controls to limit the no. of tasks going out. CPDN are going to do the same.OIFS tasks are most prone to failure if you push the RAM too much. With the amount of RAM you have on your VM, I'd probably say no more than 9 concurrent currently available OIFS tasks. The desired failure rate is <5% so if you see a higher rate, try reducing the number of concurrent tasks.Boinc is set to use 80% of the RAM, so I fail to see how it can be "pushed". There's discussion about this in some of the other threads (if you can find it!). This is why lot of folk are using app_config.xml files to give a better level of control. Some people also knowing over-provisioning memory hoping to squeeze in as many tasks as possible, but I wouldn't recommend it. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
The boinc client does not manage memory very well for large memory tasks like OpenIFS. It also does not check the memory_bound value set in the task in deciding whether to start a task or not, only the number of available CPUs. So what can happen is that the client will start more tasks than the machine has memory for, regardless of what %age you have set. Memory is only checked once the task is running, which for large memory tasks is not a good idea. LHC has the same problem with their large memory tasks and had to work around it by setting server controls to limit the no. of tasks going out. CPDN are going to do the same.Yes I've seen this in LHC aswell, it kept starting programs then immediately suspending them, and again and again. What server controls? Do you mean the options given to the user? I prefer to do it locally, especially as I have 8 machines which are all different. I limited the max number allowed to run in app_config. They have a few different programs, so I limited the memory hungry one, but their server refuses to give me a selection of different types. If I select more than one in my settings, I just get the one they prefer you to do (or the one which gets in the server send buffer most). This sounds like a major Boinc bug - surely it should be checking to make sure there's enough RAM before starting a task! Or else what's the point in it having a "use no more than x% of RAM" option?! Has it been raised in their github discussion page? |
Send message Joined: 29 Oct 17 Posts: 1049 Credit: 16,578,380 RAC: 15,009 |
Could we please have the client notifications removed? The last 2 have been pretty big comparatively, and both are still showing up despite the fact that all tasks have been snapped up and majority have already been competed.This was discussed at length both on the forums and internally. The general consensus on the forums was people wanted more information about what they were running, up front before the tasks went out. CPDN set up notices in response. The information really does need to go out as a client notification as it's the best way to reach volunteers. I don't know how many read the forums, but I'm sure it's a lot less than are running tasks. And information on the forums is not easy to find. The information appears in a News forum post so it gets picked up by the RSS feed which sends it as a client notice. The technical information is what people asked for. I think a paragraph on the scientific side, as for the last expt is fine. Some of the other projects I subscribe to put out multi-lingual notices which are significantly longer than CPDN put out. If you want to delete them, it's straightforward. Notices from all projects are stored in /var/lib/boinc/notices. Could set up a cron that deletes any notice files in there after a day or two? --- CPDN Visiting Scientist |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
If you want to delete them, it's straightforward. Notices from all projects are stored in /var/lib/boinc/notices. Could set up a cron that deletes any notice files in there after a day or two?I use Boinctasks, which has a red X next to each notice to delete what you've seen. People actually use the Boinc Manager? Isn't that just a basic thing to get you up and running? It's not really a proper interface, especially for someone running 4 machines like the person complaining is. It can't even show two machines at once, and doesn't group queues of 100 tasks onto one line. Also it's in black and white! |
Send message Joined: 12 Apr 21 Posts: 317 Credit: 14,925,468 RAC: 12,903 |
Glenn, Mr. P Hucker, Just to clarify, I was not asking to remove notifications, just to modify them. I agree that they're helpful for CPDN as work is not constant so it's helpful to know that work appeared and one can make adjustments to one's client. It's also helpful to know some technical details, such as RAM and # of CPUs (in future presumably it'll vary) so one can make adjustments. I follow the boards closely and did not get the impression that users want lengthy notifications that one has to sometimes scroll to read on a standard sized Manager. The modification I'm proposing is a minimal notification with essential info (name of study, RAM, # of CPUs, Run Tiime) which is actionable. There's always a "More" link at the end of notifications that users who want to read additional, non-essential info, can click and they'll be taken directly to the post or article, they don't have to search the boards. I think World Community Grid does notifications well, one-liners with essential info and a link. The very lengthy ones of other projects is not something to aspire to, I'd propose. As for how long notifications stick around - it is up to the project, as far as I can tell. I don't think one can just delete contents of the 'notices' folder because they get repopulated every time the client contacts the project. Also, CPDN has it set that the client will poll for notifications every 3 days regardless. Last 10 notices per project, I believe, will show in the feed so things can get cluttered very quickly if multiple projects let things sit around. I suspect that as the amount of notices grows, other users will start saying something, I'm just trying to foresee and prevent that too. I'm just asking that notices are removed no later than when the tasks are all snapped up because at that point it's pretty much obsolete. Perhaps there's a way to automate their removal, Richard may know, in which case I'd suggest 3 days. By then all tasks are usually snapped up. As for LHC server limit control - it doesn't seem to be working as I've never seen any evidence of these kinds of limits. Unless I control things with app_config, BOINC will download many tasks and will fill all available threads with tasks, even with ATLAS, and it'll really slow my PC down. I don't have 4 machines, only 2 with a WSL2 instance on each, so it looks like 4. BOINC Manager can control multiple clients, just not from the same screen. Switching between clients is simple and isn't a big deal with my setup. I wouldn't be surprised that a majority of users use BOINC Manager as it's the standard manager and it works well enough for most cases. |
©2024 cpdn.org