climateprediction.net (CPDN) home page
Thread '*** Running 32bit CPDN from 64bit Linux - Discussion ***'

Thread '*** Running 32bit CPDN from 64bit Linux - Discussion ***'

Questions and Answers : Unix/Linux : *** Running 32bit CPDN from 64bit Linux - Discussion ***
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 19 · Next

AuthorMessage
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 52944 - Posted: 25 Nov 2015, 13:04:38 UTC - in response to Message 52943.  

To expand on what Dave said. So long as the UK Met Office can take advantage of the fact that the 32 bit models run on 64 bit Windows right out of the box, and the fact that there are so many more Windows computers out there for them to run the models on, it is unlikely that they will ever go to the time and expense of producing a 64 bit Linux (or Windows or Mac) version.

ID: 52944 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 52945 - Posted: 25 Nov 2015, 15:56:15 UTC - in response to Message 52942.  

It's not terribly hard to configure Linux with 32 bit support (at least on Ubuntu and related distros), therefore able to run CPDN tasks. But apparently it's difficult enough to cause big problems running models on Linux.


Well, it's just that there are so many people who just attach and forget it. They never look at their webpage results output, or the message log in boinc. They just assume that if they attach to a project, it's going to work. Obviously lots and lots of people are like this, and on many projects, it works that way. Here...not so much.

No secret nearly all Linux applications these days are 64 bit, which leaves me curious why CPDN Linux models cling to the older 32 bit design. Sure seems you are right, it's not at all using the power of Linux computers to their potential on behalf of CPDN.


I don't believe it is a huge deal to compile for 64bit vs. 32bit. There is not a whole rewrite of the code needed, but perhaps some changes might need to be made. I think it is more to do with alpha/beta testing and validation time and effort. The results output for the same task may not be bit for bit identical, but then it is also not for Windows vs. Linux with the different platforms and optimizations, or even AMD and Intel processors of various generations. And we've been running that way for many years with any differences in output about equivalent to initial condition perturbations in ensemble output.
ID: 52945 · Report as offensive     Reply Quote
jrapdx

Send message
Joined: 4 Jul 15
Posts: 63
Credit: 3,223,760
RAC: 0
Message 52948 - Posted: 26 Nov 2015, 11:19:58 UTC - in response to Message 52945.  

Honestly I can't understand the "start and forget" attitude some users evidently have. I tend to watch my systems like a hawk, and wouldn't dream of doing otherwise. We're talking about computing after all, things notoriously go wrong all the time. Especially with systems like Linux that aren't "off the shelf" like Windows, rather surprising that anyone geek-minded enough to bother with it would neglect how their machines are handling CPDN tasks.

A while back, out of curiosity I counted the number of Linux installations there were among the "top 100" CPDN computers. There were a lot more than I expected, 14/100 if memory serves correctly. It struck me as quite a significant chunk of the computer resources serving CPDN, thus a reason to try to keep Linux systems crunching for the project's benefit.

If converting to 64 bit applications would reduce wasted effort and speed up results, it seems likely to be worth it given Linux is hardly a rarity out here.
ID: 52948 · Report as offensive     Reply Quote
Profiletullio

Send message
Joined: 6 Aug 04
Posts: 264
Credit: 965,476
RAC: 0
Message 52949 - Posted: 26 Nov 2015, 12:23:01 UTC - in response to Message 52948.  
Last modified: 26 Nov 2015, 12:23:48 UTC

Of my six BOINC projects only ATLAS@home and CMS-dev require a 64-bit Linux. All others use 32-bit Linux apps. My two Linux boxes run 32-bit OS, although one has a 64-bit CPU. My 64-bit Windows 10 CPU runs three 64-bit Linux Virtual Machines, one 64-bit Solaris Virtual machine and one 32-bit Virtual Machine, vLHC@home. CERN has said it will update vLHC@home but so far it has not happened. The CERN 2015 Challenge, not a BOINC project. uses 64-bit Linux Virtual Machine. I am running a CPDN task on this SuSE Linux 32-bit host, plus vlHC@home. SETI@home runs 32-bit on my other Linux box, together with Einstein@home. Both SETI@home and Einstein@home run on the 64-bit Windows 10 PC.
Tullio
ID: 52949 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 53029 - Posted: 5 Dec 2015, 12:16:36 UTC

For completeness, I hadn't bothered looking at getting graphics working since upgrading to latest ubuntu as they weren't available on the tasks I was running anyway. Now there are tasks with graphics, on my box,

sudo apt-get install libjpeg62:i386

Got them working.
ID: 53029 · Report as offensive     Reply Quote
Wes

Send message
Joined: 24 Aug 08
Posts: 7
Credit: 37,536,564
RAC: 163
Message 53053 - Posted: 8 Dec 2015, 15:44:28 UTC

What can cause the message Download Failed? There is plenty of free space on the disk.
ID: 53053 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53054 - Posted: 8 Dec 2015, 15:59:44 UTC - in response to Message 53053.  

Look in the Stderr list on the page for that model.

Of more concern, is that some of your computers are crashing everything, possibly due to a permissions problem.

ID: 53054 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 53055 - Posted: 8 Dec 2015, 15:59:50 UTC - in response to Message 53053.  

The download failed ones are because a file is in the wrong place on the server if I remember correctly.
ID: 53055 · Report as offensive     Reply Quote
jrapdx

Send message
Joined: 4 Jul 15
Posts: 63
Credit: 3,223,760
RAC: 0
Message 53060 - Posted: 9 Dec 2015, 2:51:28 UTC - in response to Message 53055.  
Last modified: 9 Dec 2015, 3:07:45 UTC

I had two tasks fail with the same download error that Wes reported. If I understand what's going on, the server wasn't able to find >=1 file the client requested, though not necessarily the same files on repeated tries. I'm guessing this error is affecting more than a few clients.

The fact that for the same work unit successive clients receive download errors for different sets of files is a curiosity, though the important thing appears to be that quite a few files intended to be downloaded aren't in the proper location. Is it fair to call this a problem of server misconfiguration?
ID: 53060 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53061 - Posted: 9 Dec 2015, 5:35:06 UTC - in response to Message 53060.  

There may be a couple of "errors" that apply in this case:

1) Some files for one of the several separate batches of work, were needed to be in a location that didn't have the correct permissions; once they were in place, they couldn't be read.

2) One of the several separate batches of work had an error in the greenhouse gases provided for them.

All of this was fixed soon after being reported to the project. But the server software is such that once the model and it's auxiliary files has been grabbed by someone, those can't be pulled from the queue. So only those that HADN'T been issued were pulled. Then the whole batch was re-issued.
For the rest, it's just a matter of waiting for a computer to grab one of the faulty sets, and thus reduce it's "Number of re-tries", and hope that it's not one of your computers.



ID: 53061 · Report as offensive     Reply Quote
jrapdx

Send message
Joined: 4 Jul 15
Posts: 63
Credit: 3,223,760
RAC: 0
Message 53064 - Posted: 9 Dec 2015, 8:19:29 UTC - in response to Message 53061.  

Thanks for the information, interesting, I hadn't considered permissions, then again I'd think "read" access would be available for a file in the right directory.

More relevant is the history of the two "fail to download" tasks. In both instances the tasks had been previously crunched until "error while computing" (exit code 193), followed by download errors on being sent to two computers on Dec 5 2015, my computer being the last. (Re: workunits 9887820 and 10008664.)

It wasn't clear when the problem was fixed, maybe after Dec 5. In that case it looks like I was "unlucky" enough to get the bum tasks. At least it seems my systems were not the source of the issue, well, this time anyway.

Thanks,
JRA
ID: 53064 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 53233 - Posted: 11 Jan 2016, 16:57:39 UTC
Last modified: 11 Jan 2016, 16:59:42 UTC

On my laptop, sudo apt-get install wine followed by downloading the boinc .exe file and clicking on it is straightforward. Just need to make sure that the linux version of boinc isn't running or the windows version doesn't work. Same happens the other way around on my boxes at least, the linux version doesn't connect if the other one is running under WINE.

I now have 4 of the latest batch of WAH2_eu tasks running all over an hour in with no problems so far. Will post later to confirm that uploads etc work normally.

I read that there were minor problems with the processor type being reported incorrectly but that seems to have been resolved with the latest wine and boinc.
ID: 53233 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 53252 - Posted: 13 Jan 2016, 16:29:36 UTC

First zips all gone without issue. took a bit of searching to find the projects folder for the wine installation but got there in the end. Looks like so far no nasty surprises.
ID: 53252 · Report as offensive     Reply Quote
ChrisD

Send message
Joined: 8 Aug 04
Posts: 69
Credit: 1,561,341
RAC: 0
Message 53270 - Posted: 16 Jan 2016, 11:05:09 UTC - in response to Message 52570.  

sudo apt-get install libxmu:i386 gets the graphics to work, at least it does on my beta tasks, none of the other work I have has graphics available.


E:unable to locate package

seems this lib is obsolete now.

Can You help me with another package name?

Tnx :)


ChrisD


ID: 53270 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 53271 - Posted: 16 Jan 2016, 11:54:47 UTC - in response to Message 53270.  

which distribution are you using? it should be possible if unavailable for yours to find another .deb or .rpm package which you can download and install. It may be that none of the new tasks will have graphics so this may be academic.
ID: 53271 · Report as offensive     Reply Quote
Bob Knippel

Send message
Joined: 3 Mar 13
Posts: 2
Credit: 13,423,511
RAC: 0
Message 53272 - Posted: 16 Jan 2016, 16:24:19 UTC - in response to Message 52840.  
Last modified: 16 Jan 2016, 16:25:13 UTC

I used to do this when I ran Lubuntu, and it always worked. I found the information somewhere on the internet, but I don't remember where.

1. sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 libstdc++6 libstdc++5 freeglut3

2. You also need from a 32 bit system libgcc_s.so.1 and libstdc++.so.6
Copy and paste those two files into /lib32 . I believe I got those files from a 32 bit Lubuntu distribution which I now keep on a USB flash drive.
ID: 53272 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 53286 - Posted: 18 Jan 2016, 11:24:20 UTC

1. sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 libstdc++6 libstdc++5 freeglut3

2. You also need from a 32 bit system libgcc_s.so.1 and libstdc++.so.6
Copy and paste those two files into /lib32 . I believe I got those files from a 32 bit Lubuntu distribution which I now keep on a USB flash drive.


That on it's own doesn't satisfy the dependencies for the graphics to work.

E:unable to locate package

seems this lib is obsolete now.


A bit surprised as it worked for me on the latest xubuntu distribution. If it doesn't get installed automagically by one of the others I think you may also need sudo apt-get install libjpeg:i386
ID: 53286 · Report as offensive     Reply Quote
WB8ILI

Send message
Joined: 1 Sep 04
Posts: 161
Credit: 81,522,141
RAC: 1,164
Message 53539 - Posted: 29 Feb 2016, 14:06:58 UTC

I found on some thread (not related to BOINC or CPDN) on the Internet in a reply to a reply that installing libstdc++6:i386 might solve missing libstdc++.so.6 issues. It seems to have done just that for me.

There has to be a better way of determining if a computer has the necessary libraries installed than downloading 10 CPDN tasks and having them all abort. For the not so expert LINUX user, clear, unambiguous instructions on how to resolve missing libraries would be nice too.
ID: 53539 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 53540 - Posted: 29 Feb 2016, 14:48:16 UTC - in response to Message 53539.  

There is, go to the projects/climateprediction.net directory and in a terminal type ldd executable name and you will get an output something like

ldd hadam3p_eu_7.23_i686-pc-linux-gnu
linux-gate.so.1 => (0xf7715000)
libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf76c5000)
libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf76c0000)
libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf7548000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf74fb000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf74dd000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7322000)
/lib/ld-linux.so.2 (0x565cc000)

As all the dependencies for this executable are met it doesn't show any unmet ones but it will show you what needs to be installed. Sometimes you may need to do some research to find which package contains the missing libs. Yes it would be nice if there was a command to find and install them but if there is a command/script to do that I haven't found it yet.
ID: 53540 · Report as offensive     Reply Quote
WB8ILI

Send message
Joined: 1 Sep 04
Posts: 161
Credit: 81,522,141
RAC: 1,164
Message 53541 - Posted: 29 Feb 2016, 15:14:28 UTC

Thanks Dave -

That addresses 90% of my concerns. I installed (upgraded) to 64-bit UBUNTU a few months ago but haven't been able to use it on CPDN because of the because of the absence of work. I had about 10 tasks download and abort before I realized what was happening.

If you have a "new" installation or maybe a new .gnu files from CPDN, you would have to -
1) Download at least one task and let it abort to get the new .gnu file(s)
2) Run the ldd command to find missing libraries
3) Install some stuff
4) Repeat steps 2 and 3 until nothing missing

That should work. I wonder what percentage of the CPDN LINUX users know this?






ID: 53541 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 19 · Next

Questions and Answers : Unix/Linux : *** Running 32bit CPDN from 64bit Linux - Discussion ***

©2024 cpdn.org