climateprediction.net home page
Future CPDN on Windows?

Future CPDN on Windows?

Questions and Answers : Windows : Future CPDN on Windows?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile geoguy09

Send message
Joined: 6 Sep 04
Posts: 22
Credit: 349,445
RAC: 68
Message 66155 - Posted: 30 Sep 2022, 9:23:53 UTC

Is CPDN planned to be able to run on Windows 10 / 11 in the future? If not, could CPDN run in a Linux VirtualBox system?
Fedora Linux 39 64-bit Workstation
Dell Latitude E7440 Intel Core i5-4300U x 4
RAM = 16GB / 240 GB SSD
ID: 66155 · Report as offensive     Reply Quote
wateroakley

Send message
Joined: 6 Aug 04
Posts: 195
Credit: 28,193,230
RAC: 10,406
Message 66156 - Posted: 30 Sep 2022, 11:14:06 UTC - in response to Message 66155.  

Is CPDN planned to be able to run on Windows 10 / 11 in the future? If not, could CPDN run in a Linux VirtualBox system?
CPDN will run on different flavours of Linux VMs, including WSL and VirtualBox.
This message link and thread https://www.cpdn.org/forum_thread.php?id=9025&postid=64962#64962 gives a rundown on using Oracle VirtualBox 6 and ubuntu 20.04 LTS:
Make sure you find the correct 32 bit libraries for your Linux version: https://www.cpdn.org/forum_thread.php?id=8916
Please note the importance of pausing and managing M$ Windoze updates, otherwise the updates will routinely crash models. The VM has migrated quite happily to a newer Windows PC.

I've also got a Mac Mojave VM running too, although the VM was unhappy after the migration.
ID: 66156 · Report as offensive     Reply Quote
Profile geoguy09

Send message
Joined: 6 Sep 04
Posts: 22
Credit: 349,445
RAC: 68
Message 66158 - Posted: 30 Sep 2022, 15:51:44 UTC - in response to Message 66156.  

Thank you.
Fedora Linux 39 64-bit Workstation
Dell Latitude E7440 Intel Core i5-4300U x 4
RAM = 16GB / 240 GB SSD
ID: 66158 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 314
Credit: 14,557,388
RAC: 18,279
Message 66160 - Posted: 30 Sep 2022, 20:14:04 UTC

Windows work is definitely rare here, seems to be about once a year. However, as mentioned above, virtualization is definitely a good option. To virtualize Linux on Windows 10/11 I'd recommend WSL2 as it uses a lot less resources and is relatively easy to set up. Almost all of my CPDN work has been done in WSL2 Ubuntu. However, a few other distributions of Linux are available for WSL2 as well. CPDN also has mac only work sometimes but even a 32-bit macOS (Mojave or earlier) can be virtualized on Windows but it's trickier to get it to work.

Also, as mentioned above, if you're going to set up your computers to do Linux work please make sure to install the necessary 32-bit libraries. This is extremely important and has been a big problem for the project. Also, it's best to let tasks run to completion without interruptions as BOINC and computer restarts tend to crash them and you end up loosing all of the work.

I believe there's work going to be done in the future to make virtualization more automated, like Rosetta python projects or LHC CMS Simulation, if you're familiar with those projects. For now you have to set everything up yourself.
ID: 66160 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4529
Credit: 18,661,594
RAC: 14,529
Message 66163 - Posted: 1 Oct 2022, 7:30:21 UTC

This month, there should be some testing work for the new 64bit openIFS tasks which will eventually run on up to 8 cores. Long term the plan is to have these available as virtual disk images to run on Windows machines. Given that this is CPDN's first venture into the multi-core arena, I would expect that they will want to make sure it is working on native Linux machines first and any problems with that ironed out. I doubt if the Windows work will be out this year. With regards to native Windows tasks, for the moment I think there will be more but only sporadically and with little notice before these endangered species appear.
ID: 66163 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1044
Credit: 16,196,312
RAC: 12,647
Message 66181 - Posted: 10 Oct 2022, 10:24:24 UTC - in response to Message 66163.  

This month, there should be some testing work for the new 64bit openIFS tasks which will eventually run on up to 8 cores. Long term the plan is to have these available as virtual disk images to run on Windows machines. Given that this is CPDN's first venture into the multi-core arena, I would expect that they will want to make sure it is working on native Linux machines first and any problems with that ironed out. I doubt if the Windows work will be out this year. With regards to native Windows tasks, for the moment I think there will be more but only sporadically and with little notice before these endangered species appear.
Small correction, we won't go beyond 4 cores for the OpenIFS tasks as the speedup tails off too much after 4. I've tested OpenIFS runs fine with virtualbox, it's just a case of getting time with the CPDN team to implement on their server. I know the CPDN team are very keen to make more use of their Windows based volunteers as they have 3-4x as many windows machines as all other OSes combined.

Although I use WSL I find managing two boinc clients under windows & WSL a bit of faff to keep an eye on cpu & memory usage, and have given up with the WSL client. And VBox has been hassle for W11 (see e.g: https://www.techradar.com/news/oracle-virtualbox-users-will-finally-be-able-to-use-windows-11).
ID: 66181 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 314
Credit: 14,557,388
RAC: 18,279
Message 66182 - Posted: 11 Oct 2022, 7:35:25 UTC - in response to Message 66181.  
Last modified: 11 Oct 2022, 7:37:02 UTC

I know the CPDN team are very keen to make more use of their Windows based volunteers as they have 3-4x as many windows machines as all other OSes combined.

Is there really going to be that much work available on a consistent basis that this is a concern? I thought the project is oversubscribed as it is. Up to about 1000 of properly configured machines can easily handle the frequency and the amount of work that's been coming through in a recent year or so. Is the project thinking about going Windows VBox for everything? I hope not as that would suck for all of the long term Linux only contributors.

From a user perspective, BOINC clients on WSL and Windows can both be managed from one BOINC manager in Windows. You can only look at/manage one at a time though but switching is easy. I have my PC set up this way.

I hope OpenIFS is going to be available natively for Linux not just VBox.
ID: 66182 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4529
Credit: 18,661,594
RAC: 14,529
Message 66183 - Posted: 11 Oct 2022, 9:16:33 UTC - in response to Message 66182.  

I hope OpenIFS is going to be available natively for Linux not just VBox.


Rest assured, OpenIFS is going to be available natively for Linux and Macs, there will eventually be VM work in it for Windows but that probably won't be available right away. Once the 64bit models are more widely available I have no idea how much work there will be. These tasks are going to use up to 8 cpus which as the tasks take the same amount of RAM whether using one or 8 cores, memory should actually become less of an issue.

There is little point in using VMs for Mac or Linux as it is just an additional overhead, not to mention a complication if the forums on some of the projects using VB are anything to go by. (For some at least running tasks in a VM on Linux is not straightforward.
ID: 66183 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1044
Credit: 16,196,312
RAC: 12,647
Message 66184 - Posted: 11 Oct 2022, 13:13:09 UTC - in response to Message 66183.  

I hope OpenIFS is going to be available natively for Linux not just VBox.
Rest assured, OpenIFS is going to be available natively for Linux and Macs
Linux native yes, but I would prefer to use VM for macOS as well. I built the early OpenIFS model for older macs, but neither I nor the CPDN team currently have a newer apple silicon machine and the macOS build environment is subtly different (regarding shared libs) I can't simply move my linux compile scripts straight over. Bottom line is, producing any model native for macs means more work. Going the VM route for windows also works for macs. Otherwise CPDN have to manage 3 different app versions for each model (linux native, mac native, & VM for Windows) and that's just too much.
ID: 66184 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1044
Credit: 16,196,312
RAC: 12,647
Message 66185 - Posted: 11 Oct 2022, 19:29:25 UTC - in response to Message 66183.  

if the forums on some of the projects using VB are anything to go by. (For some at least running tasks in a VM on Linux is not straightforward.
p.s. I forgot to add that I had a problem with VBox & boinc on my linux mint 21 machine. There is a bug in the boinc-client.service file in the current client package. ProtectSystem=strict needs to be changed to ProtectSystem=full in order for VirtualBox to run correctly; 'strict' does not allow VBox access to the tmp filesystem that it needs. After that change, LHC runs fine. OpenIFS (or any Hadley model) using VBox would have the same problem. Hopefully they will roll out the fix soon to the package managers.
ID: 66185 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4529
Credit: 18,661,594
RAC: 14,529
Message 66186 - Posted: 12 Oct 2022, 7:49:54 UTC - in response to Message 66185.  
Last modified: 12 Oct 2022, 8:26:21 UTC

Thanks Glen, That got VB tasks working for me.
ID: 66186 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4529
Credit: 18,661,594
RAC: 14,529
Message 66419 - Posted: 15 Nov 2022, 13:18:25 UTC - in response to Message 66415.  

I reckon it could be possible for the developers to compile a Windows version of the OpenIFS applications if they wanted to.
OpenIFS has been developed by a pan European group. It is open source. However when it came out I was told the developers had no plans to develop a Windows version. There is a Mac version but there will be fewer differences between that and a Linux version as MacOS is Unix based. Certainly, CPDN does not have the resources to develop a Windows version of OpenIFS and it is OpenIFS itself that does not have a Windows version so unless a Windows version is developed then there will never be Windows apps using it. Longer term however there is a plan to let Windows run the tasks using images created forVirtual Box.
ID: 66419 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1044
Credit: 16,196,312
RAC: 12,647
Message 66424 - Posted: 15 Nov 2022, 14:13:04 UTC - in response to Message 66415.  

I reckon it could be possible for the developers to compile a Windows version of the OpenIFS applications if they wanted to.
you've obviously never developed on Windows!!
ID: 66424 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1044
Credit: 16,196,312
RAC: 12,647
Message 66425 - Posted: 15 Nov 2022, 14:20:20 UTC - in response to Message 66419.  

I reckon it could be possible for the developers to compile a Windows version of the OpenIFS applications if they wanted to.
OpenIFS has been developed by a pan European group. It is open source. However when it came out I was told the developers had no plans to develop a Windows version. There is a Mac version but there will be fewer differences between that and a Linux version as MacOS is Unix based. Certainly, CPDN does not have the resources to develop a Windows version of OpenIFS and it is OpenIFS itself that does not have a Windows version so unless a Windows version is developed then there will never be Windows apps using it. Longer term however there is a plan to let Windows run the tasks using images created forVirtual Box.
Dave, sorry, just to correct a few things there.

IFS is the operational forecast model at the European Centre for Medium Range Weather Forecasts. It was developed primarily between ECMWF (where I used to work) and MeteoFrance. There have been contributions from other European organisations too, but it's mainly those two.

OpenIFS is a version of IFS with some of the model components (like the ocean) removed. It's available free to research institutes but it is NOT open source, despite the name (which was a poor choice in hindsight).

We did look at porting OpenIFS on Windows but there are so many differences to unix/linux and complications because needed system libraries are different, it was not worth the effort. The approach we are taking now is to develop a virtualbox application for OpenIFS in Windows. That's what I've been looking at recently as Dave says.

Hope that helps.
ID: 66425 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4529
Credit: 18,661,594
RAC: 14,529
Message 66443 - Posted: 15 Nov 2022, 19:57:30 UTC

Dave, sorry, just to correct a few things there.


Thanks for setting me straight Glen. No need to be sorry. I would rather be corrected than be allowed to continue to spread false information albeit unwittingly.
ID: 66443 · Report as offensive     Reply Quote
Corsair
Avatar

Send message
Joined: 27 Jan 07
Posts: 3
Credit: 15,723,923
RAC: 5,636
Message 66643 - Posted: 29 Nov 2022, 23:39:17 UTC

Really sad to see that there are no future plans for windows, when supposed to be the most spread pc running platform.

will see how the future comes.
Roses don't bloom on the sailor's grave

ID: 66643 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1044
Credit: 16,196,312
RAC: 12,647
Message 66645 - Posted: 30 Nov 2022, 0:06:41 UTC - in response to Message 66643.  

Really sad to see that there are no future plans for windows, when supposed to be the most spread pc running platform
please read earlier posts. There are plans for Windows.

Operational weather forecast models are not developed on windows as it's not designed for HPC. That's why it's not easy to port them.
ID: 66645 · Report as offensive     Reply Quote
Hans Sveen

Send message
Joined: 31 Aug 04
Posts: 5
Credit: 17,330,980
RAC: 9,061
Message 68499 - Posted: 27 Feb 2023, 9:54:01 UTC

Hi!
For the first time since I Joined Climateprediction 31 Aug 2004, my Rac
has decreased to 0.00 !
Hope it will increase in the near future when the VirtualBox
Wrapper is finished (soon?)🤔

With regards,
Hans Sveen
Oslo
Hans Sveen
Oslo,Norway

ID: 68499 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1058
Credit: 36,583,942
RAC: 15,889
Message 68500 - Posted: 27 Feb 2023, 10:51:11 UTC - in response to Message 68499.  

There is a - hopefully temporary - problem at the moment which is preventing RAC being calculated, even on platforms where work has been available in recent weeks and months. It's affecting all of us.
ID: 68500 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4529
Credit: 18,661,594
RAC: 14,529
Message 68501 - Posted: 27 Feb 2023, 11:17:19 UTC - in response to Message 68500.  

There is a - hopefully temporary - problem at the moment which is preventing RAC being calculated, even on platforms where work has been available in recent weeks and months. It's affecting all of us.
Some while since I had last looked at my average. Mine has been dropping every week on the graph since 29th January despite having done a fair amount of OIFS work. It is clearly lying!
ID: 68501 · Report as offensive     Reply Quote
1 · 2 · Next

Questions and Answers : Windows : Future CPDN on Windows?

©2024 cpdn.org