climateprediction.net (CPDN) home page
Thread 'WaH v8.29 bug leaves files behind in BOINC/data/projects/climateprediction -- please delete by hand'

Thread 'WaH v8.29 bug leaves files behind in BOINC/data/projects/climateprediction -- please delete by hand'

Message boards : Number crunching : WaH v8.29 bug leaves files behind in BOINC/data/projects/climateprediction -- please delete by hand
Message board moderation

To post messages, you must log in.

AuthorMessage
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,490,541
RAC: 15,784
Message 70535 - Posted: 23 Feb 2024, 13:05:09 UTC

A bug has been found in the latest v8.29 Weather@Home app currently running EAS25 batches as app-name wah2_ri.

When the task finishes it mistakenly leaves some text files behind in the task directory inside the BOINC/data/projects/climateprediction.net directory. Task directories have long names such as 'wah2_eas25_a3pf_200912_24_1007_012269659'. This is the same as the task 'Name' column in boincmgr.

When a task finishes, it should delete the task directory (e.g. wah2_eas25_a3pf_200912_24_1007_012269659) in the projects/climateprediction.net folder, but unfortunately it leaves behind the 3 text files:
stdout_mon.txt
stdout_rm.txt
stdout_um.txt
Task directories with only these 3 files can be safely deleted.
DO NOT delete task directories which also contain 'datain', 'dataout', 'jobs' and any other files. These indicate running tasks. Also do not delete the entire 'climateprediction.net' directory as this will kill all running tasks.

Apologies for the inconvenience. The bug has been identified and corrected. A new version will be issued before any more batches are released.

Thanks to George (@geophi) for the initial report.
---
CPDN Visiting Scientist
ID: 70535 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 70539 - Posted: 23 Feb 2024, 16:58:20 UTC - in response to Message 70535.  

Where is this directory usually located for Windows machines?
ID: 70539 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 70540 - Posted: 23 Feb 2024, 17:54:35 UTC - in response to Message 70539.  

C:program data >BOINC> Projects>climateprediction.net Is the default on my Win10 installation in a VM. (I had to look as crashed tasks don't leave all their detritus around nearly as often as they used to.
ID: 70540 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 70543 - Posted: 23 Feb 2024, 20:01:20 UTC

Ah, thanks. Found it, I'll clean it up at some point after tasks are done.

... and c:\Programdata is a hidden folder. So helpful, Microsoft... so helpful.
ID: 70543 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,890,678
RAC: 18,887
Message 70544 - Posted: 23 Feb 2024, 21:25:13 UTC - in response to Message 70535.  

On one of my PCs I found a number of these directories from old tasks, with not only the text files but those folders too: 3 from batch 996, 1 from 1002, 2 from 1003, 2 from 1004.
ID: 70544 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 70545 - Posted: 23 Feb 2024, 22:09:49 UTC - in response to Message 70544.  

On one of my PCs I found a number of these directories from old tasks, with not only the text files but those folders too: 3 from batch 996, 1 from 1002, 2 from 1003, 2 from 1004.

I have often seen folders, sometimes with hundreds of megabytes in from crashed tasks though not recently. Don't remember it with tasks that complete.
ID: 70545 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,490,541
RAC: 15,784
Message 70548 - Posted: 24 Feb 2024, 10:04:53 UTC - in response to Message 70545.  

Thanks for filling in the bits of info I missed in the original post.

A design flaw in the original WaH code is responsible for leftover files/dirs in the project dir. The task directories should really be in the boinc/slots directory, not the projects directory. Only the output files waiting to transfer should go into the projects directory. That way, the boinc client cleans out the slot dir even if the task has a catastrophic failure. The OpenIFS tasks were designed this way and I'm planning on making this change for WaH too.
---
CPDN Visiting Scientist
ID: 70548 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 70549 - Posted: 24 Feb 2024, 15:33:15 UTC - in response to Message 70548.  

The OpenIFS tasks were designed this way and I'm planning on making this change for WaH too.


Yay! More code quality of life tweaks from Glenn! :D
ID: 70549 · Report as offensive     Reply Quote
wujj123456

Send message
Joined: 14 Sep 08
Posts: 127
Credit: 42,025,219
RAC: 69,000
Message 70553 - Posted: 24 Feb 2024, 18:09:26 UTC

This seems to be minimal compared to the hundreds of MB that crashed tasks leave behind. It's probably easier to just reset the project once I'm out of work, unless my disk space is running short. Glad more improvements are coming too.
ID: 70553 · Report as offensive     Reply Quote

Message boards : Number crunching : WaH v8.29 bug leaves files behind in BOINC/data/projects/climateprediction -- please delete by hand

©2024 cpdn.org