climateprediction.net (CPDN) home page
Thread 'PHANTOM DOWNLOADS'

Thread 'PHANTOM DOWNLOADS'

Message boards : Number crunching : PHANTOM DOWNLOADS
Message board moderation

To post messages, you must log in.

AuthorMessage
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 39511 - Posted: 8 Apr 2010, 19:25:46 UTC

I seem to have experience a phantom download. As I am just about finish an AM3P WU I reset my “preferences” and clicked “allow new tasks”. The usual files appeared in the “transfer” tab. The files seemed to download normally, but, the WUs never appeared in “tasks”. Now the scheduler won’t give me any more WUs until tomorrow. I am about to run out of work for one of my cores.

4/8/2010 11:57:58 AM climateprediction.net Sending scheduler request: Requested by user.
4/8/2010 11:57:58 AM climateprediction.net Requesting new tasks
4/8/2010 11:58:03 AM climateprediction.net Scheduler request completed: got 2 new tasks
4/8/2010 11:58:06 AM climateprediction.net Started download of hadam3p_msn4_1978_2_006138954.zip
4/8/2010 11:58:06 AM climateprediction.net Started download of ic19611117_14_N96.gz
4/8/2010 11:58:07 AM climateprediction.net Finished download of hadam3p_msn4_1978_2_006138954.zip
4/8/2010 11:58:07 AM climateprediction.net Started download of 1978_12_init.gz
14/8/2010 11:58:52 AM climateprediction.net Finished download of ic19611117_14_N96.gz
4/8/2010 11:58:52 AM climateprediction.net Started download of so2dms_N96_1978_12_1981_02.gz
4/8/2010 12:01:05 PM climateprediction.net Finished download of so2dms_N96_1978_12_1981_02.gz
4/8/2010 12:01:05 PM climateprediction.net Started download of HadISST_SI_N96_1978_12_1981_01f.gz
4/8/2010 12:02:32 PM climateprediction.net Finished download of HadISST_SI_N96_1978_12_1981_01f.gz
4/8/2010 12:02:32 PM climateprediction.net Started download of HadISST_SST_N96_1978_12_1981_01f.gz
4/8/2010 12:06:07 PM climateprediction.net Finished download of HadISST_SST_N96_1978_12_1981_01f.gz
4/8/2010 12:06:07 PM climateprediction.net Started download of hadam3p_mszt_1993_2_006139411.zip
4/8/2010 12:06:08 PM climateprediction.net Finished download of hadam3p_mszt_1993_2_006139411.zip
4/8/2010 12:06:08 PM climateprediction.net Started download of ic19610807_11_N96.gz
4/8/2010 12:07:13 PM climateprediction.net Finished download of ic19610807_11_N96.gz
4/8/2010 12:07:13 PM climateprediction.net Started download of so2dms_N96_1993_12_1996_02.gz
4/8/2010 12:09:08 PM climateprediction.net Finished download of so2dms_N96_1993_12_1996_02.gz
4/8/2010 12:09:08 PM climateprediction.net Started download of 1993_12_init.gz
4/8/2010 12:14:56 PM climateprediction.net Finished download of 1978_12_init.gz
4/8/2010 12:14:56 PM climateprediction.net Started download of HadISST_SI_N96_1993_12_1996_01f.gz
4/8/2010 12:15:54 PM climateprediction.net Finished download of HadISST_SI_N96_1993_12_1996_01f.gz
4/8/2010 12:15:54 PM climateprediction.net Started download of HadISST_SST_N96_1993_12_1996_01f.gz
4/8/2010 12:19:58 PM climateprediction.net Finished download of HadISST_SST_N96_1993_12_1996_01f.gz
4/8/2010 12:24:06 PM climateprediction.net Finished download of 1993_12_init.gz
4/8/2010 1:21:15 PM climateprediction.net Sending scheduler request: To send trickle-up message.
4/8/2010 1:21:15 PM climateprediction.net Not reporting or requesting tasks
4/8/2010 1:21:20 PM climateprediction.net Scheduler request 4/8/2010 11:56:17 AM climateprediction.net Sending scheduler request: To fetch work.
4/8/2010 11:56:17 AM climateprediction.net Requesting new tasks
4/8/2010 11:56:22 AM climateprediction.net Scheduler request completed: got 0 new tasks
4/8/2010 11:56:22 AM climateprediction.net Message from server: No work sent
4/8/2010 11:56:22 AM climateprediction.net Message from server: No work available for the applications you have selected. Please check your settings on the web site.
4/8/2010 11:57:58 AM climateprediction.net update requested by user
4/8/2010 11:57:58 AM climateprediction.net Sending scheduler request: Requested by user.
4/8/2010 11:57:58 AM climateprediction.net Requesting new tasks
4/8/2010 11:58:03 AM climateprediction.net Scheduler request completed: got 2 new tasks
4/8/2010 11:58:06 AM climateprediction.net Started download of hadam3p_msn4_1978_2_006138954.zip
4/8/2010 11:58:06 AM climateprediction.net Started download of ic19611117_14_N96.gz
4/8/2010 11:58:07 AM climateprediction.net Finished download of hadam3p_msn4_1978_2_006138954.zip
4/8/2010 11:58:07 AM climateprediction.net Started download of 1978_12_init.gz
14/8/2010 11:58:52 AM climateprediction.net Finished download of ic19611117_14_N96.gz
4/8/2010 11:58:52 AM climateprediction.net Started download of so2dms_N96_1978_12_1981_02.gz
4/8/2010 12:01:05 PM climateprediction.net Finished download of so2dms_N96_1978_12_1981_02.gz
4/8/2010 12:01:05 PM climateprediction.net Started download of HadISST_SI_N96_1978_12_1981_01f.gz
4/8/2010 12:02:32 PM climateprediction.net Finished download of HadISST_SI_N96_1978_12_1981_01f.gz
4/8/2010 12:02:32 PM climateprediction.net Started download of HadISST_SST_N96_1978_12_1981_01f.gz
4/8/2010 12:06:07 PM climateprediction.net Finished download of HadISST_SST_N96_1978_12_1981_01f.gz
4/8/2010 12:06:07 PM climateprediction.net Started download of hadam3p_mszt_1993_2_006139411.zip
4/8/2010 12:06:08 PM climateprediction.net Finished download of hadam3p_mszt_1993_2_006139411.zip
4/8/2010 12:06:08 PM climateprediction.net Started download of ic19610807_11_N96.gz
4/8/2010 12:07:13 PM climateprediction.net Finished download of ic19610807_11_N96.gz
4/8/2010 12:07:13 PM climateprediction.net Started download of so2dms_N96_1993_12_1996_02.gz
4/8/2010 12:09:08 PM climateprediction.net Finished download of so2dms_N96_1993_12_1996_02.gz
4/8/2010 12:09:08 PM climateprediction.net Started download of 1993_12_init.gz
4/8/2010 12:14:56 PM climateprediction.net Finished download of 1978_12_init.gz
4/8/2010 12:14:56 PM climateprediction.net Started download of HadISST_SI_N96_1993_12_1996_01f.gz
4/8/2010 12:15:54 PM climateprediction.net Finished download of HadISST_SI_N96_1993_12_1996_01f.gz
4/8/2010 12:15:54 PM climateprediction.net Started download of HadISST_SST_N96_1993_12_1996_01f.gz
4/8/2010 12:19:58 PM climateprediction.net Finished download of HadISST_SST_N96_1993_12_1996_01f.gz
4/8/2010 12:24:06 PM climateprediction.net Finished download of 1993_12_init.gz
4/8/2010 1:21:15 PM climateprediction.net Sending scheduler request: To send trickle-up message.
4/8/2010 1:21:15 PM climateprediction.net Not reporting or requesting tasks
4/8/2010 1:21:20 PM climateprediction.net Scheduler request completed

completed



ID: 39511 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 39515 - Posted: 8 Apr 2010, 21:32:06 UTC

Everybody who runs a lot of CPDN gets a phantom model from time to time. I\'ve had two or three.

I see that both your computers have 2 cores and they\'re allowed 3 and 4 models per core per day. So the problem isn\'t that you\'ve reached your daily quota.

I\'m sure you\'re more than capable of selecting model types according to what\'s available so I just don\'t believe the Boinc message that no models were available for the type you\'d selected. I\'ve had this message before when I know it wasn\'t true. I think there\'s sometimes something \'misconfigured\' about this message but as I don\'t understand what I don\'t feel able to complain.

I wonder whether Boinc or the CPDN server thinks that the two phantom models are actually there in your Tasks tab. If this is the case it remains to be seen whether you\'ll get another model after your midnight.

It may be worth attaching to another project as well to make sure the computers can be kept busy in this sort of situation.
Cpdn news
ID: 39515 · Report as offensive     Reply Quote
ProfileThyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 39517 - Posted: 8 Apr 2010, 22:29:16 UTC - in response to Message 39511.  
Last modified: 8 Apr 2010, 22:35:46 UTC

I seem to have experience a phantom download. As I am just about finish an AM3P WU I reset my “preferences” and clicked “allow new tasks”. The usual files appeared in the “transfer” tab. The files seemed to download normally, but, the WUs never appeared in “tasks”. Now the scheduler won’t give me any more WUs until tomorrow. I am about to run out of work for one of my cores.

I see you\'re running BOINC 6.10 Jim, so I\'m guessing that you\'ve fallen into the trap that caught me out on the Tasks tab of BOINC Manager\'s advanced view.

The top command button was \"Show graphics\" in previous versions of BOINC but it\'s now \"Show active tasks\". Tasks which haven\'t started running aren\'t displayed in the list when that button has been clicked; the only indication that this has happened is the label changing to \"Show all tasks\".
"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 39517 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 39519 - Posted: 9 Apr 2010, 0:49:45 UTC

My bad. I did get the 2 new WUs. Thyme was right. The 6.10 was hiding the inactive WUs from me. I realized that when the AM3P finished and one of the new WU’s popped up.

ID: 39519 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 39520 - Posted: 9 Apr 2010, 1:43:20 UTC

The phantoms are revealed at dead of night!

I also got caught out by the new buttons. The Properties button\'s a useful improvement though. And I do like being able to select the messages about one project.
Cpdn news
ID: 39520 · Report as offensive     Reply Quote
Ingleside

Send message
Joined: 5 Aug 04
Posts: 127
Credit: 24,574,485
RAC: 8,766
Message 39521 - Posted: 9 Apr 2010, 7:04:07 UTC - in response to Message 39515.  

Everybody who runs a lot of CPDN gets a phantom model from time to time. I\'ve had two or three.

It wasn\'t really a phantom in this case, but as for \"ghost-wu\", the solution is for the project to turn-on re-issue of \"lost\" wu. Now, unfortunately this puts larger load on database, so can be a reason for not using it.

I\'m sure you\'re more than capable of selecting model types according to what\'s available so I just don\'t believe the Boinc message that no models were available for the type you\'d selected. I\'ve had this message before when I know it wasn\'t true. I think there\'s sometimes something \'misconfigured\' about this message but as I don\'t understand what I don\'t feel able to complain.

Actually, the message very likely is true...

For performance-reasons, the Scheduling-server doesn\'t look directly at the database for work, until have found a likely canditate. Instead, it looks on a shared memory-segment, setup by the Feeder. The Feeder as default sets up 100 \"slots\", but CPDN can run with a larger Feeder. Using the default, this mean it\'s at max only 100 models ready to send at any given time. As users gets assigned work, the \"slot\" will be empty, until it\'s re-filled. Again depending on configuration, it can take some seconds before it\'s re-filled. Until it\'s re-filled, it\'s possible there aren\'t any work at all available, even status-page lists there\'s 10k+ models available. For the Scheduling-server, it\'s never more than N models available at any given time, there N depends on #slots the Feeder has.

Also, it\'s various methods to configure the Feeder, the two most common with multi-applications is either \"random\" application, or each application gets a fraction of the Feeder. So, if it\'s \"random\", it\'s possible all slots is filled with HADSM3, while user only asks for HADAM3P, and therefore it can be some time before next HADAM3P gets loaded by the Feeder.

If it\'s based on fraction, project can example say HADSM3 gets 70% of the slots, while HADAM3P gets 30%, so if there\'s a total of 100 slots this means the 30 HADAM3P-slots can be empty when user asks for work, but there are HADSM3 available. Until the HADAM3P-part of feeder is re-filled, there won\'t be any HADAM3P available.

ID: 39521 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 39525 - Posted: 9 Apr 2010, 8:48:54 UTC

Thank you. What you say explains the Boinc message which has sometimes seemed very strange.
Cpdn news
ID: 39525 · Report as offensive     Reply Quote
metalius
Avatar

Send message
Joined: 28 Nov 06
Posts: 89
Credit: 12,052,709
RAC: 2,860
Message 39807 - Posted: 31 May 2010, 10:12:19 UTC
Last modified: 31 May 2010, 10:25:05 UTC

Phantom downloads - that is a real problem, which is hitting hard the morale. I found - this problem appears on multi-project machines, when CPDN is allowed to receive new tasks permanent.
List of my tasks, received in last few days, which will never start:
hadsm3dhet2_jw80_006604754_9
hadsm3dhet2_jw7y_006604752_8
hadsm3dhet2_jw7o_006604742_1
hadam3p_ncrb_1990_2_006165025_3
ID: 39807 · Report as offensive     Reply Quote
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,884,997
RAC: 4,577
Message 39810 - Posted: 31 May 2010, 16:34:07 UTC - in response to Message 39807.  

Phantom downloads - that is a real problem, which is hitting hard the morale. ...

Since your version of BOINC predates the confusing \'Show active tasks\' button, I assume your phantoms are real. These happen occasionally and are usually attributed to a server load problem. Some people have been reporting difficulties getting a new model, which might be expected along with phantoms.

Messages have been left for the project staff, which should be actioned tomorrow (today is a UK holiday).

If some tasks are lost then that\'s the project\'s problem: just keep trying from your side and a task should arrive eventually.
ID: 39810 · Report as offensive     Reply Quote
metalius
Avatar

Send message
Joined: 28 Nov 06
Posts: 89
Credit: 12,052,709
RAC: 2,860
Message 39820 - Posted: 1 Jun 2010, 10:57:19 UTC - in response to Message 39810.  
Last modified: 1 Jun 2010, 11:03:56 UTC

Since your version of BOINC predates the confusing \'Show active tasks\' button, I assume your phantoms are real.

No, I have Phantom tasks on several machines with different versions of BOINC manager, including 6.2.19, which do not have \'Show active tasks\' button at all. ;-)
Thank You for fast reply.
ID: 39820 · Report as offensive     Reply Quote
[B^S] mavau

Send message
Joined: 30 Aug 04
Posts: 142
Credit: 9,936,132
RAC: 0
Message 40073 - Posted: 3 Jul 2010, 9:44:45 UTC

I've just noticed I have 2 of these phantom models in my list:
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=11594077
and
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=11585789

Could it be connected with the daily quota issue?

Forum search Site search
ID: 40073 · Report as offensive     Reply Quote
metalius
Avatar

Send message
Joined: 28 Nov 06
Posts: 89
Credit: 12,052,709
RAC: 2,860
Message 40191 - Posted: 20 Jul 2010, 16:19:55 UTC - in response to Message 40073.  

The question is - How to free such phantom tasks?
I found no solution. I tried Reset project and Detach -> Re-attach... No effect. The tasks are still shown as In progress.
ID: 40191 · Report as offensive     Reply Quote
Profile[B@H] Ray
Avatar

Send message
Joined: 19 Aug 05
Posts: 104
Credit: 1,866,495
RAC: 0
Message 40192 - Posted: 20 Jul 2010, 17:27:49 UTC - in response to Message 40191.  

The question is - How to free such phantom tasks?
I found no solution. I tried Reset project and Detach -> Re-attach... No effect. The tasks are still shown as In progress.

You can not take them off your list of tasks, they will stay there till the unit is finished by others and it is removed from the server database. That may take a long time as they tend to leave all work there for a few years.
Keep on crunching Pizza@Home
ID: 40192 · Report as offensive     Reply Quote
metalius
Avatar

Send message
Joined: 28 Nov 06
Posts: 89
Credit: 12,052,709
RAC: 2,860
Message 40193 - Posted: 20 Jul 2010, 20:14:10 UTC - in response to Message 40192.  
Last modified: 20 Jul 2010, 20:18:11 UTC

You can not take them off your list of tasks...

If we speak about CPDN, You are absolutely right. ;o) But if to speak about another projects, You may be wrong. Detach from project, having a goal to free a phantom task or tasks, was at least an old and known focus. I say was at least instead of is, because I am not sure, is this old feature still working - a lot of projects upgraded already their servers to newest or newer versions.
ID: 40193 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40194 - Posted: 20 Jul 2010, 21:02:27 UTC

Most other projects need to delete the completed tasks quickly because the tasks are short so there are very large quantities. On CPDN the tasks are longer so there are fewer and they can be kept longer.

A year ago the database was cleaned up. But the completed tasks still need to be kept for the researchers.

The phantoms will stay on our lists until one of the programmers cleans up the database again. It's a big job.

There was once a big accident on the server. All the waiting tasks flooded out in minutes as phantoms to the computers connected at the time. One lady's computer got over 2000 phantoms. She apparently never noticed.
Cpdn news
ID: 40194 · Report as offensive     Reply Quote
metalius
Avatar

Send message
Joined: 28 Nov 06
Posts: 89
Credit: 12,052,709
RAC: 2,860
Message 40196 - Posted: 21 Jul 2010, 4:39:13 UTC - in response to Message 40194.  

mo.v, as You know, for a quite long period of time CPDN was similar to projects like SETI or Einstein - HADAM3P work units used minimum quorum 2 by initial replication 2. So every phantom slowed down the progress for 1 year.
But at this moment I want to say another thing.
I am very glad to inform - the hocus-pocus Detach and Re-attach is working here at CPDN!!!
One fact for evidence... Tasks 11109287, 11109266 and 11109159 (all 3 detached on 20 Jul 2010 16:23:50 UTC) were phantoms. Now all 3 are "killed".
Simple I forgot again - sometimes CPDN servers are refreshing hosts' pages quite slow.
ID: 40196 · Report as offensive     Reply Quote

Message boards : Number crunching : PHANTOM DOWNLOADS

©2024 cpdn.org