AWS Thinkbox Discussion Forums

Requeuing orphaned task X

Is this normal? I see it constantly in the pulse log:

2013-11-05 12:25:53: Requeuing orphaned task ‘139’ for LAPRO0659: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘134’ for LAPRO0602: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘93’ for LAPRO1463: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘80’ for LAPRO0303: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘81’ for LAPRO0045: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘82’ for LAPRO0267: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘9’ for LAPRO1291: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘18’ for LAPRO1200: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘19’ for LAPRO0428: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘20’ for LAPRO1388: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘36’ for LAPRO1317: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘26’ for LAPRO0096: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:53: Requeuing orphaned task ‘27’ for LAPRO0047: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘28’ for LAPRO0057: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘30’ for LAPRO1387: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘30’ for LAPRO0414: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘20’ for LAPRO1325: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘128’ for LAPRO0171: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘107’ for LAPRO1249: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘61’ for LAPRO0641: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘108’ for LAPRO0439: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘41’ for LAPRO1470: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘41’ for LAPRO0173: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘40’ for LAPRO1299: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘25’ for LAPRO1290: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘56’ for LAPRO0214: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘25’ for LAPRO1234: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘26’ for LAPRO1307: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘27’ for LAPRO1464: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘28’ for LAPRO1310: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘29’ for LAPRO1451: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘30’ for LAPRO1391: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘31’ for LAPRO1430: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘32’ for LAPRO1347: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘33’ for LAPRO1305: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘34’ for LAPRO1467: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘30’ for LAPRO1232: the task is in the rendering state, but the slave is no longer rendering this task
2013-11-05 12:25:54: Requeuing orphaned task ‘31’ for LAPRO1369: the task is in the rendering state, but the slave is no longer rendering this task

I looked at the code, and it looks like this could be the reason you see your slaves canceling their tasks randomly. The problem is that the slave states are loaded at the start of housecleaning, and the orphaned task check is one of the last things to happen during housecleaning (it happens after job cleanup occurs, and after dependencies were scanned). Since these operations take a while for you, the housecleaning process is working with out-of-date slave states, and thus requeues the tasks when it shouldn’t be.

In beta 10, the slave states will be refreshed prior to checking for orphaned tasks, and some additional sanity checks will be put in place to help prevent false positives from occurring.

Privacy | Site terms | Cookie preferences