Checking all rendered tiles exist on file server?

Deadline
1

Hi,

I’m using the VrayPrepass Verify event plugin as a base (https://github.com/ThinkboxSoftware/Deadline/tree/master/Custom/events/VRayPrePassVerify) to check for missing tiles on the network that should have been rendered.

It works pretty well for looking at the main file output, but I had a slave that ran out of disk space and only managed to copy the RGB output back to the server, none of the elements - so the verify script thought everything was ok with the render.

I’ve had a look at changing it to look at all the render element paths, but it usually has a lot of outputs, and a different number on each job, so hard coding it to look for a certain number of render elements won’t work.

This is the meaty bit of the python file that is checking the render element output:

        # Calculate the exr file path from the job properties (find the relevant info in the deadline job properties)
    RenderElementOutputFilename0 = self.GetPluginInfoEntryWithDefault( "RenderElementOutputFilename0", "" )
    filePath = Path.GetDirectoryName( RenderElementOutputFilename0 )
    fileName = Path.GetFileNameWithoutExtension( RenderElementOutputFilename0 )
    fileName = str( fileName ) + "####.exr"
    fileName = Path.Combine( filePath, fileName )
    padding = "####"
    taskIdsToRequeue = []

I’ve tried wildcarding the RenderElementOutputFilename, but it just looks for a file called RenderElementOutputFilename.*.0000.exr which is very wrong :slight_smile:

Any ideas on how to tackle this? My python-fu is beginner at the moment.

Thanks.

2

Something along these lines .
(haven’t tested that, so take it with a grain of salt)

frameList =  job.JobFramesList
for o in job.JobOutputFileNames:
    fileNameParts = o.split(".")
    for f in frameList:
        #This is assuming that the naming convention follows your example
        frame = "{0:04d}".format(int(f))
        testFile = "{0}{1}.{2}.{3}"format(filePath, fileNameParts[0], frame, fileNameParts[2])
        #Now just test that the file exists. 
        #Might also want to break this to separate functions
3

Listing files in a directory? Here is an example of how you would pull the whole list with that “*” in the file name (jargon term: file globbing).

A good example of walking output can be found in the AfterEffects plugin ("[repo]/plugins/AfterEffects/AfterEffects.py"):

    # Walk the file list
    def some_function:
        frames = range( self.GetStartFrame(), self.GetEndFrame()+1 )
        for frame in frames:
            currFile = FrameUtils.ReplacePaddingWithFrameNumber( outputFile, frame )
            failures += self.CheckFileSize( currFile, minSizeInKB, deleteFilesUnderMinSize )

    def CheckFileSize( self, fileToCheck, minFileSizeInKB, deleteFilesUnderMinSize ):
        failures = 0

        self.LogInfo( 'Checking file size of "%s"' % fileToCheck )
        if os.path.isfile( fileToCheck ):
            # math and things go here...
        else:
            self.LogInfo( "    %s does not exist" % os.path.basename( fileToCheck ) )
            failures += 1

        return failures

I guess my real question is, how are you keeping track of how many render elements there should be? I can’t remember if they’re included in the output paths or not. Can you check that in either the PluginInfo or JobInfo files?