DeadLine Test

anon80383789 · February 26, 2010, 8:08am

Hi,

I’m Global 3D supervisor at BlueSpiritStudio http://www.spirit-prod.com/ and we are looking to move from backburner to another render management tool beacause we have too many problem with backburner and we want to mix all software we have (fusion, combustion, 3dsMax, etc.)
I’m currently testing deadline 4.0 with two slaves but i need a script to send by one click the job to deadline without any interface. The goal is to have a script runing in the paul Neal’s “Batch it Max” http://paulneale.com/scripts/batchItMax/batchItMax.htm script to send close to 700 jobs automaticly to test this tool like we are using it on backburner (but backburner crash with too many jobs)
Do you have any scripts for that?

Thanks for you help.

rrussell · February 26, 2010, 1:51pm

We don’t have any pre-made scripts to do this (our out-of-the-box submission script has an interface), but I’m sure we can help get you started on one. How good is your maxscript?

I think it may be possible to use our existing submission script as a base, so that all you have to is populate a pre-existing struct of job properties and call existing functions to submit the jobs to Deadline. I’ll talk with Bobo (our maxscript guru) to see what he thinks.

Cheers,

Ryan

anon80383789 · February 26, 2010, 2:19pm

Thanks for your reply.
I’m an intermediate Maxscript level.
The good thing will be to have possibility to send a script exactly like the previous one send with the interface with one click, or if we need something different edit the “SubmitMaxToDeadline_Defaults.ini”.
For working on series we have to make same jobs send all the day long.
It will help us to plug it with the pass manager we are using : http://www.vanilla-seed.com/ later. Now i just want to test with our .max extract from the differents files we already sent to backbuner.

Eddy.

Bobo · February 26, 2010, 2:42pm

Hi Eddy,

The functionality of the Submit Max To Deadline (SMTD) is split between the GUI and the actual submission and processing functions.
Preparing a MAXScript that batch-submits any number of files based on calling functions in the SMTD_Functions struct shouldn’t take much effort.
In fact, many products like Krakatoa and partially Grant Adam’s RPManager (which even integrates SMTD UI controls) use the existing functions to implement their own submission processes. We also do this from our inhouse render pass manager without ever showing the SMTD UI.

I can help you get something together - I will write up a simple example of the basic steps needed. In short, all you have to do is:
*Resolve the repository path and load the SMTD_Functions struct
*Save a copy of the max scene or just grab and existing path to submit
*Set any preferences you might have in the SMTD_Settings struct which holds all the values used during submission (our UI just modifies that).
*Save the two .JOB files (submit info and job info files) that control the submission process by calling two library functions
*Call the submission function with the max file name and the job files as command arguments.

Once you see the actual MAXScript code, it will become much clearer…

anon80383789 · February 26, 2010, 2:49pm

Thanks Bobo for your help.
I will wait for your script sample and make what’s missing by myself.

Thanks a lot.

Eddy.

Bobo · February 26, 2010, 5:04pm

Hi Eddy,

I am writing the example and have some questions.
There are several ways one could perform mass-submission to deadline.

Send each file to Deadline without loading it into 3dsmax, but submit the .MAX file with the job
Send each file to Deadline without loading it into 3dsmax by just sending the path to the file, assuming it is on a network path visible to the slaves
Open each file in 3ds Max and submit to Deadline.

Obviously, the 1. and 2. approaches would be faster since the scene files are never loading into the submitting copy of 3ds Max, esp. 2 which doesn’t even copy the .MAX file to the Repository. But the drawback of these methods is that you have NO knowledge of what is in the scene - what viewport would render, what other cameras are in the scene, what the renderer settings are etc. All that we could do there is specify the desired output path and hope that the scenes were saved with the correct camera set in the active viewport and that all renderer settings are correct.

Method 3. will export all relevant settings automatically so you would be able to modify settings in the Monitor’s Properties dialog, and you would have knowledge of the camera and viewport settings etc. The drawback would be that it would take longer since it includes the loading the file in the submitting copy of 3ds Max.

Right now I am inclined to show the implementation of 3.
Can you provide some feedback which approach would most closely match what you will be doing?

Bobo · February 26, 2010, 5:39pm

I went ahead and implemented method 3.

See the attached script. I used it to submit 6 files I found in my c:\temp folder and it worked
Example_CustomSubmitter.zip (4.53 KB)

anon80383789 · March 1, 2010, 7:53am

Hi Bobo,
Thanks for your awesome Script.

Problem solved with vray Frame buffer disable.
Gestion-105_00000_20100301084415.errorReport (3.52 KB)

anon80383789 · March 11, 2010, 8:31am

Hi,

Do the deadline command to send job to the repository save the acutal job to the file or copy the file from his location to the repository?

We are working on a vanilla pass Manager connection with deadline and if it’s a simple file copy it will be difficult.

Regards,

Eddy

anon80383789 · March 24, 2010, 11:33am

Hi,

I do have a problem with an PreFrameScriptFile in the script you give me i add those line:

SMTDSettings.RunPreFrameScript = true SMTDSettings.PreFrameScriptFile = "J:\\common\\tools\\render\\DeadLine_BatchIt\\01_ColorMapping_vfb_test.ms"

But when the job started on the slave i have this error:

[code]Error Message
Exception during render: An error occurred in RenderTasks(): RenderTask: Unexpected exception (Exception caught in 3ds max: mxs::file_in() - Tried to execute a maxscript file which doesn’t exist: “C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/01_ColorMapping_vfb_test.ms”
2010/03/24 12:15:55 INF: Loaded C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011_persos_light_.max
2010/03/24 12:15:58 WRN: Gamma Correction is Being Enabled
2010/03/24 12:16:03 INF: Job: C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011_persos_light_.max
)
Ã Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)

Slave Log
em_workInCameraSpace set to true
0: INFO: +system_instancedObjects set to 2
0: INFO: +system_checkForMissingFiles set to false
0: INFO: +system_optimizedAtmospherics set to false
0: INFO: +system_lowThreadPriority set to false
0: INFO: +system_vrayLog_show set to true
0: INFO: +system_vrayLog_level set to 3
0: INFO: +system_raycaster_memLimit set to 400
0: INFO: +system_vrayLog_file set to c:\VRayLog.txt
0: INFO: +Image Sampler set to Adaptive Subdivision
0: INFO: +Primary Bounces GI Engine set to Irradiance Map
0: INFO: +Primary Secondary GI Engine set to None
0: INFO: +Adv.Irradiance Map Interpolation Type set to Least squares fit (good/smooth)
0: INFO: +Adv.Irradiance Map Sample Lookup set to Density-based (best)
0: INFO: +Adv.Irradiance Map Mode set to Single Frame
0: INFO: +Region Sequence set to Spiral
0: INFO: +Frame Stamp Justify set to Left
0: INFO: >Performing XREF workaround…
0: Plugin rendering frame(s): 0-5
0: INFO: Render Tasks called
0: INFO: Executing script: C:\Users\eddy\AppData\Local\Prime Focus\Deadline\slave\jobsData\01_ColorMapping_vfb_test.ms
LICENSE-FREE MODE: Repository has two slaves or less - no license required.
Scheduler Thread - Render Thread 0 threw an error:
Scheduler Thread - Exception during render: An error occurred in RenderTasks(): RenderTask: Unexpected exception (Exception caught in 3ds max: mxs::file_in() - Tried to execute a maxscript file which doesn’t exist: “C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/01_ColorMapping_vfb_test.ms”
2010/03/24 12:15:55 INF: Loaded C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011_persos_light_.max

2010/03/24 12:15:58 WRN: Gamma Correction is Being Enabled

2010/03/24 12:16:03 INF: Job: C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011_persos_light_.max

)

Ã Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)

Error Type
RenderPluginException

Error Stack Trace
Ã Deadline.Plugins.Plugin.RenderTask(Int32 startFrame, Int32 endFrame)
Ã Deadline.Slaves.SlaveRenderThread.RenderCurrentTask()[/code]

Do i have to copy this file localy?

rrussell · March 24, 2010, 12:32pm

You also need to submit the script file with the job. It would be included with the auxiliary files that are appended to the submission command. For example:

deadlinecommand.exe “job_info_file” “plugin_info_file” “myscene.max” “J:\common\tools\render\DeadLine_BatchIt\01_ColorMapping_vfb_test.ms”

Hope that helps!

Ryan

anon80383789 · March 24, 2010, 4:27pm

I tried with the SMTD 4 (not the script you gived me) and put the script to the “run post load script” but i have a problem.
I try with try()catch() but i still have the same problem but the script works perfectly in the viewport (using the original file or the file saved in jobs)

The Script:

[code]vr = renderers.current as string
if findstring vr “V_Ray” != undefined then
(
if findstring maxFileName “occlusion” != undefined then
(
try(

		--task wanted
		renderers.current.output_on = false
		
		renderers.current.colorMapping_gamma = 1.0
		renderers.current.colorMapping_gamma = 1.0
		renderers.current.colorMapping_linearWorkflow = true 
		renderers.current.colorMapping_linearWorkflow = false 
	)
	catch
	(
		--task wanted need to be done twice
		renderers.current.output_on = false
		
		renderers.current.colorMapping_gamma = 1.0
		renderers.current.colorMapping_gamma = 1.0
		renderers.current.colorMapping_linearWorkflow = true 
		renderers.current.colorMapping_linearWorkflow = false 
	)
)
else
(
	try(	
		--task wanted
		renderers.current.output_on = false
		
		renderers.current.colorMapping_gamma = 1.0
		renderers.current.colorMapping_gamma = 2.2
		renderers.current.colorMapping_linearWorkflow = true 
		renderers.current.colorMapping_linearWorkflow = false 
	)
	catch
	(
		--task wanted need to be done twice
		renderers.current.output_on = false
		
		renderers.current.colorMapping_gamma = 1.0
		renderers.current.colorMapping_gamma = 2.2
		renderers.current.colorMapping_linearWorkflow = true 
		renderers.current.colorMapping_linearWorkflow = false 
	)
)

)[/code]

the error:

[code]Error Message
An error occurred in StartJob(): RenderTask: Unexpected exception (Maxscript exception:-- Unable to convert: false to type: String
2010/03/24 17:21:21 INF: Loaded C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011rk2_persos_light.max
2010/03/24 17:21:24 WRN: Gamma Correction is Being Enabled
2010/03/24 17:21:27 INF: Job: C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011rk2_persos_light.max
)

Slave Log

0: INFO: +system_region_sequence set to 3
0: INFO: +system_region_sequence_reverse set to 0
0: INFO: +system_previousRender set to 0
0: INFO: +system_frameStamp_on set to false
0: INFO: +system_frameStamp_string set to V-Ray %vrayversion // render time: %rendertime
0: INFO: +system_frameStamp_fullWidth set to true
0: INFO: +system_frameStamp_justify set to 0
0: INFO: +system_workInCameraSpace set to true
0: INFO: +system_instancedObjects set to 2
0: INFO: +system_checkForMissingFiles set to false
0: INFO: +system_optimizedAtmospherics set to false
0: INFO: +system_lowThreadPriority set to false
0: INFO: +system_vrayLog_show set to true
0: INFO: +system_vrayLog_level set to 3
0: INFO: +system_raycaster_memLimit set to 400
0: INFO: +system_vrayLog_file set to c:\VRayLog.txt
0: INFO: +Image Sampler set to Adaptive Subdivision
0: INFO: +Primary Bounces GI Engine set to Irradiance Map
0: INFO: +Primary Secondary GI Engine set to None
0: INFO: +Adv.Irradiance Map Interpolation Type set to Least squares fit (good/smooth)
0: INFO: +Adv.Irradiance Map Sample Lookup set to Density-based (best)
0: INFO: +Adv.Irradiance Map Mode set to Single Frame
0: INFO: +Region Sequence set to Spiral
0: INFO: +Frame Stamp Justify set to Left
0: INFO: >Performing XREF workaround…
0: INFO: Executing script: C:\Users\eddy\AppData\Local\Prime Focus\Deadline\slave\jobsData\01_ColorMapping_vfb_test.ms
Scheduler Thread - Render Thread 0 threw an error:
Scheduler Thread - An error occurred in StartJob(): RenderTask: Unexpected exception (Maxscript exception:-- Unable to convert: false to type: String
2010/03/24 17:21:21 INF: Loaded C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011rk2_persos_light.max

2010/03/24 17:21:24 WRN: Gamma Correction is Being Enabled

2010/03/24 17:21:27 INF: Job: C:/Users/eddy/AppData/Local/Prime Focus/Deadline/slave/jobsData/e16sr011rk2_persos_light.max

)

Error Type
RenderPluginException

Error Stack Trace
Ã Deadline.Plugins.ScriptPlugin.StartJob(Job job)
Ã Deadline.Plugins.Plugin.StartJob(Job job)
Ã Deadline.Slaves.SlaveRenderThread.RenderCurrentTask()[/code]

rrussell · March 24, 2010, 5:12pm

Try adding “return true” to the end of the script:

    vr = renderers.current as string
    if findstring vr "V_Ray" != undefined then
    (
       if findstring maxFileName "occlusion" != undefined then
       (
          try(      
             
             --task wanted
             renderers.current.output_on = false
             
             renderers.current.colorMapping_gamma = 1.0
             renderers.current.colorMapping_gamma = 1.0
             renderers.current.colorMapping_linearWorkflow = true
             renderers.current.colorMapping_linearWorkflow = false
          )
          catch
          (
             --task wanted need to be done twice
             renderers.current.output_on = false
             
             renderers.current.colorMapping_gamma = 1.0
             renderers.current.colorMapping_gamma = 1.0
             renderers.current.colorMapping_linearWorkflow = true
             renderers.current.colorMapping_linearWorkflow = false
          )
       )
       else
       (
          try(   
             --task wanted
             renderers.current.output_on = false
             
             renderers.current.colorMapping_gamma = 1.0
             renderers.current.colorMapping_gamma = 2.2
             renderers.current.colorMapping_linearWorkflow = true
             renderers.current.colorMapping_linearWorkflow = false
          )
          catch
          (
             --task wanted need to be done twice
             renderers.current.output_on = false
             
             renderers.current.colorMapping_gamma = 1.0
             renderers.current.colorMapping_gamma = 2.2
             renderers.current.colorMapping_linearWorkflow = true
             renderers.current.colorMapping_linearWorkflow = false
          )
       )
    )
    return true

Deadline is looking for “true” result from the script, and I think the line “renderers.current.colorMapping_linearWorkflow = false” currently sets the result to “false”.

Cheers,

Ryan

Bobo · March 24, 2010, 5:23pm

As a rule of thumb, MAXScripts run on Deadline are expected to return TRUE for the job to continue.
Since MAXScript always returns a value from any expression, in this case the False was returned by accident as it was the last value assigned to a property inside the code.

That being said, we should handle the return of ‘false’ with a better error message explaining that the script actually returned false and the job had to fail because of that. As it is now, the error report looks more like a syntax error than a user-requested canceling…

anon80383789 · March 25, 2010, 7:49am

If i add the script inside a function it’s working fine with the “return = true” at the end it’s work. Thanks for you help.

[code]fn todoPostLoad =
(
vr = renderers.current as string
if findstring vr “V_Ray” != undefined then
(
if findstring maxFileName “occlusion” != undefined then
(
try(

			--task wanted
			renderers.current.output_on = false
			
			renderers.current.colorMapping_gamma = 1.0
			renderers.current.colorMapping_gamma = 1.0
			renderers.current.colorMapping_linearWorkflow = true 
			renderers.current.colorMapping_linearWorkflow = false 
		)
		catch
		(
			--task wanted need to be done twice
			renderers.current.output_on = false
			
			renderers.current.colorMapping_gamma = 1.0
			renderers.current.colorMapping_gamma = 1.0
			renderers.current.colorMapping_linearWorkflow = true 
			renderers.current.colorMapping_linearWorkflow = false 
		)
	)
	else
	(
		try(	
			--task wanted
			renderers.current.output_on = false
			
			renderers.current.colorMapping_gamma = 1.0
			renderers.current.colorMapping_gamma = 2.2
			renderers.current.colorMapping_linearWorkflow = true 
			renderers.current.colorMapping_linearWorkflow = false 
		)
		catch
		(
			--task wanted need to be done twice
			renderers.current.output_on = false
			
			renderers.current.colorMapping_gamma = 1.0
			renderers.current.colorMapping_gamma = 2.2
			renderers.current.colorMapping_linearWorkflow = true 
			renderers.current.colorMapping_linearWorkflow = false 
		)
	)	
)
return true

)

todoPostLoad()[/code]

anon80383789 · April 7, 2010, 2:48pm

Hi Bobo and rrussel.

Thanks for you help i wanna create more tools to integrate deadline in our workflow. we will test 25 slaves soon. I want to know through MXS how many jobs are waiting in queue line in the deadline jobs repository. Can you help me?

One more thing i have an issue with this function: getRepositoryListFilename()
It’s looking for a file : local path = GetNetworkRoot() + "\\submission\\3dsmax\\RemoteSubmissionRepositories.ini" that do not exist in this repository.

rrussell · April 7, 2010, 4:58pm

You can make calls to deadlinecommand.exe to query information from the repository:
software.primefocusworld.com/sof … ommand.php

This command will print out how many jobs are in each state:

deadlinecommand.exe GetJobStatistics

Just parse the Queued Jobs line in the stdout to get the number of queued jobs.

The getRepositoryListFilename() function is only meant for internal use only, so you shouldn’t ever have to call this function. Is there a reason you’re trying to call it?

Cheers,

Ryan