Submit 3dsMax Job with Python Standalone API

Hi @cmoore, @Bobo, @eamsler @kwatts
I want to send job with standalone api but it throws such an error:

=======================================================
Error
=======================================================
Initialize: Error: GetIntegerPluginInfoEntry: Could not parse value for key: Version into an integer because: GetPluginInfoEntry: Script accessed non-existent plugin info key: Version (Deadline.Plugins.RenderPluginException)
konum: Deadline.Plugins.DeadlinePlugin.GetIntegerPluginInfoEntry(String key)
konum: Python.Runtime.Dispatcher.TrueDispatch(ArrayList args)
konum: Python.Runtime.Dispatcher.Dispatch(ArrayList args)
konum: FranticX.Processes.ManagedProcess.InitializeProcess()
konum: FranticX.Processes.ManagedProcess.Initialize(LogFunctions logFunctions)

=======================================================
Type
=======================================================
RenderPluginException

=======================================================
Stack Trace
=======================================================
konum: Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage akb)
konum: Deadline.Plugins.SandboxedPlugin.Initialize(Job job)
konum: Deadline.Slaves.SlaveRenderThread.e(String aaj, Job aak)
konum: Deadline.Slaves.SlaveRenderThread.b(TaskLogWriter aah)

=======================================================
Log
=======================================================
2019-01-16 05:34:18:  0: Loading Job's Plugin timeout is Disabled
2019-01-16 05:34:20:  0: Executing plugin command of type 'Sync Files for Job'
2019-01-16 05:34:20:  0: All job files are already synchronized
2019-01-16 05:34:21:  0: Plugin 3dsmax was already synchronized.
2019-01-16 05:34:21:  0: Done executing plugin command of type 'Sync Files for Job'
2019-01-16 05:34:21:  0: Executing plugin command of type 'Initialize Plugin'
2019-01-16 05:34:21:  0: INFO: Executing plugin script 'C:\Users\AdminPC\AppData\Local\Thinkbox\Deadline10\slave\Rendernode-01\plugins\5c3e97a093c3510f9c1a5ac4\3dsmax.py'
2019-01-16 05:34:22:  0: INFO: 3dsmax Plugin Initializing...
2019-01-16 05:34:22:  0: Encountered an error while executing plugin command of type 'Initialize Plugin'

=======================================================
Details
=======================================================
Date: 01/16/2019 05:34:27
Frames: 0
Elapsed Time: 00:00:00:06
Job Submit Date: 01/16/2019 05:32:00
Job User: adminpc
Average RAM Usage: 7572084736 (12%)
Peak RAM Usage: 7647469568 (12%)
Average CPU Usage: 96%
Peak CPU Usage: 100%
Used CPU Clocks (x10^6 cycles): 122528264
Total CPU Clocks (x10^6 cycles): 127633611

=======================================================
Slave Information
=======================================================
Slave Name: Rendernode-01
Version: v10.0.23.4 Release (e0d42871d)
Operating System: Windows 10 Pro
Running As Service: Yes
Machine User: AdminPC
IP Address: 192.168.1.31
MAC Address: D0:50:99:C0:90:71
CPU Architecture: x64
CPUs: 56
CPU Usage: 0%
Memory Usage: 4.1 GB / 63.9 GB (6%)
Free Disk Space: 123.337 GB 
Video Card: Microsoft Basic Display Adapter

SUBMITTER_SCRIPT:

import json, os, pathlib2

max_submit_info_file = r'C:\Users\AdminTest\AppData\Local\Thinkbox\Deadline10\temp\16.01.2019_02_17_12\max_submit_info.job'
max_job_info_file = r'C:\Users\AdminTest\AppData\Local\Thinkbox\Deadline10\temp\16.01.2019_02_17_12\max_job_info.job'
max_file = r'C:\Users\AdminTest\AppData\Local\Thinkbox\Deadline10\temp\16.01.2019_02_17_12\3dsmaxScene-2018.max'



def config_to_json(config_file):
    max_submit_info = {}
    with open(config_file, 'r') as f:
        for line in f.readlines():
            args = line.strip().split('=')
            if len(args) == 2: #(ValueError: too many values to unpack) corona_stringOptions=# additional string options: one per line, format: [type] [name] = [value] 
                variable, options = args
                # print('{}={}'.format(variable, options))
                if options == '\n' or options.startswith('#'):
                    continue
                max_submit_info[variable] = options
    return max_submit_info



submit_info = config_to_json(max_submit_info_file)
job_info = config_to_json(max_submit_info_file)
# print(json.dumps(submit_info, indent=4, sort_keys=True))

from render_api import con
print(os.path.isfile(max_file))
print(pathlib2.Path(max_file).exists())
print(con.Jobs.SubmitJob(info=submit_info, plugin=job_info))

max_job_info.py (12.5 KB)
max_submit_info.py (1.4 KB)

Hello,

From first glance it appears that the issue you are running into is that you are setting both submit_info and job_info to be the contents of the same file.

submit_info = config_to_json(max_submit_info_file)
job_info = config_to_json(max_submit_info_file)

This means that all of the information that the plugin needs is missing.

A second issue with your submission script is you currently are not including the scene file with your submission. In order to include it you will need include the scene you can use the aux argument.

print(con.Jobs.SubmitJob(info=submit_info, plugin=job_info, aux=[max_file] ))

Hi @grant.bartel

That was my first try. No project file, throwing error:

Error: 'C:\Users\AdminTest\Desktop\submit\16.01.2019_02_17_12\3dsmaxScene-2018.max'  yolunun bir parçası bulunamadı. (System.IO.DirectoryNotFoundException)

For the above error they exist.

print(os.path.isfile(max_file)) #True
print(pathlib2.Path(max_file).exists()) #True

From the First message:

Initialize: Error: GetIntegerPluginInfoEntry: Could not parse value for key: Version into an integer because: GetPluginInfoEntry: Script accessed non-existent plugin info key: Version (Deadline.Plugins.RenderPluginException)

In the plugin info data set, your missing a key, value pair that determines the Version.

Once that is provided to the plugin , it should render on the farm.

Another way to do this a little cleaner, is use the deadline submitter, to submit a simple job.
Then right click on the job, look at submission Params, make sure that the plugin info that you provide containns all the key, value pairs that the simple job has, that way you know that the plugin shouldnt fail with any other “Could not parse value for key” errors.

Hope this helps.

Kym

Hi guys,

Why do you not understand? :slight_smile: I’m getting a key error. I didn’t make it, and I opened a topic subject here. But you say what I do. I already did this. As you can see, the files of a job already done. I want to test directly over those files so there is no risk. These are the files of a successful job.(AppData\Local\Thinkbox\Deadline10\temp\16.01.2019_02_17_12)

That was the first time. Then when I looked at the API, I saw that it was already doing.

submit_info = {‘JobInfo’ : config_to_json(max_submit_info_file) }
job_info = {‘PluginInfo’ : config_to_json(max_job_info_file) }

def SubmitJob(self, info, plugin, aux = [], idOnly = False):
    """    Submit a new Job.
        Input:  info: Dictionary of Job information.
                plugin: Dictionary of Plugin information for the Job.
                aux: Array of any additional auxiliary submission files, defaults to empty.
                idOnly: If True, only the Job's ID is returned, defaults to False.
        Returns: The new Job.
    """
    if not isinstance(aux, list):
        aux = [aux]
    
    body = '{"JobInfo":'+json.dumps(info)+',"PluginInfo":'+json.dumps(plugin)+',"AuxFiles":'+json.dumps(aux)
    if idOnly:
        body += ',"IdOnly":true'
    body += '}'
    return self.connectionProperties.__post__("/api/jobs", body)

As Grant explained in his first reply, a job submission requires either 3 arguments (the submit info and job info .job files, and the .MAX scene file as aux. file), or two arguments (just the two .job files, where the job info file contains a key called SceneFile= pointing at the network location of the .MAX scene file).

In your case, your submission call only uses the two job files, but the SceneFile is not included in the job info file (at least not in the one you attached). Since the files you are testing with were created from another submitter that that included the MAX file in the command line call, you MUST use the 3 arguments approach in your submission, or the job will not work.

If you did include the .MAX file in the submission, please clarify and post the example where you did that. The samples you have posted define the location of the .MAX file in a variable, but it is unclear what happens to it after that…

The initial problem with the undefined key error was, as Grant also pointed out, because you used the wrong argument to generate the job_info JSON. I assume you solved that problem.

It is unclear to me where the error

Error: 'C:\Users\AdminTest\AppData\Local\Thinkbox\Deadline10\temp\16.01.2019_02_17_12\3dsmaxScene-2018.max' yolunun bir parçası bulunamadı. (System.IO.DirectoryNotFoundException)

comes from. Can you provide more info about what code is throwing that?

This files: 16.01.2019_02_17_12.zip (60.5 KB)

it doesn’t seem to be a problem. I thought it was a path problem. I moved it to D:\ directory. But I still throwing the not found file error.

Do you try the submit_job py script?

I have used the your config read function. That’s a different error.

Error: ‘Newtonsoft.Json.Linq.JValue’ türündeki nesne ‘Newtonsoft.Json.Linq.JObject’ türüne atanamadı. (System.InvalidCastException)

This error is caused by json.

PS C:\Users\AdminTest> python.exe C:\Users\AdminTest\Desktop\render\api\python\submit_job.py
{“´╗┐UserName”:“admintest”,“Group”:“none”,“ChunkSize”:“1”,“SendJobErrorWarning”:“true”…[CROPPED]

Ekran%20Al%C4%B1nt%C4%B1s%C4%B12

If you look carefully, you will see non-ascii characters in username.

import json, pathlib2

max_submit_info_file = r'C:\Users\AdminTest\Desktop\submit\16.01.2019_02_17_12\max_submit_info.job'
max_job_info_file = r'C:\Users\AdminTest\Desktop\submit\16.01.2019_02_17_12\max_job_info.job'
max_file = r'C:\Users\AdminTest\Desktop\submit\16.01.2019_02_17_12\3dsmaxScene-2018.max'

def fileRead(filelocation):
    
    file = open(filelocation, 'r')
    
    obj = '{'
    
    for line in file:
        
        line = line.replace('\n', '')
        line = line.replace('\t', '')
        
        tokens = line.split("=",1)
        if len(tokens) == 2:
            obj = obj + '"'+tokens[0].strip()+'":"'+tokens[1].strip()+'",'
        
    obj = obj[:-1]
    
    obj = obj + '}'
    
    return obj



submit_info = fileRead(max_submit_info_file)
job_info = fileRead(max_job_info_file)

print('Really exists ? :', pathlib2.Path(max_file).exists())
from render724 import con
print(con.Jobs.SubmitJob(info=submit_info, plugin=job_info, aux=[max_file]))

I suspect that those are Unicode BOM bytes, although their values and representation do not match what I would expect to see from UTF-8 with BOM encoding.

In the MAXScript code producing the .job files, we specifically enforce UTF-8 with BOM:

local JobInfoFile = CreateFile filename encoding:#utf8 writeBOM:true

This is because otherwise language-specific paths and names (e.g. Turkish, Bulgarian, German, French, etc.) containing special characters would be mangled and could break the job. There is a global switch in 3ds Max’s Preferences to enforce this, and some customers had it on, and some had it off, causing headaches for our support team. So eventually we hard-coded the encoding mode to enforce UTF-8 with BOM in all text output calls within the Submit Max To Deadline code.

So these files are not pure ASCII, they are in fact Unicode, and you must consider that when reading from python. At least you could skip the first three characters of any file you read…

Thanks Bob :slight_smile:

This file is already BOM: D

userforubuntu@Notebook:~$ file 
/mnt/c/Users/AdminTest/Desktop/submit/16.01.2019_02_17_12/max_submit_info.job
/mnt/c/Users/AdminTest/Desktop/submit/16.01.2019_02_17_12/max_submit_info.job: UTF-8 Unicode (with BOM) text, with CRLF line terminators