2gb file size limit on vrscene submiting

Paul_Oblomov · June 17, 2015, 8:00am

Hello guys!
I’m very new to deadline, so here’s a question (that i’ve already sent to support person mail I was given, but still no answer):
we’re using centos 6.6 x64 at renderfarm nodes and deadline version 7.0.1.3 R + vray3.0 standalone nightlies
when trying manual submission of vrscene, that is more, than 2gb in size deadline says, that it can’t read such a files.

2015-06-17 10:55:52: 0: Got task! 2015-06-17 10:55:52: Constructor: Vray 2015-06-17 10:55:52: 0: Loaded plugin Vray (/mnt/dl/plugins/Vray) 2015-06-17 10:55:52: 0: Start Job timeout is disabled. 2015-06-17 10:55:52: 0: Task timeout is disabled. 2015-06-17 10:55:52: 0: Loaded job: Untitled (558127ff0e35522588b21ed2) 2015-06-17 10:55:52: 0: INFO: Executing plugin script /root/Thinkbox/Deadline7/slave/node50/plugins/558127ff0e35522588b21ed2/Vray.py 2015-06-17 10:55:52: 0: INFO: About: VRay Standalone Plugin for Deadline 2015-06-17 10:55:52: 0: INFO: The job's environment will be merged with the current environment before rendering 2015-06-17 10:55:52: 0: Plugin rendering frame(s): 1 2015-06-17 10:55:53: CheckPathMapping: Swapped "R:/render_sc/episode_01/sc08/sc08_vfx010/sc08_vfx010_whole.vrscene" with "/mnt/pf/render_sc/episode_01/sc08/sc08_vfx010/sc08_vfx010_whole.vrscene" 2015-06-17 10:55:53: 0: INFO: Starting VRay Task 2015-06-17 10:55:53: 0: INFO: Performing path mapping on vrscene file 2015-06-17 10:55:54: 0: An exception occurred: Error: Reading more than 2GB with this call is not supported (System.IO.IOException) 2015-06-17 10:55:54: at System.IO.File.ReadAllBytes (System.String path) [0x00000] in <filename unknown>:0 2015-06-17 10:55:54: at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplace (System.String inFileName, System.String outFileName, System.String[] stringsToReplace, System.String[] newStrings, Deadline.Controllers.DataController dataController, FranticX.GenericDelegate1`1 logFunction, System.String regionName) [0x00000] in <filename unknown>:0 2015-06-17 10:55:54: at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplaceSeparator (System.String inFileName, System.String outFileName, System.String separatorToReplace, System.String newSeparator, Deadline.Controllers.DataController dataController, FranticX.GenericDelegate1`1 logFunction, System.String regionName) [0x00000] in <filename unknown>:0 2015-06-17 10:55:54: at Deadline.Scripting.RepositoryUtils.CheckPathMappingInFileAndReplaceSeparator (System.String inFileName, System.String outFileName, System.String separatorToReplace, System.String newSeparator) [0x00000] in <filename unknown>:0 2015-06-17 10:55:54: at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) 2015-06-17 10:55:54: at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 2015-06-17 10:55:54: at Deadline.Plugins.ScriptPlugin.RenderTasks (System.String taskId, Int32 startFrame, Int32 endFrame, System.String& outMessage, FranticX.Processes.AbortLevel& abortLevel) [0x00000] in <filename unknown>:0 (Deadline.Plugins.RenderPluginException) 2015-06-17 10:55:54: 0: Unloading plugin: Vray 2015-06-17 10:55:55: Scheduler Thread - Render Thread 0 threw a major error: 2015-06-17 10:55:55: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2015-06-17 10:55:55: Exception Details 2015-06-17 10:55:55: RenderPluginException -- Error: Reading more than 2GB with this call is not supported (System.IO.IOException) 2015-06-17 10:55:55: at System.IO.File.ReadAllBytes (System.String path) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplace (System.String inFileName, System.String outFileName, System.String[] stringsToReplace, System.String[] newStrings, Deadline.Controllers.DataController dataController, FranticX.GenericDelegate1`1 logFunction, System.String regionName) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplaceSeparator (System.String inFileName, System.String outFileName, System.String separatorToReplace, System.String newSeparator, Deadline.Controllers.DataController dataController, FranticX.GenericDelegate1`1 logFunction, System.String regionName) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: at Deadline.Scripting.RepositoryUtils.CheckPathMappingInFileAndReplaceSeparator (System.String inFileName, System.String outFileName, System.String separatorToReplace, System.String newSeparator) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) 2015-06-17 10:55:55: at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: at Deadline.Plugins.ScriptPlugin.RenderTasks (System.String taskId, Int32 startFrame, Int32 endFrame, System.String& outMessage, FranticX.Processes.AbortLevel& abortLevel) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: RenderPluginException.Cause: JobError (2) 2015-06-17 10:55:55: RenderPluginException.Level: Major (1) 2015-06-17 10:55:55: RenderPluginException.HasSlaveLog: True 2015-06-17 10:55:55: RenderPluginException.SlaveLogFileName: /var/log/Thinkbox/Deadline7/deadlineslave_renderthread_0-node50-0000.log 2015-06-17 10:55:55: Exception.HResult: -2146233088 2015-06-17 10:55:55: Exception.Source: deadline 2015-06-17 10:55:55: Exception.TargetSite: Void RenderTask(System.String, Int32, Int32) 2015-06-17 10:55:55: Exception.Data: ( ) 2015-06-17 10:55:55: Exception.StackTrace: 2015-06-17 10:55:55: at Deadline.Plugins.Plugin.RenderTask (System.String taskId, Int32 startFrame, Int32 endFrame) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: at Deadline.Slaves.SlaveRenderThread.RenderCurrentTask (Deadline.IO.TaskLogWriter tlw) [0x00000] in <filename unknown>:0 2015-06-17 10:55:55: <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

So, here’s a question - what is wrong ? We have mixed env windows workstations/linux nodes with (as I was told, samba on linux, that makes it possible to share files seamlessly). Our IT guys are very dumb, so I have to fix it by myself

Yours truly,
Jingliks team.

dwallbridge · June 17, 2015, 9:29pm

Hello Paul,

So I looked into this and it’s actually something we have fixed in Deadline 7.1, so if you can upgrade to that, this should work better for you. Basically what we did was change how we do path mapping, so that we can handle bigger files.

Paul_Oblomov · June 18, 2015, 8:12am

Nice
So, we have to upgrade slaves too ?

MikeOwen · June 18, 2015, 9:24am

I believe the fix was in the core FranticX library (core/re-compile required), so, yes, a newer version of Deadline Repository installed and your slaves updated (auto upgrade helps here) would be required here.