Arnold | Error: IOException : Input/output error

Hello,
I have an issue, when i try to render using either Arnold Standalone AMI or my custom Arnold AMI I get an Error ;

=======================================================
Error

Error: IOException : Input/output error
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func2 errorRewriter) at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode) at System.IO.FileStream.OpenHandle(FileMode mode, FileShare share, FileOptions options) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) at System.IO.File.OpenRead(String path) at Deadline.IO.PathMappingUtils.c(String bvi, String bvj, String bvk, String[] bvl, String[] bvm, IList1 bvn, Boolean bvo, DataController bvp, GenericDelegate11 bvq, String bvr, OS bvs) at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplace(String inFileName, String outFileName, String forceSeperator, String[] stringsToReplace, String[] newStrings, DataController dataController, GenericDelegate11 logFunction, String regionID, Boolean readFileAsBytes)
at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplace(String inFileName, String outFileName, String forceSeperator, String stringsToReplace, String newStrings, DataController dataController, GenericDelegate11 logFunction, String regionID) at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplace(String inFileName, String outFileName, String[] stringsToReplace, String[] newStrings, DataController dataController, GenericDelegate11 logFunction, String regionID)
at Deadline.IO.PathMappingUtils.CheckPathMappingInFileAndReplaceSeparator(String inFileName, String outFileName, String separatorToReplace, String newSeparator, DataController dataController, GenericDelegate11 logFunction, String regionName) at Deadline.Scripting.RepositoryUtils.CheckPathMappingInFileAndReplaceSeparator(String inFileName, String outFileName, String separatorToReplace, String newSeparator) (Python.Runtime.PythonException) File "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-59-248/plugins/61697cbd70d2134be05ec02e/Arnold.py", line 104, in RenderArgument RepositoryUtils.CheckPathMappingInFileAndReplaceSeparator( filename, localFilename, "\\", "/" ) at Python.Runtime.Dispatcher.Dispatch(ArrayList args) at __FranticX_GenericDelegate01[[System_String, System_Private_CoreLib, Version=4_0_0_0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]Dispatcher.Invoke()
at FranticX.Processes.ManagedProcess.RenderArgument()
at Deadline.Plugins.DeadlinePlugin.RenderArgument()
at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type

RenderPluginException

=======================================================
Stack Trace

at Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage bfl, CancellationToken bfm)
at Deadline.Plugins.SandboxedPlugin.RenderTask(Task task, CancellationToken cancellationToken)
at Deadline.Slaves.SlaveRenderThread.c(TaskLogWriter ajp, CancellationToken ajq)

=======================================================
Log

2021-10-15 13:16:33: 0: Loading Job’s Plugin timeout is Disabled
2021-10-15 13:16:33: 0: WARNING: Python version for ‘Arnold’ plugin is not specified! Defaulting to Py2.
2021-10-15 13:16:33: 0: SandboxedPlugin: Render Job As User disabled, running as current user ‘ec2-user’
2021-10-15 13:16:36: 0: Executing plugin command of type ‘Initialize Plugin’
2021-10-15 13:16:36: 0: INFO: Executing plugin script ‘/var/lib/Thinkbox/Deadline10/workers/ip-10-128-59-248/plugins/61697cbd70d2134be05ec02e/Arnold.py’
2021-10-15 13:16:36: 0: INFO: About: Arnold Plugin for Deadline
2021-10-15 13:16:36: 0: INFO: The job’s environment will be merged with the current environment before rendering
2021-10-15 13:16:36: 0: Done executing plugin command of type ‘Initialize Plugin’
2021-10-15 13:16:36: 0: Start Job timeout is disabled.
2021-10-15 13:16:36: 0: Task timeout is disabled.
2021-10-15 13:16:36: 0: Loaded job: ***** - render_cam_def1 - masterLayer - Export (61697cbd70d2134be05ec02e)
2021-10-15 13:16:36: 0: Executing plugin command of type ‘Start Job’
2021-10-15 13:16:36: 0: INFO: Sending StartTaskRequest to S3BackedCacheClient.
2021-10-15 13:16:36: 0: DEBUG: Request:
2021-10-15 13:16:36: 0: DEBUG: JobId: 61697cbd70d2134be05ec02e
2021-10-15 13:16:36: 0: DEBUG: JobUploadWhitelist: .exr
2021-10-15 13:16:36: 0: DEBUG: JobUploadWhitelistRe: ^.+.abc$, ^.+.avi$, ^.+.bmp$, ^.+.bw$, ^.+.cin$, ^.+.cjp$, ^.+.cjpg$, ^.+.cxr$, ^.+.dds$, ^.+.dpx$, ^.+.dwf$, ^.+.dwfx$, ^.+.dwg$, ^.+.dxf$, ^.+.dxx$, ^.+.eps$, ^.+.exr$, ^.+.fbx$, ^.+.fxr$, ^.+.hdr$, ^.+.icb$, ^.+.iff$, ^.+.iges$, ^.+.igs$, ^.+.int$, ^.+.inta$, ^.+.iris$, ^.+.jpe$, ^.+.jpeg$, ^.+.jpg$, ^.+.jp2$, ^.+.mcc$, ^.+.mcx$, ^.+.mov$, ^.+.mxi$, ^.+.pdf$, ^.+.pic$, ^.+.png$, ^.+.prt$, ^.+.ps$, ^.+.psd$, ^.+.rgb$, ^.+.rgba$, ^.+.rla$, ^.+.rpf$, ^.+.sat$, ^.+.sgi$, ^.+.stl$, ^.+.sxr$, ^.+.targa$, ^.+.tga$, ^.+.tif$, ^.+.tiff$, ^.+.tim$, ^.+.vda$, ^.+.vrimg$, ^.+.vrmesh$, ^.+.vrsm$, ^.+.vrst$, ^.+.vst$, ^.+.wmf$, ^.+.ass$, ^.+.gz$, ^.+.ifd$, ^.+.mi$, ^.+.mi2$, ^.+.mxi$, ^.+.rib$, ^.+.rs$, ^.+.vrscene$
2021-10-15 13:16:36: 0: DEBUG: S3BackedCache Client Returned Sequence: 1
2021-10-15 13:16:36: 0: INFO: Executing global asset transfer preload script ‘/var/lib/Thinkbox/Deadline10/workers/ip-10-128-59-248/plugins/61697cbd70d2134be05ec02e/GlobalAssetTransferPreLoad.py’
2021-10-15 13:16:36: 0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer…
2021-10-15 13:16:36: 0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in /opt/Thinkbox/S3BackedCache/bin/task.py…
2021-10-15 13:16:36: 0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2021-10-15 13:16:36: 0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2021-10-15 13:16:36: 0: Done executing plugin command of type ‘Start Job’
2021-10-15 13:16:36: 0: Plugin rendering frame(s): 0
2021-10-15 13:16:37: 0: Executing plugin command of type ‘Render Task’
2021-10-15 13:16:37: 0: INFO: Sending StartTaskRequest to S3BackedCacheClient.
2021-10-15 13:16:37: 0: DEBUG: Request:
2021-10-15 13:16:37: 0: DEBUG: JobId: 61697cbd70d2134be05ec02e
2021-10-15 13:16:37: 0: DEBUG: JobUploadWhitelist: .exr
2021-10-15 13:16:37: 0: DEBUG: JobUploadWhitelistRe: ^.+.abc$, ^.+.avi$, ^.+.bmp$, ^.+.bw$, ^.+.cin$, ^.+.cjp$, ^.+.cjpg$, ^.+.cxr$, ^.+.dds$, ^.+.dpx$, ^.+.dwf$, ^.+.dwfx$, ^.+.dwg$, ^.+.dxf$, ^.+.dxx$, ^.+.eps$, ^.+.exr$, ^.+.fbx$, ^.+.fxr$, ^.+.hdr$, ^.+.icb$, ^.+.iff$, ^.+.iges$, ^.+.igs$, ^.+.int$, ^.+.inta$, ^.+.iris$, ^.+.jpe$, ^.+.jpeg$, ^.+.jpg$, ^.+.jp2$, ^.+.mcc$, ^.+.mcx$, ^.+.mov$, ^.+.mxi$, ^.+.pdf$, ^.+.pic$, ^.+.png$, ^.+.prt$, ^.+.ps$, ^.+.psd$, ^.+.rgb$, ^.+.rgba$, ^.+.rla$, ^.+.rpf$, ^.+.sat$, ^.+.sgi$, ^.+.stl$, ^.+.sxr$, ^.+.targa$, ^.+.tga$, ^.+.tif$, ^.+.tiff$, ^.+.tim$, ^.+.vda$, ^.+.vrimg$, ^.+.vrmesh$, ^.+.vrsm$, ^.+.vrst$, ^.+.vst$, ^.+.wmf$, ^.+.ass$, ^.+.gz$, ^.+.ifd$, ^.+.mi$, ^.+.mi2$, ^.+.mxi$, ^.+.rib$, ^.+.rs$, ^.+.vrscene$
2021-10-15 13:16:37: 0: DEBUG: S3BackedCache Client Returned Sequence: 1
2021-10-15 13:16:37: 0: INFO: Stdout Redirection Enabled: True
2021-10-15 13:16:37: 0: INFO: Asynchronous Stdout Enabled: False
2021-10-15 13:16:37: 0: INFO: Stdout Handling Enabled: True
2021-10-15 13:16:37: 0: INFO: Popup Handling Enabled: True
2021-10-15 13:16:37: 0: INFO: QT Popup Handling Enabled: False
2021-10-15 13:16:37: 0: INFO: WindowsForms10.Window.8.app.
Popup Handling Enabled: False
2021-10-15 13:16:37: 0: INFO: Using Process Tree: True
2021-10-15 13:16:37: 0: INFO: Hiding DOS Window: True
2021-10-15 13:16:37: 0: INFO: Creating New Console: False
2021-10-15 13:16:37: 0: INFO: Running as user: ec2-user
2021-10-15 13:16:37: 0: INFO: Executable: “/usr/autodesk/arnold/maya2022/bin/kick”
2021-10-15 13:16:37: 0: CheckPathMapping: Swapped "C:\Users**\Documents\maya\projects*__
/data/masterLayer/****.ass" with "/mnt/Data/CUsers
Documentsmayaprojects52e5ab43e28f0c6fea5f069465eb5d85/****__****/data/masterLayer/****.ass"
2021-10-15 13:16:37: 0: INFO: Sending EndTaskRequest to S3BackedCacheClient.
2021-10-15 13:16:37: 0: DEBUG: Request:
2021-10-15 13:16:37: 0: DEBUG: JobId: 61697cbd70d2134be05ec02e
2021-10-15 13:16:37: 0: Done executing plugin command of type ‘Render Task’

I replaced the file names here by ***** but any jobs/scenes submitted from maya get the same error.
I don’t find any answers or “clues” in order for me to resolve this problem.

It would be very appreciated if someone could help me or explain what this Input/output error means.

Thanks in advance,

You said you swapped the file paths, but are you really submitted scenes from local files?

2021-10-15 13:16:37: 0: CheckPathMapping: Swapped "C:\Users**\Documents\maya\projects*__/data/masterLayer/.ass" with "/mnt/Data/CUsers*Documentsmayaprojects52e5ab43e28f0c6fea5f069465eb5d85/__*****/data/masterLayer/****.ass"

This looks all kinds of wrong

Hi Anthony,
Yes it’s submitted from local files, sorry I made a typo when replacing file names by ***. It looks like this ;

2021-10-15 13:16:37: 0: CheckPathMapping: Swapped “C:\Users\USER\Documents\maya\projects\Project_name/data/masterLayer/Scene_name.ass” with “/mnt/Data/CUsersUSERDocumentsmayaprojects52e5ab43e28f0c6fea5f069465eb5d85/Project_name/data/masterLayer/Scene_name.ass”

Does that get transferred to the AMI and mounted like that?

Sorry, I’ve never used it this way so not seen it like that before.

Can you access the machine and see if

/mnt/Data/CUsersUSERDocumentsmayaprojects52e5ab43e28f0c6fea5f069465eb5d85/

Is actually mounted?

I’m not sure what you’re asking… The instances have been terminated since.
I’ll start a new infrastructure, launch a new spot fleet with the Arnold Standalone AMI and try access one of the instance via AWS management console to see if it’s actually mounted.

I can see the worker via AWS management console but can’t access it with WinScp to see if it’s mounted as it doesn’t show me an ipv4 adress to access to.

I still get the same error after the “CheckPathMapping: Swapped […]”