Collected bugs

[anon29114577]

I haven’t tried these out in beta 7 yet, but here’s some bugs I’ve noticed in the past week:
(This is still mono 2.6.7)

1. Underscores vanish in the output in the Slave GUI window.
2. Monitor hides jobs above the list window when “Notification Jobs” filter is on:
   Steps to reproduce: Have enough jobs listed that the scroll bar is active. Have some jobs that will notify you near the bottom of the list (requiring the scroll position at the bottom) as well as above. With that job selected, click the “Show Notification Jobs Only” button. Jobs above the selected one will now be shoved off the top of the list view, but the scroll bar will vanish meaning you can’t scroll up to them.
3. Nuke reports wrong progress when writing multiple outputs at once. i.e.: If the task size is set to 5, and there are two write nodes that are writing, after the first frame has been written, Deadline says the progress is at 40%, then the next set it says 80%, and then it’s at 100% for the last three frames.
4. I don’t know if this is related to the above, but on Windows, Nuke seems to render abnormally slow on windows once the progress gets stuck at 100%. A task that would take 3 minutes on a Linux box takes 22 minutes on windows.

[rrussell]

1. I was only able to reproduce this on an openSuse 11.0 64 bit machine running Mono 2.6.7. Every other operating system (including Fedora 13) showed the underscores just fine. That openSuse machine has a strange display (not Deadline specific) where everything like text, etc is really small. It might just be a display setting on the machine that I’m not aware of.

2. Which beta version are you on? I can’t reproduce this one, but it sounds like an issue that was resolved in a previous beta.

3. Yup, I can reproduce this. We’ll have to change our stdout handlers.

4. Can you check the task manager to see if it’s the slave or Nuke process eating up cpu cycles? Also, can you try rendering from the command line outside of Deadline (you can check the job log to get the command line arguments). That should help us figure out if this is a Deadline or Nuke problem.

Cheers,

• Ryan

[anon29114577]

Sorry for such the long delay in getting back!

Just upgraded to RC4, so here’s a status update:

1. Underscores still vanish in the slave GUI (Linux)
2. No longer happens!
3. Ditto (saw you fixed it in previous release notes)
4. Nuke was the one eating the CPU, and running via command line was faster. However, when submitted again, it was much faster than before (was taking ~4min a task). It may just be that one nuke submission, as it hasn’t been an issue since then.

Here’s some new ones:

1. The Monitor GUI becomes exceedingly slow and unresponsive when refreshing and dealing with a job that has a large number of tasks (>13,000 tasks). (Linux). We only have shots that long once in a blue moon (this one was about ~65,000 frames), but we do get them
2. The host/dns name for Windows machines has incorrect casing. (ie: listed as ‘ASSEMBLY01’ when it should be ‘assembly01’). Linux and Mac host/dns names are listed with correct casing.
3. We’re starting to experiment with 3Delight as our main renderer, and when testing it via Deadline, it fails on the Windows machines. It appears as though it’s losing all its environment setting (it doesn’t know where to find anything like display drivers, or even the system defined $DELIGHT env var.
   Here’s a log output:

[code]=======================================================
Error Message

Exception during render: An error occurred in RenderTasks(): ERROR S2050: cannot find shader ‘defaultsurface’, will use ‘null’
at Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)

=======================================================
Slave Log

0: Task timeout is disabled.
0: Loaded job: 3delightTest (999_050_999_675e149b)
0: INFO: StartJob: initializing script plugin RIB
0: INFO: Any stdout that matches the regular expression “ERROR.|error.|RIB error.|.Unable to find shader.|.Unable to find out device." will be handled as appropriate
0: INFO: Any stdout that matches the regular expression "R90000[[:space:]]([0-9]+)%” will be handled as appropriate
0: INFO: Any stdout that matches the regular expression “.* ([0-9]+.[0-9]+)%." will be handled as appropriate
0: INFO: Any stdout that matches the regular expression ". ([0-9]+)% ." will be handled as appropriate
0: INFO: Any stdout that matches the regular expression "[[:space:]]([0-9]+)%” will be handled as appropriate
0: INFO: About: RIB Plugin for Deadline
0: Plugin rendering frame(s): 2
0: INFO: Renderer: ThreeDelight
0: INFO: Stdout Handling Enabled: True
0: INFO: Popup Handling Enabled: False
0: INFO: Using Process Tree: True
0: INFO: Hiding DOS Window: True
0: INFO: Creating New Console: False
0: INFO: Render Executable: “C:\Program Files\3Delight\bin\renderdl.exe”
0: INFO: CheckPathMapping: Swapped “/axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib” with “//axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib”
0: INFO: Rendering file: //axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib
0: INFO: Render Argument: -progress -stats3 “//axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib”
0: INFO: Startup Directory: “C:\Program Files\3Delight\bin”
0: INFO: Process Priority: BelowNormal
0: INFO: Process is now running
0: STDOUT: 3DL INFO: $DELIGHT environment variable is not set
0: STDOUT: 3DL ERROR S2050: cannot find shader ‘defaultsurface’, will use ‘null’
0: WARNING: Another error occurred in regex function: ERROR P1029: cannot find display driver ‘dsm’
0: STDOUT: 3DL ERROR P1029: cannot find display driver ‘dsm’
0: WARNING: Another error occurred in regex function: ERROR P1029: cannot find display driver ‘tiff’
0: STDOUT: 3DL ERROR P1029: cannot find display driver ‘tiff’

=======================================================
Error Type

RenderPluginException

=======================================================
Error Stack Trace

at Deadline.Plugins.Plugin.RenderTask(Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.RenderCurrentTask(TaskLogWriter tlw)[/code]

4. When trying to submit a .rib file via monitor, the submission dialog hangs for an abnormally long time (minutes) after choosing the .rib file

[rrussell]

Thanks for the update!

Existing issues:

1. I’m not sure if/what we can do about this, because in the majority of cases, it works. We have a bug logged for this, and we’re currently targeting the next Deadline 5 point release (although it is currently a low priority bug).

New issues:

1. Yeah, this is a known issue, and the only workaround is to keep the task list at a manageable amount. This can be done by bumping up the Frames per Task, or submitting a few jobs that each have a part of the overall frame range. This issue should affect all operating systems (albeit to different degrees).

2. We found out why this is. This will be fixed in RC 5 (which will likely be the final release candidate).

3. Did you install 3delight on the slave(s) while Deadline was running? If so, you’ll have to restart all the Deadline apps (including the Launcher) so that it recognizes the new environment changes. Restarting the machine is definitely the best way to ensure everything is “refreshed”.

4. How many rib files are in the sequence? When Deadline is figuring out the frame range, it’s checking for each file’s existence, so I guess a large enough sequence could slow things down. We’ll have to run some more tests here.

Cheers,

• Ryan

[anon29114577]

All the (windows) machines had been restarted before submitting the 3Delight job, so the env should have been setup. The env does show correctly when listed via bash (cygwin).

There’s actually only one rib file we tested (multiple frames in one rib). It was a 24 frame sequence. I could upload the rib, but it along with supporting files ends up being ~150MB.

Cheers!
-Matt

[rrussell]

What happens if you open a command prompt (not a cygwin prompt) on windows and run this:

"C:\Program Files\3Delight\bin\renderdl.exe" -progress -stats3 "//axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib"

Do you get the same error? Is it possible that cygwin has its own set of env vars?

Uploading the rib file itself should be fine. The supporting files aren’t necessary since we’re just testing the submitter, not any rendering.

Thanks!

• Ryan

[anon29114577]

That works fine on the standard windows cmd.

Cygwin does augment the windows env, but in this case all the 3Delight vars have been defined in the windows system env itself, and not in cygwin.

Attached is the rib file.

Cheers!
-Matt

[rrussell]

Thanks! We discovered that the submitter was going through the rib file line by line looking for information that it didn’t even use. Removing that part fixed the delay problem.

I installed 3Delight to test the env var problem, and I was actually able to reproduce it the first time (even after restarting all the Deadline apps). I checked my env vars in My Computer, and DELIGHT was there. So I opened a command prompt and ran a test and got the same error! In My Computer, I chose to edit the DELIGHT var, but then just pressed OK without changing anything. This time, my command prompt was able to see it, and after a restart of all the Deadline apps, so was Deadline.

So for whatever reason, the 3Delight installer wasn’t setting DELIGHT properly. I have no idea why the env var would get messed up this way, or if this is even related to the problem you’re seeing. But I was able to confirm that as long as the environment was setup properly, I could render 3Delight jobs with Deadline.

Cheers,

• Ryan

[anon29114577]

Getting closer!

Now I get a different error. It looks like it can’t open the exr output driver (the path it writes to does exist, so that shouldn’t be the issue):

[code]=======================================================
Error Message

Exception during render: An error occurred in RenderTasks(): ERROR D2004: failed to open display driver ‘exr’ (display driver returned ‘4’)
at Deadline.Plugins.ScriptPlugin.RenderTasks(Int32 startFrame, Int32 endFrame, String& outMessage)

=======================================================
Slave Log

0: Task timeout is disabled.
0: Loaded job: ribtest (999_050_999_7bc2f161)
0: INFO: StartJob: initializing script plugin RIB
0: INFO: Any stdout that matches the regular expression “ERROR.|error.|RIB error.|.Unable to find shader.|.Unable to find out device." will be handled as appropriate
0: INFO: Any stdout that matches the regular expression "R90000[[:space:]]([0-9]+)%” will be handled as appropriate
0: INFO: Any stdout that matches the regular expression “.* ([0-9]+.[0-9]+)%." will be handled as appropriate
0: INFO: Any stdout that matches the regular expression ". ([0-9]+)% ." will be handled as appropriate
0: INFO: Any stdout that matches the regular expression "[[:space:]]([0-9]+)%” will be handled as appropriate
0: INFO: About: RIB Plugin for Deadline
0: Plugin rendering frame(s): 5
0: INFO: Renderer: ThreeDelight
0: INFO: Stdout Handling Enabled: True
0: INFO: Popup Handling Enabled: False
0: INFO: Using Process Tree: True
0: INFO: Hiding DOS Window: True
0: INFO: Creating New Console: False
0: INFO: Render Executable: “C:\Program Files\3Delight\bin\renderdl.exe”
0: INFO: CheckPathMapping: Swapped “/axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib” with “//axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib”
0: INFO: Rendering file: //axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib
0: INFO: Render Argument: -progress -stats3 “//axel/projects/Good_Luck_Charlie/Episode_216/GLC_EP216_3D/3delight/evilvortex_v01/rib/evilvortex_v01_prodPass.rib”
0: INFO: Startup Directory: “C:\Program Files\3Delight\bin”
0: INFO: Process Priority: BelowNormal
0: INFO: Process is now running
0: STDOUT: 3DL WARNING D2045: PixelFilter reset to ‘box’ 1x1 (display driver requirement)
0: STDOUT: Statistics for frame 1
0: STDOUT: * Stats for group Task Profiling :
0: STDOUT:
0: STDOUT: task user kernel real calls
0: STDOUT: ReadArchive 0.20 0.09 0.48 2
0: STDOUT: REYES 2.36 0.14 0.46 1
0: STDOUT: Total : 2.56 0.23 0.94 3
0: STDOUT: * Stats for group REYES (Statistics concerning REYES rendering module) :
0: STDOUT: Total number of u-polygons : 966440
0: STDOUT: Average number of u-polygons per grid : 180.104
0: STDOUT:
0: STDOUT: projected grids : 5366
0: STDOUT: shaded grids : 3271
0: STDOUT: displaced grids : 0
0: STDOUT:
0: STDOUT: total primitives : 1
0: STDOUT: efficiently rejected prims : 1007
0: STDOUT:
0: STDOUT: Number of non-empty buckets : 556
0: STDOUT: Average # of objects per bucket : 460.534
0: STDOUT: Maximum # of objects in one bucket : 1939
0: STDOUT:
0: STDOUT: Total trimming operations : 0
0: STDOUT:
0: STDOUT: Maximum u-polygon area : 1.10805
0: STDOUT: Minimum u-polygon area : 0.890228
0: STDOUT: Average u-polygon area : 1.0231
0: STDOUT:
0: STDOUT: Distribution of u-polygon area
0: STDOUT: ±----±----±----±----±----±----±----±----±----±----±----±----±----±----±----+
0: STDOUT: |<=.06|<=.09|<=.12|<=.18|<=.25|<=.35|<=.50|<=.71|<= 1|<=1.4|<= 2|<=2.8|<= 4|<=5.7|>5.7 |
0: STDOUT: ±----±----±----±----±----±----±----±----±----±----±----±----±----±----±----+
0: STDOUT: ±----±----±----±----±----±----±----±----±----±----±----±----±----±----±----+
0: STDOUT:
0: STDOUT: Distribution of fragment list lengths
0: STDOUT: ±----±----±----±----±----±----±----±----±----+
0: STDOUT: |<= 1|<= 2|<= 3|<= 4|<= 5|<= 6|<= 7|<= 8|> 8 |
0: STDOUT: ±----±----±----±----±----±----±----±----±----+
0: STDOUT: ±----±----±----±----±----±----±----±----±----+
0: STDOUT:
0: STDOUT: Curves grouping efficiency : 0.00 %
0: STDOUT: * Stats for group Map Manager (statistics about map resources) :
0: STDOUT: Total number of maps used : 0
0: STDOUT: Total number of texture maps : 0
0: STDOUT: Total number of shadow maps : 0
0: STDOUT: Total number of environment maps : 0
0: STDOUT: Total number of deep shadow maps : 0
0: STDOUT: Total number of ‘invalid’ maps : 0
0: STDOUT: Total number of map requests : 0
0: STDOUT: Texture IDs
0: STDOUT: Texture stats
0: STDOUT: ID Refs Same Search Faults Rate Mipmap (min) Mipmap (max)
0: STDOUT: * Stats for group Scene Manager (Statistics about the scene) :
0: STDOUT: Number of objects in the scene : 1
0: STDOUT: Number of attributes in the scene : 1
0: STDOUT: Number of transforms in the scene : 1
0: STDOUT: * Stats for group Memory Profiling :
0: STDOUT:
0: STDOUT: item current peak overall peak
0: STDOUT: unprocessed gprim 2646 K 11 M 10 M
0: STDOUT: gprim 0 128 K 102 K
0: STDOUT: gprim (polygons) 0 457 K 418 K
0: STDOUT: gprim (subdiv) 0 0 0
0: STDOUT: gprim (curves) 0 0 0
0: STDOUT: gprim variables 0 988 K 963 K
0: STDOUT: particles 0 0 0
0: STDOUT: topology data 0 0 0
0: STDOUT: grid 0 1990 K 1967 K
0: STDOUT: shaded grid 0 480 K 295 K
0: STDOUT: grid AOV 0 1548 K 1547 K
0: STDOUT: texture objects 0 0 0
0: STDOUT: texture cache 0 0 0
0: STDOUT: hider and visible points 0 422 K 385 K
0: STDOUT: image data 0 304 K 295 K
0: STDOUT: buffered display data 0 0 0
0: STDOUT: shader code 1205 1205 1205
0: STDOUT: shader instance 1980 1980 1980
0: STDOUT: shader variables 0 51 K 18 K
0: STDOUT: attributes 256 527 527
0: STDOUT: transform 1332 1480 1480
0: STDOUT: subsurface 0 0 0
0: STDOUT: point based occlusion 0 0 0
0: STDOUT: point cloud 0 0 0
0: STDOUT: photon map 0 0 0
0: STDOUT: inline archive 406 406 406
0: STDOUT: misc 0 663 K 620 K
0: STDOUT: Total : 2651 K 18 M 17 M
0: STDOUT:
0: STDOUT: Peak VM size : 41640 K
0: STDOUT: Peak resident : 44732 K
0: STDOUT: Peak from malloc : 0 K
0: STDOUT: 3DL ERROR D2004: failed to open display driver ‘exr’ (display driver returned ‘4’)
0: WARNING: Another error occurred in regex function: ERROR D2094: failed to open display driver ‘exr’ (display driver returned with a null handle)
0: STDOUT: 3DL ERROR D2094: failed to open display driver ‘exr’ (display driver returned with a null handle)

=======================================================
Error Type

RenderPluginException

=======================================================
Error Stack Trace

at Deadline.Plugins.Plugin.RenderTask(Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.RenderCurrentTask(TaskLogWriter tlw)
[/code]

[rrussell]

Progress is good!

Googling that error turned up some hits:
3delight.com/en/modules/PunB … php?id=522
3delight.com/en/modules/PunB … hp?id=1142
3delight.com/en/modules/PunB … php?id=496

Looks like it could be due to a 32/64 bit issue, and seems to be completely unrelated to Deadline. Hopefully your answer lies in one of those threads!

Cheers,

• Ryan

[anon29114577]

Heh, I’ve actually been reading those exact threads

I’m not sure it is a 32/64 issue, as we only have the 64bit version of 3Delight installed, as well as just the 64bit exr library posted on their wiki. It works if I run it bare from the command line, so I’m still kinda scratching my head. I should have said before, but this is Windows 7.

[rrussell]

That’s strange that it works from the command line. I did some more searching, and found this:
3delight.com/en/modules/PunB … hp?id=1973

As suggested in the final post, I checked out the first paragraph of the openexr display driver in the 3delight manual, and it says this:

This display driver writes out ILM’s EXR files using the `half’ floating point format. Floating point input is requested for this display driver, so the RIB should contain the following lines before the display driver is invoqued:

Quantize “rgba” 0 0 0 0

I see “quantize” lines in your rib file, but the way things read, they seem to be specified AFTER the Display line for each frame. I’m no rib-syntax expert though.

Cheers,

• Ryan

[anon29114577]

I appear to have fixed it!

The Quantize wasn’t the issue, it’s an env one.

I solved this by running everything in sh on windows machines, which sets up a proper env for things to run in. This does add the dependency of need sh, but I think that could probably be included by itself? (I know this is what Rush does - Rush runs everything in sh (or is it tcsh or bash? I can’t remember, it’s quite been a while since I’ve used it), so it’s included in the binary distribution for windows.) Right now I’m running sh right from the cygwin install, but I’m wondering if sh could just be copied to the deadline bin directory? I don’t know if the cygwin version of sh is dependant on any external libraries.

I did break the ability to have spaces in paths and filenames though. However, our naming conventions here don’t allow for spaces, so for us it isn’t an issue.

I’m attaching my modified RIB plugin .py file so you can see my changes.

Cheers!
-Matt
RIB.py.tar.gz (4.39 KB)

[rrussell]

That’s really interesting. I’m just glad to hear you were able to get it working!

Cheers,

• Ryan